Day 5 of 10 · Harbour Mutual

Constraints and indexes

Make the database refuse bad data: primary and foreign keys, NOT NULL, CHECK and UNIQUE constraints, and indexes that make the everyday joins fast.

Theory for today: Fraud and data quality, Integrity and normalisation.

The lesson

Every report trusts the data. Constraints are rules the database enforces on every insert and update, so data that breaks them never gets in (see Fraud and data quality).

The constraints you already have

  • A primary key (policy_id SERIAL PRIMARY KEY) identifies each row: unique and never null.
  • A foreign key (customer_id INT REFERENCES customers) must point at a row that exists: no policy for a customer who does not.
  • NOT NULL makes a value compulsory.
SELECT conname, pg_get_constraintdef(oid)
FROM pg_constraint
WHERE conrelid = 'policies'::regclass;
Every rule on policies, as PostgreSQL stores it. Try it in the playground

CHECK and UNIQUE

A check constraint is a condition every row must meet; a unique constraint refuses a second row with the same value. Add them to an existing table with ALTER TABLE; PostgreSQL first checks the rows already there, and refuses the constraint if any break it.

ALTER TABLE brokers
    ADD CONSTRAINT brokers_commission_sensible CHECK (commission_rate BETWEEN 0 AND 0.5);

-- Now this is refused:
UPDATE brokers SET commission_rate = 0.9 WHERE broker_id = 1;
Name your constraints: the name appears in the error, and lets you drop them. Try it in the playground

To run such a script again, drop the constraint first: ALTER TABLE brokers DROP CONSTRAINT IF EXISTS brokers_commission_sensible;.

Indexes

An index is like the index of a book: instead of reading every row, PostgreSQL looks the value up. Primary keys and unique constraints get one automatically; foreign keys do not, so the joins you run every day may read whole tables.

CREATE INDEX IF NOT EXISTS claim_payments_claim_id ON claim_payments (claim_id);

SELECT indexname, indexdef
FROM pg_indexes
WHERE tablename = 'claim_payments';
Indexes speed up reading, and cost a little on every write: add the ones your queries need. Day 8 measures the difference. Try it in the playground
Check the data before you add a rule, then let the rule keep it that way.

Briefs from the business

0 of 6 done

Write SQL to answer each brief, run it, then check your answer against Harbour Mutual's. Your database is the same one as in the playground.

Starting PostgreSQL in your browser…

Brief 5.1 · Check before you enforce

Before making policy numbers unique, the data team wants proof there are no duplicates today. Show two numbers: how many rows policies has, and how many distinct policy numbers.

Tables: policies

Brief 5.2 · Cover must end after it starts

Make the database refuse any policy whose end date is before its start date. Policies of any normal length must still be accepted.

Tables: policies

Brief 5.3 · No negative premiums

Make the database refuse a policy with a negative annual premium. A premium of zero, or more, is allowed.

Tables: policies

Brief 5.4 · One number, one policy

Make policy numbers unique: a second policy with a number that already exists must be refused.

Tables: policies

Brief 5.5 · Losses are reported after they happen

Make the database refuse a claim reported before its loss happened. A claim reported on the day of the loss, or later, is fine.

Tables: claims

Brief 5.6 · Index the claims team's join

Every morning the claims team joins claims to their policies. Add an index on the column of claims that the join uses.

Tables: claims

An unhandled error has occurred. Reload 🗙

Reconnecting…

Your connection to Data Analytics SQL Lab dropped, probably a Wi-Fi blip. Hold on while it reconnects. Your work is kept.

The connection could not be restored. The page will reload in a few seconds.