Methodology
Where every number comes from.
This page names every source, walks through the extraction pipeline, and states the guarantees on every record. If a number fails a spot check, the corrections policy at the bottom states what happens next.
Every record starts at a regulator.
We collect from state departments of insurance and federal publications only. Nothing is licensed from data vendors, nothing is inferred from private feeds, and nothing enters the database without a source URL.
| Source | What we take | Scope |
|---|---|---|
| SERFF Filing Access (NAIC) filingaccess.serff.com |
Rate and rule filing metadata: carrier, SERFF tracking number, filing type, status, submission and disposition dates, document lists. | 49 jurisdictions (48 states + DC) |
| Florida OIR IRFS irfssearch.floir.gov |
Florida filing metadata and detail. Florida does not participate in SERFF Filing Access, so it gets its own client. | FL |
| Louisiana DOI Rate Filing Search ldi.la.gov roadmap |
Louisiana P&C filings. Louisiana's SERFF instance exposes health filings only, so homeowners metadata requires this custom path. Until that build ships, Louisiana is excluded from the sweep: the one gap in the 51-jurisdiction coverage. | LA |
| California DOI weekly public notices insurance.ca.gov |
The CDI's weekly published list of rate applications: requested rate change percentages and prior approvals, straight from the regulator's own spreadsheet. | CA |
| California FAIR Plan cfpnet.com |
Insurer-of-last-resort enrollment statistics, refreshed as published. | CA |
| US Senate Budget Committee | County-level homeowners non-renewal dataset from the committee's 2024 climate-insurance investigation. | National, 2018–2023 |
| US Treasury, Federal Insurance Office | ZIP-level homeowners premium and claims metrics from the FIO's national climate data call. | National, 2018–2022 |
Filing documents themselves (actuarial memoranda, rate applications) are fetched from the same portals, per filing, when the extraction pipeline needs them.
Deterministic first. A language model only where templates end. Evidence always.
Rate filings mix structured forms with free-form actuarial prose, so the pipeline runs in ordered passes:
1. Deterministic parsers run first
Where a state uses a standardized form, we parse it directly with no model in the loop. California's rate-application template, for example, yields requested rate changes, policy counts, and effective dates as plain cell reads. Deterministic values are the highest-trust tier in the merge.
2. The LLM pass is evidence-bound
Fields the templates cannot reach (indicated rates buried in actuarial memoranda, territory detail, count bases) go to a language model under a strict contract: one PDF per call, structured output enforced by the schema, and for every non-null value the model must cite the 1-indexed PDF page number and a verbatim snippet supporting it. A value without evidence does not enter the database.
3. Confidence grading and merge rules
Every extracted field carries a confidence grade: high, medium, or low. When multiple passes disagree, the higher-confidence non-null value wins, and nulls never overwrite. Requested, indicated, and approved rate changes are kept as distinct fields because conflating them is the most common error in this domain.
4. Golden-set testing in CI
We maintain a golden set of hand-verified filings per state, checked value by value against the source PDFs. The extraction suite runs against it in continuous integration; changes to the pipeline do not ship unless the golden set passes. Human spot-check sampling of fresh extractions continues on top of that.
The provenance guarantee.
Every record served by the API carries a provenance object: the source_url pointing at the regulator's own system, and retrieved_at, the timestamp we fetched it. Not at the response level. On each record.
"provenance": { "source_url": "https://filingaccess.serff.com/sfa/search/filingSummary.xhtml?filingId=...", "retrieved_at": "2026-07-08T04:08:36Z" }
Follow the URL, open the filing, and check any value against the regulator's copy.
How we treat public infrastructure.
State filing portals are a public good run on public budgets. Collection runs inside fixed constraints:
- Rate-limited. Requests are spaced seconds apart, enforced in code, not in configuration someone can forget.
- Incremental. Daily sweeps use a rolling lookback window with idempotent upserts. We fetch what changed; we do not re-download archives.
- Browser-equivalent access. Pages are fetched the way a person at a browser would fetch them, through the portals' public search interfaces, with no circumvention of any access control.
- Stop-first. If a portal signals distress or a regulator objects, the sweep for that state stops before any discussion starts.
Corrections policy.
A wrong rate number is worse than a missing one. If you find an error in any ratefeed record:
- Email hello@ratefeed.io with the record and what you believe is wrong.
- We verify against the source document and fix confirmed errors within 24 hours.
- Every confirmed correction is logged below, permanently. We do not silently rewrite history.
Corrections log
No corrections filed to date (log opened 2026-07-08). This line changes the day the first one lands.