Skip to content
Substrata
Join

Is any of this out of date?

Late · How this is computedA feed is fresh within one scheduled interval plus half an interval (at least 20 minutes) of its last good run, late up to three intervals, stale beyond; failing when its newest failure is newer than its last good run.Method →Code → late, How this is computedA feed is fresh within one scheduled interval plus half an interval (at least 20 minutes) of its last good run, late up to three intervals, stale beyond; failing when its newest failure is newer than its last good run.Method →Code → fresh, How this is computedA feed is fresh within one scheduled interval plus half an interval (at least 20 minutes) of its last good run, late up to three intervals, stale beyond; failing when its newest failure is newer than its last good run.Method →Code → on demand · measured 12:28 UTC

Each state is computed from the run tables and the dates the committed files record, by one published rule. A committed file past its allowed age fails the build, so it cannot be deployed as current. Machine-readable: /api/health/freshness.

Needs attention: Review queue

01Scheduled feeds

What the server fetches by itself
FeedStatusLast good runExpectedLast failureRuns this week
Research sweepWeb search per bottleneck for news that may tighten or loosen it; finds leads, publishes nothing.Leads on the desk and in /review →FreshHow this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →every 6 hnone on record28
Event draftsReads sweep leads and drafts an event from each, quote checked against the page — on a reader’s own AI key, never the free AI.Unreviewed drafts in /review →On demandRuns when a reader presses “Summarise with AI”, and hourly only for readers who switched on automatic updates — each time on their own key.How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →on demand · readers’ own keys0
SEC filingsNew 8-K, 6-K, 10-Q, 10-K and 20-F filings from SEC EDGAR for every listed holder.Filings on the desk and company pages →FreshHow this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →hourly168
Science feedsPapers, preprints and grants from OpenAlex, arXiv, NSF, OpenAIRE and USAspending.Science pipeline →FreshHow this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →hourlynone on record170
Job boardsOpen roles from the public Greenhouse, Lever and Ashby boards of directory companies.Careers →FreshHow this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →dailynone on record7
Official statisticsPrice and output series from the BLS public API.Data series →FreshHow this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →dailynone on record7
Producer sourcingSearches for a primary source for each unsourced producer row; candidates wait for review.Unchecked sources on bottleneck pages and in /review →FreshHow this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →every 6 hnone on record28
Data-quality checksRe-opens a rotating slice of source links, quotes, tickers (SEC, OpenFIGI) and USGS table rows, and scores every dataset. No AI.Data quality →FreshHow this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →every 6 hnone on record27

02Review queue

Open leads: found, not yet read, not expired
Late

How this is computedWaiting: open sweep leads — no reviewer decision, not expired. Oldest: whole days since the earliest open one was found. Drafts ready: waiting leads whose AI draft suggests an event and passed the checks.Method →Code → leads waiting; the oldest was found How this is computedWaiting: open sweep leads — no reviewer decision, not expired. Oldest: whole days since the earliest open one was found. Drafts ready: waiting leads whose AI draft suggests an event and passed the checks.Method →Code → ago. Leads should be read within 7 days; one nobody reviews within 30 days expires and leaves the queue, so a stale queue means the expiry itself is broken. Open the review inbox →

How this is computedA sweep lead is open while nobody has reviewed it and it was found less than 30 days ago; unreviewed and 30 days or older, it is expired. The review queue is open leads only: late when its oldest has waited more than 7 days, stale past 31.Method →Code → leads expired, never reviewed — kept, not deleted. See the expired leads →

03Committed datasets

Checked by people and scripts
DatasetStatusChecked onAgeAllowedHow it is refreshed
Tickers and listingsresearch/listings.jsonFresh2026-09-24How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →30 dpnpm run research:listings -- --revalidate
Series read from sources (oldest quote check)research/series.jsonFresh2026-09-24How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →30 dRe-read each source and match the quote: pnpm tsx scripts/research/check-series.ts
Claims checked against seriesresearch/claims.jsonFresh2026-09-24How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →30 dRe-check each claim against its series and update checkedOn
Which board each company hires onresearch/job-boards.jsonFresh2026-09-24How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →30 dRe-check each careers page and update checkedOn
Binding assessments (oldest judgement)config/substrata-assessment.tsFresh2026-09-15How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →90 dRe-judge each bottleneck and update its judgedOn
Training and learning paths (oldest check)research/learning-paths.jsonFresh2026-09-24How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →90 dRe-open each provider page and update checkedOn
Occupations (O*NET, ESCO)research/occupations.jsonFresh2026-09-24How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →365 dpnpm run research:occupations (O*NET publishes yearly)
World production and reserves (USGS MCS)research/usgs-mcs.jsonFresh2026-09-25How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →400 dpnpm run research:usgs (new edition each February)
Oil, gas, coal and hydro by country (EIA)research/eia-energy.jsonFresh2026-09-25How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →120 dpnpm run research:eia
Producers per country (USGS Minerals Yearbook)research/usgs-producers.jsonFresh2026-09-26How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →400 dpnpm run research:producers (new country chapters through the year)
Producer sites on the globe (USGS rows placed with GeoNames)research/producer-places.jsonFresh2026-10-12How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →400 dpnpm run research:producer-places (after research:producers)
Export restrictions (OECD inventory)research/oecd-export-restrictions.jsonFresh2026-09-25How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →400 dpnpm run research:restrictions (OECD adds a data year annually)
Sanctions regimes (EU Sanctions Map, OFAC)research/sanctions.jsonFresh2026-09-26How this is computedNow minus the finish time of the last good run (feeds), or whole UTC days since the date the committed file records it was checked (datasets).Method →Code →60 dpnpm run research:sanctions