Data Lake Freshness

Stable operator surface for raw source landing freshness, recent provider activity, curated dataset publication freshness, SQL-backed dataset bridge status, and next actions before expanding pulls or model dependencies.

Run data_lake_freshness_report_20260717T222508Z | report date 2026-07-17 | generated UTC 2026-07-17T22:25:15.246445+00:00

Executive Summary

Sources tracked38
Curated datasets47
Raw assets1,398,556
Raw bytes169.52 GB
Curated assets1,315,635
Curated rows187,971,617
Curated bytes42.26 GB

Interpretation: Freshness here is an operating signal, not a provider call trigger. Check rate limits, request cache, refresh plans, and source-specific cadence before starting backfills.

Source Freshness Counts

aging_30d22
current_7d5
fresh_24h7
no_raw_assets1
retired1
stale_30d2

Curated Dataset Freshness Counts

current_14d14
fresh_3d31
sql_backed_current2

Source Freshness

Source Status Raw Assets Raw Size Latest Raw Raw 24h Raw 7d Requests 30d Remote 30d Cache 30d Latest Request SQL Backing Operator Action
yfinance
yfinance Yahoo market data adapter
no_raw_assets
candidate / api
00 B
hours ago
00000
Candidate/source metadata exists but no raw data has landed yet; run a smoke or decide whether to activate.
oecd_tiva
OECD Trade in Value Added
stale_30d
active / api
4527.69 KB2026-06-12 05:21:16.098437+00:00
857.1 hours ago
004312026-07-16 03:02:39.031226+00:00
failed 403
Latest landed raw asset is older than 30 days; review whether this source should refresh or be marked low-frequency.
local_desktop_raw_api
Legacy Desktop raw API data
stale_30d
active / file
573,92522.98 GB2026-06-01 23:52:17.700160+00:00
1,102.5 hours ago
00000
Latest landed raw asset is older than 30 days; review whether this source should refresh or be marked low-frequency.
openfigi
OpenFIGI API
aging_30d
active / api
6711.36 MB2026-07-09 04:19:27.865463+00:00
210.1 hours ago
007361122026-07-09 04:19:27.456798+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
gleif
GLEIF API
aging_30d
active / api
6,23156.66 MB2026-07-09 03:48:09.047269+00:00
210.6 hours ago
008,0613,4424,6192026-07-09 03:48:08.073982+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
github_congress_trades
Public congressional trades feed
aging_30d
active / website
1561.85 MB2026-07-07 13:13:16.906394+00:00
249.2 hours ago
0014862026-07-08 09:15:52.299169+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
ken_french_data_library
Kenneth R. French Data Library
aging_30d
active / website
6809.71 KB2026-07-03 20:49:20.671158+00:00
337.6 hours ago
00000
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
gdelt_doc_relationships
GDELT DOC Relationship Graph Adapter
aging_30d
active / api
39.13 KB2026-07-03 10:41:16.173915+00:00
347.7 hours ago
006602026-07-03 10:43:14.512409+00:00
rate_limited 429
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
openfec
OpenFEC API
aging_30d
candidate / api
322.08 KB2026-07-02 07:36:36.935758+00:00
374.8 hours ago
00141042026-07-02 07:36:36.472755+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
regulations_gov
Regulations.gov API
aging_30d
candidate / api
5350.59 KB2026-07-02 07:36:36.453804+00:00
374.8 hours ago
00141042026-07-02 07:36:35.995090+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
congress_gov
Congress.gov API
aging_30d
candidate / api
420.57 KB2026-07-02 07:36:35.976670+00:00
374.8 hours ago
00161152026-07-02 07:36:35.675097+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
eia
U.S. Energy Information Administration API
aging_30d
active / api
20117.02 MB2026-07-02 07:33:15.670844+00:00
374.9 hours ago
003161791372026-07-02 07:33:14.927985+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
treasury_fiscal_data
U.S. Treasury Fiscal Data API
aging_30d
active / api
4613.71 MB2026-07-02 07:27:30.924923+00:00
375.0 hours ago
009061292026-07-02 07:27:33.000717+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
bls
U.S. Bureau of Labor Statistics Public Data API
aging_30d
active / api
163.16 MB2026-07-02 07:26:04.905110+00:00
375.0 hours ago
00211652026-07-02 07:26:02.536982+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
fred_alfred
FRED / ALFRED
aging_30d
active / api
29693.91 MB2026-07-02 07:25:50.319911+00:00
375.0 hours ago
001,0774965812026-07-02 07:26:01.490362+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
wikidata_query_service
Wikidata Query Service
aging_30d
active / api
3267.80 KB2026-07-01 10:23:25.351473+00:00
396.0 hours ago
004014262026-07-13 06:49:43.641183+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
census_eits
U.S. Census Economic Indicators Time Series API
aging_30d
active / api
41312.47 MB2026-06-30 07:55:13.693626+00:00
422.5 hours ago
001,9166791,2372026-07-02 07:30:11.031572+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
polygon
Polygon
aging_30d
candidate / api
112.87 KB2026-06-22 12:30:49.187659+00:00
609.9 hours ago
00131122026-06-22 12:30:48.931740+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
marketdata_app
MarketData.app
aging_30d
candidate / api
2401 B2026-06-22 11:32:50.570552+00:00
610.9 hours ago
00131122026-06-22 12:30:48.551097+00:00
succeeded 203
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
gdelt_doc
GDELT DOC 2.0
aging_30d
candidate / api
1102 B2026-06-21 11:21:41.389524+00:00
635.1 hours ago
002202026-06-21 16:57:05.350515+00:00
rate_limited 429
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
tiingo
Tiingo
aging_30d
candidate / api
1265 B2026-06-21 10:15:58.135656+00:00
636.1 hours ago
0013852026-06-22 12:30:49.202158+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
eodhd
EODHD
aging_30d
candidate / api
1184 B2026-06-21 10:15:57.057896+00:00
636.1 hours ago
00131122026-06-22 12:30:47.312774+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
bea
Bureau of Economic Analysis API
aging_30d
active / api
24497.16 MB2026-06-21 04:25:20.633872+00:00
642.0 hours ago
004125162026-06-22 12:30:34.769773+00:00
succeeded 200
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
sec_statement_data_sets
SEC Financial Statement Data Sets
aging_30d
active / website
695.20 GB2026-06-21 02:54:12.585218+00:00
643.5 hours ago
00000
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
sec_statement_notes_data_sets
SEC Financial Statement and Notes Data Sets
aging_30d
active / website
7925.28 GB2026-06-20 07:30:16.845240+00:00
662.9 hours ago
00000
Latest landed raw asset is older than 7 days; verify this is expected for the source cadence.
alpaca
Alpaca Market Data
fresh_24h
active / api
15,5885.46 GB2026-07-17 21:20:37.164796+00:00
1.1 hours ago
1,2159,44115,88715,88702026-07-17 21:20:37.169452+00:00
succeeded 200
No immediate freshness action required.
un_comtrade
UN Comtrade
fresh_24h
active / api
2,87211.55 MB2026-07-17 21:06:25.977359+00:00
1.3 hours ago
3771,1955,2293,2981,9312026-07-17 21:06:23.659597+00:00
succeeded 200
No immediate freshness action required.
benzinga
Benzinga
fresh_24h
active / api
8812.03 MB2026-07-17 20:25:45.070107+00:00
2.0 hours ago
4192,9402,94002026-07-17 20:29:38.006097+00:00
succeeded 200
No immediate freshness action required.
sec_edgar
SEC EDGAR
fresh_24h
active / api
23,97077.07 GB2026-07-17 16:20:15.729447+00:00
6.1 hours ago
12846,3386,261772026-07-17 16:20:15.606431+00:00
succeeded 200
No immediate freshness action required.
fmp
Financial Modeling Prep
fresh_24h
active / api
773,94130.13 GB2026-07-17 06:42:18.195150+00:00
15.7 hours ago
1092,7731,005,662961,46744,1952026-07-17 06:42:23.638119+00:00
succeeded 200
Recent rate limits occurred; inspect rate policy and request cache before increasing worker volume.
alpha_vantage
Alpha Vantage
fresh_24h
active / api
661.80 MB2026-07-17 06:42:03.874293+00:00
15.7 hours ago
32611264482026-07-17 06:41:46.589099+00:00
succeeded 200
No immediate freshness action required.
cftc_public_reporting
CFTC Public Reporting
fresh_24h
active / api
922.61 GB2026-07-17 06:41:31.937954+00:00
15.7 hours ago
416192921002026-07-17 06:41:31.292361+00:00
succeeded 200
No immediate freshness action required.
newsdata
NewsData.io
current_7d
active / api
2061.97 MB2026-07-16 20:46:21.586084+00:00
25.6 hours ago
042494434602026-07-16 20:46:21.068488+00:00
succeeded 200
No immediate freshness action required.
newsapi
NewsAPI
current_7d
active / api
32433.15 KB2026-07-16 20:45:55.467320+00:00
25.6 hours ago
03494434602026-07-16 20:46:16.754689+00:00
succeeded 200
No immediate freshness action required.
databento
Databento Historical Market Data
current_7d
active / api
9618.58 MB2026-07-14 04:54:56.870163+00:00
89.5 hours ago
08223991242026-07-14 04:54:52.594529+00:00
succeeded 200
No immediate freshness action required.
openalex
OpenAlex API
current_7d
active / api
120344.97 KB2026-07-13 06:50:04.627312+00:00
111.6 hours ago
010888808082026-07-13 06:50:05.041092+00:00
succeeded 200
No immediate freshness action required.
federal_register
Federal Register API
current_7d
active / api
182.62 MB2026-07-13 06:49:54.150590+00:00
111.6 hours ago
018020602026-07-13 06:49:53.811118+00:00
succeeded 200
No immediate freshness action required.
local_desktop_legacy
Legacy Desktop structured data
retired
retired / file
00 B
hours ago
00000
Retired source; keep only for lineage unless reactivated.

Curated Dataset Freshness

Dataset Status Assets Scopes Rows Size Latest Published Latest Source Raw Sources Operator Action
price_history
fresh_3d
89,14789,147126,414,1503.98 GB2026-07-17 01:47:00.310840+00:00
20.6 hours ago
2026-07-17 01:28:41.826795+00:00alpaca, fmp, local_desktop_raw_apiNo immediate curated freshness action required.
fmp_eod_bulk
fresh_3d
1212736,98524.21 MB2026-07-17 01:28:42.521724+00:00
20.9 hours ago
2026-07-17 01:28:41.826795+00:00fmpNo immediate curated freshness action required.
etf_holder
fresh_3d
13,27313,2737,150,228488.05 MB2026-07-16 07:27:10.600896+00:00
39.0 hours ago
2026-07-16 07:27:10.600896+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
dcf
fresh_3d
33,73533,73533,736231.05 MB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
key_metrics_ttm
fresh_3d
71,49771,49771,4992.23 GB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
peers
fresh_3d
82,83382,83382,833503.50 MB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
price_target_summary
fresh_3d
5,1975,1975,19756.81 MB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
profile
fresh_3d
90,36490,36490,3642.38 GB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmp, local_desktop_raw_apiCurrent-asset table includes non-current statuses; inspect publication status handling.
rating
fresh_3d
51,29851,29851,298511.12 MB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
ratios_ttm
fresh_3d
71,50171,50171,5033.05 GB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
scores
fresh_3d
62,19162,19162,192680.19 MB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
upgrades_downgrades_consensus
fresh_3d
13,28813,28813,289113.07 MB2026-07-16 07:24:32.909994+00:00
39.0 hours ago
2026-07-16 07:24:32.909994+00:00fmpCurrent-asset table includes non-current statuses; inspect publication status handling.
fmp_upgrades_downgrades_consensus_bulk
fresh_3d
1113,279132.23 KB2026-07-16 04:40:39.652985+00:00
41.7 hours ago
2026-07-16 04:40:39.543683+00:00fmpNo immediate curated freshness action required.
fmp_scores_bulk
fresh_3d
1162,1584.39 MB2026-07-16 04:40:37.900346+00:00
41.7 hours ago
2026-07-16 04:40:37.281098+00:00fmpNo immediate curated freshness action required.
fmp_ratios_ttm_bulk
fresh_3d
1171,32928.06 MB2026-07-16 04:40:34.730602+00:00
41.7 hours ago
2026-07-16 04:40:32.004013+00:00fmpNo immediate curated freshness action required.
fmp_rating_bulk
fresh_3d
1148,016508.82 KB2026-07-16 04:40:24.029191+00:00
41.8 hours ago
2026-07-16 04:40:23.642874+00:00fmpNo immediate curated freshness action required.
fmp_profile_bulk
fresh_3d
4490,35452.46 MB2026-07-16 04:40:13.418703+00:00
41.8 hours ago
2026-07-16 04:40:12.285639+00:00fmpNo immediate curated freshness action required.
fmp_price_target_summary_bulk
fresh_3d
115,194132.18 KB2026-07-16 04:39:52.862970+00:00
41.8 hours ago
2026-07-16 04:39:52.797881+00:00fmpNo immediate curated freshness action required.
fmp_peers_bulk
fresh_3d
1182,7874.65 MB2026-07-16 04:39:50.919048+00:00
41.8 hours ago
2026-07-16 04:39:50.443678+00:00fmpNo immediate curated freshness action required.
fmp_key_metrics_ttm_bulk
fresh_3d
1171,32519.52 MB2026-07-16 04:39:47.845910+00:00
41.8 hours ago
2026-07-16 04:39:46.097792+00:00fmpNo immediate curated freshness action required.
fmp_etf_holder_bulk
fresh_3d
337,149,479227.01 MB2026-07-16 04:39:28.253143+00:00
41.8 hours ago
2026-07-16 04:38:57.640511+00:00fmpNo immediate curated freshness action required.
fmp_dcf_bulk
fresh_3d
1133,151703.11 KB2026-07-16 04:36:59.346184+00:00
41.8 hours ago
2026-07-16 04:36:59.085344+00:00fmpNo immediate curated freshness action required.
balance_sheet_annual
fresh_3d
34,40734,407254,447849.44 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:03.653550+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
cash_flow_annual
fresh_3d
34,06334,063249,111659.10 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:03.123111+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
earnings
fresh_3d
27,55427,5541,530,026213.09 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:02.683634+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
income_statement_annual
fresh_3d
34,31034,310255,104547.39 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:03.364994+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
key_metrics_annual
fresh_3d
34,47534,475256,533672.56 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:03.942137+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
key_metrics_quarter
fresh_3d
34,24534,245550,508743.12 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:04.167587+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
ratios_annual
fresh_3d
34,47934,479256,636937.46 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:04.476820+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
ratios_quarter
fresh_3d
34,24834,248550,6831.01 GB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:04.725076+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
splits
fresh_3d
15,33815,33833,59929.20 MB2026-07-16 03:08:24.098376+00:00
43.3 hours ago
2026-07-16 03:08:02.633376+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
balance_sheet_quarter
current_14d
70,65770,6573,148,4703.72 GB2026-07-07 13:45:14.763046+00:00
248.7 hours ago
2026-07-07 12:43:04.050020+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
cash_flow_quarter
current_14d
69,85369,8533,210,9272.88 GB2026-07-07 13:45:14.763046+00:00
248.7 hours ago
2026-07-07 12:43:04.050020+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
income_statement_quarter
current_14d
70,09970,0993,314,1102.61 GB2026-07-07 13:45:14.763046+00:00
248.7 hours ago
2026-07-07 12:43:04.050020+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
balance_sheet_growth_quarter
current_14d
70,58970,5893,271,1553.83 GB2026-07-07 12:43:06.647879+00:00
249.7 hours ago
2026-07-07 12:43:04.050020+00:00fmpNo immediate curated freshness action required.
cash_flow_growth_quarter
current_14d
69,80669,8063,330,3832.93 GB2026-07-07 12:43:06.647879+00:00
249.7 hours ago
2026-07-07 12:43:04.050020+00:00fmpNo immediate curated freshness action required.
income_statement_growth_quarter
current_14d
70,24770,2473,410,9812.64 GB2026-07-07 12:43:06.647879+00:00
249.7 hours ago
2026-07-07 12:43:04.050020+00:00fmpNo immediate curated freshness action required.
fmp_income_statement_growth_bulk
current_14d
1241243,410,757559.98 MB2026-07-07 05:53:08.232228+00:00
256.5 hours ago
2026-07-07 05:53:08.124646+00:00fmpNo immediate curated freshness action required.
fmp_income_statement_bulk
current_14d
1241243,398,369516.99 MB2026-07-07 05:37:24.419863+00:00
256.8 hours ago
2026-07-07 05:37:24.288219+00:00fmpNo immediate curated freshness action required.
fmp_cash_flow_statement_growth_bulk
current_14d
1241243,330,138473.55 MB2026-07-07 05:20:55.694703+00:00
257.1 hours ago
2026-07-07 05:20:20.927225+00:00fmpNo immediate curated freshness action required.
fmp_cash_flow_statement_bulk
current_14d
1241243,321,882457.04 MB2026-07-07 05:04:52.073751+00:00
257.3 hours ago
2026-07-07 05:04:17.069568+00:00fmpNo immediate curated freshness action required.
fmp_balance_sheet_statement_growth_bulk
current_14d
1241243,270,895750.05 MB2026-07-07 04:49:20.619705+00:00
257.6 hours ago
2026-07-07 04:49:20.470822+00:00fmpNo immediate curated freshness action required.
fmp_balance_sheet_statement_bulk
current_14d
1241243,255,194769.65 MB2026-07-07 04:36:32.772183+00:00
257.8 hours ago
2026-07-07 04:36:32.607372+00:00fmpNo immediate curated freshness action required.
fmp_earnings_surprises_bulk
current_14d
3131709,5937.58 MB2026-07-07 00:29:55.756789+00:00
261.9 hours ago
2026-07-07 00:29:55.711510+00:00fmpNo immediate curated freshness action required.
dividends
current_14d
26,13926,1391,241,770128.13 MB2026-07-06 22:46:35.314532+00:00
263.6 hours ago
2026-07-06 20:10:05.641055+00:00fmp, local_desktop_raw_apiNo immediate curated freshness action required.
sec_company_facts
SEC Company Facts
sql_backed_current
public
0000 B
hours ago
core.sec_company_fact
est. rows 0; size 40.89 GB
Dataset is SQL-backed and has no file-style curated asset yet; use the canonical SQL relation until a parquet publication bridge is added.
sec_filing_history
SEC Filing History
sql_backed_current
public
0000 B
hours ago
core.sec_filing_history
est. rows 0; size 14.89 GB
Dataset is SQL-backed and has no file-style curated asset yet; use the canonical SQL relation until a parquet publication bridge is added.