Data Sources & Provenance
Probed live: 2026-05-10 14:43 UTC · DuckDB serving cache active (refreshed 2026-05-10 14:42:29.756849 UTC)
Every tab on the master Google Sheet is written by a Python writer running on the VPS at
a private VPS. Writers run on systemd timers
(no manual intervention). The terminal pages you see read the master sheet read-only via
a Google service account, with a 5-minute cache.
Per-tab data flow
| Tab | Writer | Cadence | Rows | Freshness key | Latest value | Age |
|---|---|---|---|---|---|---|
| Universe | sec_writer |
weekly Sun 04:00 UTC | 10,377 | last_filing_date |
2026-05-08 | 2d |
| Fundamentals_Annual | sec_writer |
weekly Sun 04:00 UTC | 20,528 | period_end |
20250531 | 344d |
| Fundamentals_Quarterly | sec_writer |
weekly Sun 04:00 UTC | 63,207 | period_end |
20251031 | 191d |
| Fundamentals_TTM | sec_writer |
weekly Sun 04:00 UTC | 1,191 | as_of |
2026-05-09 | 1d |
| Treasury_DailyRates | fred_writer |
daily 13:00 UTC | 16,788 | record_date |
2026-05-07 | 3d |
| FRED_Macro | fred_writer |
daily 13:00 UTC | 12 | as_of |
2026-05-08 | 2d |
| Insider_Daily | openinsider_writer |
every 6h | 100 | filing_date |
2026-05-08 8:54:53 | 2d |
| Prices_Daily | prices_writer (yfinance) |
daily 22:00 UTC | 8,433 | as_of |
2026-05-08 | 2d |
| Yahoo_Quote | yahoo_quote_writer (yfinance .info) |
daily 23:00 UTC | 10,380 | ticker |
ZYME | — |
| Yahoo_Cashflow | yahoo_cashflow_writer (yfinance qcf+qis) |
daily 01:00 UTC | 10,380 | as_of |
2026-05-10 | 0d |
| Yahoo_Extras | yahoo_extras_writer (yfinance qbs+qis+info) |
daily 00:00 UTC | 10,380 | as_of |
2026-05-10 | 0d |
| Social_Trends | social_writer (ApeWisdom) |
hourly | 500 | as_of |
2026-05-10T14:00:06+00:00 | 0d |
Source pedigree
| Source | Provider | Auth | Cost | Reliability |
|---|---|---|---|---|
SEC EDGAR XBRL Financial Statement Data Sets |
U.S. Securities and Exchange Commission | None (User-Agent header required) | Free | Authoritative |
secfsdstools 2.4.3 |
Apache-2.0 OSS, parses SEC quarterly bundles | — | Free | Production-stable + 3 vendored bug patches |
FRED API (st. louis fed) |
Federal Reserve Bank of St. Louis | Free API key | Free | Authoritative; same data the Fed publishes |
openinsider.com scrape |
Aggregator of SEC Form 4 filings | None | Free | Reliable; one-step removed from SEC |
yfinance 1.3 |
Community wrapper around Yahoo Finance | None | Free | Fragile; Yahoo unauth endpoint rate-limits to ~500 sequential .info requests |
ApeWisdom API |
Public social-mention aggregator (WSB / stocktwits / yolo / investing) | None | Free | Aggregates already-aggregated feeds; one step removed |
What "fully autonomous" means
- SEC fundamentals re-run weekly Sunday 04:00 UTC (~10 hr full backfill on the 3.7GB VPS)
- FRED macro + Treasury yields refreshed daily 13:00 UTC (~30 sec)
- Insider transactions refreshed every 6 hours (02/08/14/20 UTC, ~10 sec)
- yfinance prices refreshed daily 22:00 UTC (~10 min)
- Yahoo per-ticker fundamentals refreshed daily 23:00 UTC (~30 min, partial coverage; large caps complete, tail tickers gradually fill on retry)
- Social mentions refreshed hourly (ApeWisdom, ~5 sec)
- Feedback API persists to SQLite (form on F8)
Known data gaps
Multi-currency CIK chunks dropped during last sec_writer run. About half the universe
(incl. AAPL, JPM, XOM) is missing from
Fundamentals_Annual/Quarterly because chunks
containing even one foreign-issuer CIK had the whole batch's standardize step skipped. Fix is a
per-CIK standardize loop instead of per-chunk batch — patched in v1.1, will fully populate on
the next weekly timer cycle.