Copy of Market Tracker
kleven_market_tracker ·
ID 1P8j369Wxy_Bzf8AMfrVFdAzAnxzPBfXi1iSQGmGFLms ·
Effort to rehab: HIGH
Purpose
Infrastructure shape
Tabs (0)
| Tab | Status | Purpose | Notes |
|---|
External dependencies
| URL / endpoint | Status | Purpose | Replacement |
|---|---|---|---|
| https://query1.finance.yahoo.com/v10/finance/quoteSummary/ | dead (auth/crawler-locked) | ||
| https://finance.yahoo.com/quote/ | 500 per url_probe | ||
| https://stockcharts.com/h-sc/ui?s= | alive 200 | ||
| http://openinsider.com/search?q= | alive 200 | ||
| https://seekingalpha.com/symbol/ | alive 200 | ||
| https://whalewisdom.com/stock/ | alive 200 | ||
| https://www.google.com/search?q= | alive 200 | ||
| https://www.sec.gov/edgar/search/?r=el#/entityName= | alive 200 |
Rehab strategy
Approach: Replace IMPORTJSON layer with VPS Python writer feeding hidden fund_cache tab via Sheets API; VLOOKUP from existing formulas.
Full markdown analysis (click to expand)
Kleven Market Tracker — Analysis
Sheet ID: 1P8j369Wxy_Bzf8AMfrVFdAzAnxzPBfXi1iSQGmGFLms
Title: "Copy of Market Tracker" (Kleven, June 2024 vintage)
Scope: 16 tabs, 89,160 formulas (65,496 unique), 159 error cells, 29 named ranges, ~35k rows.
Sister sheet: Kleven Metrics (1lzVjg_2Kuhj21_Mmqj2pn-GfUXeBy7ieg-K7-v42dlQ).
Purpose
Kleven's Market Tracker is a power-user fork of the public Tracker v0.7d (same 16-tab skeleton, same 29 named ranges, same domain pattern at stockcharts/openinsider/whalewisdom/seekingalpha/finviz). The original Tracker queries a Universe sheet for tickers, hits Yahoo Finance JSON for fundamentals, and renders three timeframe views (d, w, $) plus a custom screener. Kleven layered on:
$+heavyweight tab (1,782 rows, 144 cols, 20,608 formulas, 152 errors) — a beefed-up "$" screener with deep YahooquoteSummarypulls per ticker.custommega-tab (3,877 rows × 319 cols, 47,644 formulas — over half the sheet) — user-configurable column screener that QUERY-selects columns by header label and chains IMPORTJSON for any module not pre-fetched.DASHtab — a multi-portfolio dashboard fed viaIMPORTRANGE(metricsSheet, "DASH!B1:AB")— reads the output of Kleven's Metrics sheet and overlays five portfolio sub-panels (AE,BV,DM,FD,GUticker columns).- Three legacy/hidden tabs (
d (old),w (old),$ (old)) carrying older snapshots.
Infrastructure shape
Function inventory: ISBLANK 88,869 / IMPORTJSON 62,195 / SUM 43,453 / HYPERLINK 26,491 / MATCH 15,571 / ADDRESS 15,505 / INDIRECT 15,496 / FILTER 321 / ARRAYFORMULA 162 / QUERY 17 / IFERROR 63.
The dominance of IMPORTJSON over IFERROR (62k vs 63) is the smoking gun: this sheet has effectively zero defensive wrapping around its data layer. When Yahoo dies, the cells go red.
URL templates live in admin!AQ3/admin!AR3 (primary) and admin!AF6/admin!AG6 (secondary):
- https://query1.finance.yahoo.com/v10/finance/quoteSummary/<TICKER>?modules=financialData,balanceSheetHistoryQuarterly,cashflowStatementHistoryQuarterly,defaultKeyStatistics,summaryDetail,incomeStatementHistoryQuarterly,earningsTrend
- Modules referenced across formulas: assetProfile, balanceSheetHistoryQuarterly, cashflowStatementHistoryQuarterly, defaultKeyStatistics, earningsTrend, financialData, incomeStatementHistoryQuarterly, majorHoldersBreakdown, netSharePurchaseActivity, price, quoteType, summaryDetail, summaryProfile.
IMPORTJSON is a custom Apps Script, not a native function — it lives in the bound script project (not captured here) and was the standard "v0.7" pattern for monkey-patching Yahoo into Sheets.
What works
- Static link layer: SEC EDGAR (200), openinsider (200), stockcharts (200), whalewisdom (200), seekingalpha (200), Google search (200) — all the HYPERLINK chrome on
DASH,custom,d,w,$is fine. - Cross-sheet plumbing: the
starttab handshake (B6/B7/B8 → IMPORTRANGE auth dance for Universe / Insider Buying / Metrics) is intact and reads valid sheet IDs fromadmin!E3:E5. - Lookup/setup machinery (
setuptab QUERY-pivots from Universe,industrytaxonomy with 2,229 rows, sub-industry named range covering 5,711 rows) is structurally sound. - DASH header/sort logic, the QUERY engine in
custom, and the dynamic column-selection viaGW14header concat — all pure-Sheets work that survives Yahoo's death.
What's broken
- 152
#ERROR!cells in$+(CH4 onwards, columns CH–GG roughly). All trace toIMPORTJSON(admin!$AQ$3 & ticker & admin!$AR$3, ...)— the URL probe confirmshttps://finance.yahoo.com/quote/AAPLreturns 500, and the unauthenticatedquery1.finance.yahoo.com/v10/finance/quoteSummaryendpoint has been crawler-locked since mid-2023. EveryquoteSummarypull is a tombstone. - 5
#N/Aerrors inindustry(AE2 onwards) — HYPERLINK + MATCH against an industry list whose source rows shifted; dead anchors, cosmetic. - 2
#N/Aerrors inadmin(BE3, BF3) — TRANSPOSE/SPLIT overindustry!B2:AL6; likely the same shifted-list issue. - The visible
$+!CH4snapshot pulls 6 fields in a single IMPORTJSON call (sharesOutstanding, totalRevenue, fiftyTwoWeekHigh, etc.) — when Yahoo 500s, all six derived columns die together. setup!AR5 = IMPORTRANGE(start!B8, "yahoo!B2:BP5050")expects ayahootab inside the linked Metrics sheet — confirm Kleven Metrics still serves it.
External dependencies
From _url_probe.json, occurrences attributable to this sheet (shared with tracker_v07d):
- stockcharts.com/h-sc/ui?s= — 32,204 occurrences, alive
- openinsider.com/search?q= — 14,528, alive
- seekingalpha.com/symbol/ — 1,242, alive
- whalewisdom.com/stock/ — 1,240, alive
- finance.yahoo.com/quote/ — 1,220, 500 (display links, decorative)
- google.com/search?q= — 1,220, alive
- sec.gov/edgar/search/?r=el#/entityName= — 1,220, alive
- query1.finance.yahoo.com/v10/finance/quoteSummary/ — embedded inside ~62k IMPORTJSON formulas; not directly probed but de-facto dead for unauth scraping.
Cross-sheet dependencies
- Universe sheet (
start!B6→1tNrpUxxx12Cxe-NG4W0C14VoJ3ujo5SAHCC7cqYD3K4): supplies ticker rows viaIMPORTRANGE(universe, "universe!A4:AE")intosetup!B3andsetup!B5, plus tags viaadmin!W28. - Insider Buying sheet (
start!B7→1PcMpdUwgJOQ7S-hAjthzkalF7YkFLu2bcVy85Cj3VLU):setup!AG5pullsuniverseBuys!C4:M, and$+!BC4pullslargeBuys!A4:AE. - Kleven Metrics sheet (
start!B8→1lzVjg_2Kuhj21_Mmqj2pn-GfUXeBy7ieg-K7-v42dlQ):DASH!B1consumesDASH!B1:ABfrom Metrics;setup!AR5pullsyahoo!B2:BP5050. The DASH tab is downstream of Metrics — fixing this sheet is gated on fixing Metrics. - All three handshakes use
admin!M3 / AA3 / AL3access tokens echoed back to confirm IMPORTRANGE permission.
Rehab strategy
Kill IMPORTJSON; replace with a VPS-written cache tab.
- Static link layer — keep as is. EDGAR/openinsider/whalewisdom/seekingalpha/stockcharts links work; only the displayed
finance.yahoo.com/quote/HYPERLINKs need swapping (cosmetic; consider redirect to TradingView or remove). - Yahoo replacement (the actual job) — every IMPORTJSON formula maps to specific JSON paths under
quoteSummary/result/{module}/. The 13 modules touched all have direct EDGAR + price-feed equivalents: incomeStatementHistoryQuarterly,balanceSheetHistoryQuarterly,cashflowStatementHistoryQuarterly→ secfsdstools quarterly FY/FQ filings keyed by ticker→CIK.defaultKeyStatistics,financialData,summaryDetail→ derived from EDGAR + last-trade price (FRED/IEX/Polygon/yfinance-replacement).earningsTrend→ SEC analyst estimates not available; use FMP free tier or drop column.majorHoldersBreakdown,netSharePurchaseActivity→ SEC Form 4 (insider buys already covered by sister sheet) + 13F (whalewisdom).price,quoteType,assetProfile,summaryProfile→ static metadata table built once from EDGAR submissions API.- Architecture — Python writer on VPS computes a per-ticker fundamentals row, writes to a hidden
fund_cachetab via Sheets API. Replace each IMPORTJSON with a VLOOKUP intofund_cache. ARRAYFORMULA-friendly. Removes 62,195 formula calls → maybe 200 VLOOKUPs. - Wrap remainders in IFERROR — only 63 IFERROR uses today; bake in defensive layer during refactor.
- Industry HYPERLINK fixes — repoint the 5
#N/Acells to currentindustry!A8:Arow offsets; trivial. - Drop
(old)tabs — already hidden, no live references exceptd (old)is on the same sheet ID; safe to delete after audit. - Coordinate with Metrics rehab — DASH is read-only from Metrics; do that sheet first or in parallel.
Effort rating
HIGH — the sheet is mechanically simple to fix (one cache table replaces 62k formulas) but the semantic mapping from 13 Yahoo quoteSummary modules to EDGAR + price + 13F sources is the project's central data-engineering task. Effort lives in the writer, not in the sheet edits. Once Metrics is rehabbed, the rewrite of $+ and custom IMPORTJSON cells is mechanical (~1 day of formula surgery against a fresh fund_cache tab). Without the writer, this sheet stays dead.
Estimated split: 80% Python/EDGAR work (shared across all Tracker-family sheets), 20% sheet-side surgery here.
Raw analysis.json
{
"author": "Kleven",
"cross_sheet_dependencies": [
{
"consumed_at": [
"DASH!B1 (DASH!B1:AB)",
"setup!AR5 (yahoo!B2:BP5050)"
],
"direction": "downstream of Metrics \u2014 DASH is gated on Metrics rehab",
"named_range": "metricsSheet (admin!E5)",
"target": "kleven_metrics",
"target_id": "1lzVjg_2Kuhj21_Mmqj2pn-GfUXeBy7ieg-K7-v42dlQ"
},
{
"consumed_at": [
"setup!B3 (universe!A4:AE select 19 cols)",
"setup!B5 (universe!A4:AE select 16 cols)",
"setup!R5 (universe!AF4:AT)",
"admin!W28 (tagsList)"
],
"named_range": "universe (admin!E3)",
"target": "universe_sheet",
"target_id": "1tNrpUxxx12Cxe-NG4W0C14VoJ3ujo5SAHCC7cqYD3K4"
},
{
"consumed_at": [
"setup!AG5 (universeBuys!C4:M)",
"$+!BC4 (largeBuys!A4:AE)"
],
"named_range": "insiderBuying (admin!E4)",
"target": "insider_buying_sheet",
"target_id": "1PcMpdUwgJOQ7S-hAjthzkalF7YkFLu2bcVy85Cj3VLU"
}
],
"effort_breakdown": {
"blocking_dependency": "Yahoo replacement writer must exist before this sheet is usable; without it the sheet stays dead regardless of any in-sheet work.",
"in-sheet_surgery": "20% (mechanical once writer is producing fund_cache)",
"python_writer_edgar_work": "80% (shared across all Tracker-family sheets)"
},
"effort_rating": "HIGH",
"external_dependencies": [
{
"criticality": "blocking",
"occurrences_in_formulas": 62195,
"status": "dead (auth/crawler-locked)",
"url": "https://query1.finance.yahoo.com/v10/finance/quoteSummary/"
},
{
"criticality": "cosmetic (HYPERLINK display only)",
"occurrences": 1220,
"status": "500 per url_probe",
"url": "https://finance.yahoo.com/quote/"
},
{
"criticality": "cosmetic",
"occurrences": 32204,
"status": "alive 200",
"url": "https://stockcharts.com/h-sc/ui?s="
},
{
"criticality": "cosmetic",
"occurrences": 14528,
"status": "alive 200",
"url": "http://openinsider.com/search?q="
},
{
"criticality": "cosmetic",
"occurrences": 1242,
"status": "alive 200",
"url": "https://seekingalpha.com/symbol/"
},
{
"criticality": "cosmetic",
"occurrences": 1240,
"status": "alive 200",
"url": "https://whalewisdom.com/stock/"
},
{
"criticality": "cosmetic",
"occurrences": 1220,
"status": "alive 200",
"url": "https://www.google.com/search?q="
},
{
"criticality": "cosmetic",
"occurrences": 1220,
"status": "alive 200",
"url": "https://www.sec.gov/edgar/search/?r=el#/entityName="
}
],
"function_inventory": {
"ADDRESS": 15505,
"ARRAYFORMULA": 162,
"FILTER": 321,
"HYPERLINK": 26491,
"IFERROR": 63,
"IMPORTJSON": 62195,
"IMPORTRANGE": 12,
"INDEX": 13088,
"INDIRECT": 15496,
"ISBLANK": 88869,
"MATCH": 15571,
"QUERY": 17,
"REGEXREPLACE": 15,
"ROW": 15500,
"SORT": 171,
"SUM": 43453
},
"infrastructure_notes": [
"IMPORTJSON is a custom Apps Script (bound script project, not in inventory) \u2014 standard Tracker v0.7 pattern.",
"Defensive wrapping is essentially absent: 63 IFERROR vs 62,195 IMPORTJSON. When Yahoo dies, cells go red.",
"Yahoo URL is built from admin!AQ3 (base) + ticker + admin!AR3 (modules query string). Secondary template at admin!AF6/AG6.",
"Cross-sheet handshake at start!B6/B7/B8 with token-echo confirmation via admin!M3/AA3/AL3."
],
"lineage": "Power-user fork of Tracker v0.7d (same 16-tab skeleton, 29 named ranges, identical static link layer)",
"name": "Copy of Market Tracker",
"purpose": "Multi-timeframe market tracker (d/w/$) plus heavyweight $+ screener and configurable custom column screener; consumes a Universe sheet, an Insider Buying sheet, and Kleven\u0027s Metrics sheet via IMPORTRANGE; pulls fundamentals per ticker via IMPORTJSON against Yahoo Finance quoteSummary endpoint.",
"rehab_strategy": {
"approach": "Replace IMPORTJSON layer with VPS Python writer feeding hidden fund_cache tab via Sheets API; VLOOKUP from existing formulas.",
"data_source_mapping": {
"assetProfile": "static metadata from EDGAR submissions API",
"balanceSheetHistoryQuarterly": "secfsdstools EDGAR quarterly",
"cashflowStatementHistoryQuarterly": "secfsdstools EDGAR quarterly",
"defaultKeyStatistics": "derived from EDGAR + price feed",
"earningsTrend": "FMP free tier or drop column (no SEC equivalent)",
"financialData": "derived from EDGAR + price feed",
"incomeStatementHistoryQuarterly": "secfsdstools EDGAR quarterly",
"majorHoldersBreakdown": "13F via SEC + whalewisdom backstop",
"netSharePurchaseActivity": "SEC Form 4 (already covered by Insider Buying sister sheet)",
"price": "FRED/IEX/Polygon/yfinance-replacement",
"quoteType": "static metadata from EDGAR submissions API",
"summaryDetail": "derived from EDGAR + price feed",
"summaryProfile": "static metadata from EDGAR submissions API"
},
"ordered_steps": [
"Rehab Kleven Metrics first (or in parallel) \u2014 DASH consumes its output.",
"Build VPS writer producing per-ticker fund_cache row keyed on ticker.",
"Replace ~62k IMPORTJSON formulas with VLOOKUP into fund_cache (collapses to ~200 ARRAYFORMULA-driven lookups).",
"Wrap remaining IMPORTJSON-touch cells in IFERROR.",
"Repoint 5 industry!AE column #N/A HYPERLINKs to current industry!A8:A row offsets.",
"Fix admin!BE3/BF3 TRANSPOSE/SPLIT against same shifted industry list.",
"Audit and delete the three (old) tabs.",
"Replace cosmetic finance.yahoo.com/quote/ HYPERLINKs (TradingView or drop)."
]
},
"scale": {
"error_cells": 159,
"named_ranges": 29,
"tabs": 16,
"tabs_hidden": 3,
"total_formulas": 89160,
"total_rows": 35232,
"unique_formulas": 65496
},
"sheet_id": "1P8j369Wxy_Bzf8AMfrVFdAzAnxzPBfXi1iSQGmGFLms",
"slug": "kleven_market_tracker",
"tab_inventory": {
"$": {
"errors": 0,
"formulas": 4977,
"role": "dollar-volume timeframe view"
},
"$ (old)": {
"errors": 0,
"formulas": 1008,
"role": "legacy snapshot, hidden"
},
"$+": {
"errors": 152,
"formulas": 20608,
"primary_failure_site": true,
"role": "heavyweight screener with deep Yahoo quoteSummary pulls"
},
"DASH": {
"downstream_of": "kleven_metrics",
"errors": 0,
"formulas": 26,
"role": "multi-portfolio dashboard; reads from Metrics via IMPORTRANGE"
},
"admin": {
"errors": 2,
"formulas": 11,
"role": "config tab \u2014 Yahoo URL templates, sheet IDs, tag list"
},
"custom": {
"errors": 0,
"formulas": 47644,
"role": "user-configurable mega-screener; over half the sheet\u0027s formulas live here"
},
"d": {
"errors": 0,
"formulas": 4977,
"role": "daily timeframe view"
},
"d (old)": {
"errors": 0,
"formulas": 306,
"role": "legacy snapshot, hidden"
},
"industry": {
"errors": 5,
"formulas": 370,
"role": "GICS-style industry taxonomy + jump-link index"
},
"notes": {
"errors": 0,
"formulas": 0,
"role": "documentation"
},
"send": {
"errors": 0,
"formulas": 3,
"role": "concat aggregator pushing tickers back to Metrics"
},
"setup": {
"errors": 0,
"formulas": 29,
"role": "QUERY-pivot of Universe sheet rows; sub-industry named range source"
},
"start": {
"errors": 0,
"formulas": 8,
"role": "user-facing setup wizard for IMPORTRANGE handshakes"
},
"toDo": {
"errors": 0,
"formulas": 0,
"role": "user notes"
},
"w": {
"errors": 0,
"formulas": 4977,
"role": "weekly timeframe view"
},
"w (old)": {
"errors": 0,
"formulas": 4216,
"role": "legacy snapshot, hidden"
}
},
"vintage": "June 2024",
"what_is_broken": [
{
"cells": "CH4 onward through column GG (152 errors)",
"error_type": "#ERROR!",
"root_cause": "IMPORTJSON(admin!$AQ$3 \u0026 ticker \u0026 admin!$AR$3, ...) against query1.finance.yahoo.com/v10/finance/quoteSummary \u2014 endpoint crawler-locked since mid-2023; URL probe confirms finance.yahoo.com/quote/ returns 500",
"tab": "$+"
},
{
"cells": "AE2 onward (5 errors)",
"error_type": "#N/A",
"root_cause": "HYPERLINK+MATCH against industry list with shifted row anchors; cosmetic",
"tab": "industry"
},
{
"cells": "BE3, BF3 (2 errors)",
"error_type": "#N/A",
"root_cause": "TRANSPOSE/SPLIT over industry!B2:AL6 \u2014 same shifted-list issue",
"tab": "admin"
}
],
"what_works": [
"Static link layer: stockcharts/openinsider/whalewisdom/seekingalpha/EDGAR/Google all return 200.",
"IMPORTRANGE handshake to Universe, Insider Buying, Metrics is structurally intact.",
"setup-tab QUERY pivots from Universe sheet work.",
"industry taxonomy (2229 rows) and sub-industry named range (5711 rows) are sound.",
"DASH layout, custom-tab QUERY engine, and dynamic header-concat column selection are pure-Sheets logic."
],
"yahoo_modules_used": [
"assetProfile",
"balanceSheetHistoryQuarterly",
"cashflowStatementHistoryQuarterly",
"defaultKeyStatistics",
"earningsTrend",
"financialData",
"incomeStatementHistoryQuarterly",
"majorHoldersBreakdown",
"netSharePurchaseActivity",
"price",
"quoteType",
"summaryDetail",
"summaryProfile"
]
}