docs: fondamentaux (architecture, schéma, mapping, normalisation, argument point-in-time, exemple Apple Q2 FY2024, prototype, exploitation, estimation du backfill), formules des ratios générées, AsyncAPI 3 du stream

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Simon-Pierre Boucher committed 20 days ago (Sep 5, 2026) parent 9a19a1a

3 changed files +935 −0

added docs/asyncapi.yaml +292 −0

		`@@ -0,0 +1,292 @@`
	1	+asyncapi: 3.0.0
	2	+info:
	3	+ title: HF Market Data — Filings stream
	4	+ version: 1.0.0
	5	+ description: \|
	6	+ WebSocket stream of SEC EDGAR filing events for the companies covered by `/v1/fundamentals`.
	7	+ One socket, one channel (`filings`), JSON text frames. Authentication by API key
	8	+ (`?api_key=<key>` or `Authorization: Bearer <key>`); keyless connections receive an `error`
	9	+ message and are closed with code 4001. At most 5 concurrent connections per key (6th →
	10	+ `STREAM_CONNECTION_LIMIT`, close 4029). Heartbeat every 20 s. The server keeps the last
	11	+ 1 000 events; `resume_token` (the last `seq` you received) replays what you missed.
	12	+ Each `filing` message delivered is charged 1 row on your rows quota (reduced rate).
	13	+ contact:
	14	+ name: Simon-Pierre Boucher
	15	+ email: contact@spboucher.ai
	16	+ url: https://www.hfmarketdata.io
	17	+ license:
	18	+ name: Data: SEC EDGAR (public domain) — see terms
	19	+ url: https://www.hfmarketdata.io/docs/terms
	20	+defaultContentType: application/json
	21	+
	22	+servers:
	23	+ production:
	24	+ host: www.hfmarketdata.io
	25	+ pathname: /v1/stream
	26	+ protocol: wss
	27	+ description: Production WebSocket endpoint.
	28	+ security:
	29	+ - $ref: '#/components/securitySchemes/apiKeyQuery'
	30	+ - $ref: '#/components/securitySchemes/apiKeyBearer'
	31	+
	32	+channels:
	33	+ stream:
	34	+ address: /v1/stream
	35	+ title: Single multiplexed socket
	36	+ description: \|
	37	+ All messages flow on the one socket. Client → server: `subscribe`, `unsubscribe`, `ping`.
	38	+ Server → client: `hello`, `subscribed`, `unsubscribed`, `filing`, `heartbeat`, `pong`, `error`.
	39	+ messages:
	40	+ subscribe:
	41	+ $ref: '#/components/messages/subscribe'
	42	+ unsubscribe:
	43	+ $ref: '#/components/messages/unsubscribe'
	44	+ ping:
	45	+ $ref: '#/components/messages/ping'
	46	+ hello:
	47	+ $ref: '#/components/messages/hello'
	48	+ subscribed:
	49	+ $ref: '#/components/messages/subscribed'
	50	+ unsubscribed:
	51	+ $ref: '#/components/messages/unsubscribed'
	52	+ filing:
	53	+ $ref: '#/components/messages/filing'
	54	+ heartbeat:
	55	+ $ref: '#/components/messages/heartbeat'
	56	+ pong:
	57	+ $ref: '#/components/messages/pong'
	58	+ error:
	59	+ $ref: '#/components/messages/error'
	60	+
	61	+operations:
	62	+ subscribeFilings:
	63	+ action: send
	64	+ channel:
	65	+ $ref: '#/channels/stream'
	66	+ summary: Subscribe to the `filings` channel (optionally filtered by tickers / forms, resumable).
	67	+ messages:
	68	+ - $ref: '#/channels/stream/messages/subscribe'
	69	+ reply:
	70	+ channel:
	71	+ $ref: '#/channels/stream'
	72	+ messages:
	73	+ - $ref: '#/channels/stream/messages/subscribed'
	74	+ - $ref: '#/channels/stream/messages/error'
	75	+ unsubscribe:
	76	+ action: send
	77	+ channel:
	78	+ $ref: '#/channels/stream'
	79	+ messages:
	80	+ - $ref: '#/channels/stream/messages/unsubscribe'
	81	+ ping:
	82	+ action: send
	83	+ channel:
	84	+ $ref: '#/channels/stream'
	85	+ messages:
	86	+ - $ref: '#/channels/stream/messages/ping'
	87	+ reply:
	88	+ channel:
	89	+ $ref: '#/channels/stream'
	90	+ messages:
	91	+ - $ref: '#/channels/stream/messages/pong'
	92	+ receiveEvents:
	93	+ action: receive
	94	+ channel:
	95	+ $ref: '#/channels/stream'
	96	+ summary: Server pushes — the first frame is always `hello`.
	97	+ messages:
	98	+ - $ref: '#/channels/stream/messages/hello'
	99	+ - $ref: '#/channels/stream/messages/filing'
	100	+ - $ref: '#/channels/stream/messages/heartbeat'
	101	+ - $ref: '#/channels/stream/messages/error'
	102	+
	103	+components:
	104	+ securitySchemes:
	105	+ apiKeyQuery:
	106	+ type: httpApiKey
	107	+ name: api_key
	108	+ in: query
	109	+ description: '`wss://www.hfmarketdata.io/v1/stream?api_key=hfmd_live_…`'
	110	+ apiKeyBearer:
	111	+ type: http
	112	+ scheme: bearer
	113	+ bearerFormat: hfmd_live_…
	114	+ description: '`Authorization: Bearer hfmd_live_…` on the upgrade request.'
	115	+
	116	+ messages:
	117	+ subscribe:
	118	+ name: subscribe
	119	+ title: Subscribe
	120	+ summary: Start (or replace) the subscription of this socket.
	121	+ payload:
	122	+ type: object
	123	+ required: [action]
	124	+ properties:
	125	+ action:
	126	+ const: subscribe
	127	+ channel:
	128	+ type: string
	129	+ enum: [filings]
	130	+ default: filings
	131	+ tickers:
	132	+ description: List of tickers (any share class) or the string `"all"`.
	133	+ oneOf:
	134	+ - type: array
	135	+ minItems: 1
	136	+ items: { type: string }
	137	+ - const: all
	138	+ default: all
	139	+ forms:
	140	+ description: Form types to receive; omitted = all (10-K, 10-Q, 8-K, 20-F, 40-F, 6-K and amendments).
	141	+ type: array
	142	+ items: { type: string, examples: ['10-K', '10-Q', '8-K', '20-F', '10-K/A'] }
	143	+ resume_token:
	144	+ description: Last `seq` received; events with a greater `seq` still in the 1 000-event buffer are replayed first.
	145	+ type: integer
	146	+ examples:
	147	+ - name: Apple and Microsoft periodic reports, resumed
	148	+ payload: { action: subscribe, channel: filings, tickers: [AAPL, MSFT], forms: ['10-K', '10-Q'], resume_token: 1287 }
	149	+ unsubscribe:
	150	+ name: unsubscribe
	151	+ payload:
	152	+ type: object
	153	+ properties:
	154	+ action: { const: unsubscribe }
	155	+ ping:
	156	+ name: ping
	157	+ payload:
	158	+ type: object
	159	+ properties:
	160	+ action: { const: ping }
	161	+ hello:
	162	+ name: hello
	163	+ summary: First server frame after authentication.
	164	+ payload:
	165	+ type: object
	166	+ required: [type, seq, heartbeat_seconds]
	167	+ properties:
	168	+ type: { const: hello }
	169	+ seq: { type: integer, description: Current last sequence number (use it later as `resume_token`). }
	170	+ heartbeat_seconds: { type: number, const: 20 }
	171	+ max_connections_per_key: { type: integer, const: 5 }
	172	+ buffer: { type: integer, const: 1000 }
	173	+ ts: { type: string, format: date-time }
	174	+ subscribed:
	175	+ name: subscribed
	176	+ payload:
	177	+ type: object
	178	+ properties:
	179	+ type: { const: subscribed }
	180	+ channel: { const: filings }
	181	+ tickers:
	182	+ oneOf: [{ type: array, items: { type: string } }, { const: all }]
	183	+ forms:
	184	+ oneOf: [{ type: array, items: { type: string } }, { const: all }]
	185	+ resume_from: { type: integer }
	186	+ ts: { type: string, format: date-time }
	187	+ unsubscribed:
	188	+ name: unsubscribed
	189	+ payload:
	190	+ type: object
	191	+ properties:
	192	+ type: { const: unsubscribed }
	193	+ ts: { type: string, format: date-time }
	194	+ filing:
	195	+ name: filing
	196	+ title: Filing event
	197	+ summary: A new EDGAR filing of a covered company (published by the ingestion within ~2 minutes of EDGAR acceptance).
	198	+ payload:
	199	+ type: object
	200	+ required: [type, ticker, cik, form, filed_date, url, accn, seq]
	201	+ properties:
	202	+ type: { const: filing }
	203	+ ticker: { type: string, examples: [AAPL] }
	204	+ cik: { type: integer, examples: [320193] }
	205	+ form: { type: string, examples: ['10-Q'] }
	206	+ period: { type: [string, 'null'], format: date, description: Period of report. }
	207	+ filed_date: { type: string, format: date }
	208	+ url: { type: string, format: uri, description: Primary document on EDGAR. }
	209	+ accn: { type: string, description: Accession number. }
	210	+ seq: { type: integer, description: Monotonic sequence number (resume token). }
	211	+ summary:
	212	+ description: Standardized figures of the filing's own period (null for 8-K and when not yet normalized). Raw USD.
	213	+ type: [object, 'null']
	214	+ properties:
	215	+ revenue: { type: [number, 'null'] }
	216	+ net_income: { type: [number, 'null'] }
	217	+ eps_diluted: { type: [number, 'null'] }
	218	+ total_assets: { type: [number, 'null'] }
	219	+ operating_cash_flow: { type: [number, 'null'] }
	220	+ fiscal_year: { type: integer }
	221	+ fiscal_quarter: { type: integer, description: 1–4, 0 = fiscal year }
	222	+ yoy:
	223	+ type: object
	224	+ description: Year-over-year growth versus the same fiscal period one year earlier (null when the base is missing or ≤ 0).
	225	+ properties:
	226	+ revenue: { type: [number, 'null'] }
	227	+ net_income: { type: [number, 'null'] }
	228	+ eps_diluted: { type: [number, 'null'] }
	229	+ examples:
	230	+ - name: Apple 10-Q for the quarter ended 2024-03-30
	231	+ payload:
	232	+ type: filing
	233	+ ticker: AAPL
	234	+ cik: 320193
	235	+ form: 10-Q
	236	+ period: '2024-03-30'
	237	+ filed_date: '2024-05-03'
	238	+ url: https://www.sec.gov/Archives/edgar/data/320193/000032019324000069/aapl-20240330.htm
	239	+ accn: 0000320193-24-000069
	240	+ seq: 1288
	241	+ summary:
	242	+ revenue: 90753000000
	243	+ net_income: 23636000000
	244	+ eps_diluted: 1.53
	245	+ total_assets: 337411000000
	246	+ operating_cash_flow: 22690000000
	247	+ fiscal_year: 2024
	248	+ fiscal_quarter: 2
	249	+ yoy: { revenue: -0.0431, net_income: -0.0222, eps_diluted: 0.0066 }
	250	+ heartbeat:
	251	+ name: heartbeat
	252	+ summary: Sent every 20 s while the socket is open.
	253	+ payload:
	254	+ type: object
	255	+ properties:
	256	+ type: { const: heartbeat }
	257	+ ts: { type: string, format: date-time }
	258	+ seq: { type: integer }
	259	+ pong:
	260	+ name: pong
	261	+ payload:
	262	+ type: object
	263	+ properties:
	264	+ type: { const: pong }
	265	+ ts: { type: string, format: date-time }
	266	+ seq: { type: integer }
	267	+ error:
	268	+ name: error
	269	+ summary: Same envelope fields as the REST errors (`code`, `message`, `docs`). Fatal errors are followed by a close frame.
	270	+ payload:
	271	+ type: object
	272	+ required: [type, code, message, docs]
	273	+ properties:
	274	+ type: { const: error }
	275	+ code:
	276	+ type: string
	277	+ enum: [AUTH_REQUIRED, STREAM_CONNECTION_LIMIT, VALIDATION_ERROR, NOT_FOUND]
	278	+ message: { type: string }
	279	+ docs: { type: string, format: uri }
	280	+ limit: { type: integer, description: Present on STREAM_CONNECTION_LIMIT. }
	281	+ examples:
	282	+ - name: keyless connection (then close 4001)
	283	+ payload:
	284	+ type: error
	285	+ code: AUTH_REQUIRED
	286	+ message: 'A valid API key is required for the stream: pass ?api_key=… or Authorization: Bearer ….'
	287	+ docs: https://www.hfmarketdata.io/docs/errors#auth_required
	288	+
	289	+x-close-codes:
	290	+ '4001': AUTH_REQUIRED — no or invalid API key
	291	+ '4029': STREAM_CONNECTION_LIMIT — more than 5 concurrent sockets for this key
	292	+ '4400': protocol error

added docs/fundamentals-ratios.md +327 −0

		`@@ -0,0 +1,327 @@`
	1	+# Fundamentals — ratio formulas
	2	+
	3	+Generated from `hfmarketdata/api/fundamentals/ratios.py` (`render_docs()`); do not edit by hand.
	4	+
	5	+Conventions: flows are trailing twelve months (sum of the last four discrete quarters — or the fiscal year with `period=annual`), balances are the latest balance sheet, `price` is the last close known at the valuation date. A ratio is `null` (with a reason in `meta.reasons`) whenever an input is missing or the denominator is not positive — nothing is ever invented.
	6	+
	7	+## Valuation
	8	+
	9	+### `market_cap`
	10	+
	11	+`market_cap = price × shares_outstanding`
	12	+
	13	+`shares_outstanding` is the cover-page share count (dei:EntityCommonStockSharesOutstanding, all
	14	+classes summed) of the latest filing known at the valuation date; falls back to weighted-average
	15	+diluted shares (flagged `shares_source=weighted_diluted`).
	16	+
	17	+Inputs: `price`, `shares_outstanding`
	18	+
	19	+### `enterprise_value`
	20	+
	21	+`enterprise_value = market_cap + total_debt − cash_and_equivalents − short_term_investments`
	22	+
	23	+Missing `short_term_investments` is treated as 0 (flagged); missing `total_debt` is treated as 0 only
	24	+when the balance sheet has no debt account at all (flagged `total_debt_assumed_zero`).
	25	+
	26	+Inputs: `market_cap`, `total_debt`, `cash_and_equivalents`, `short_term_investments`
	27	+
	28	+### `pe`
	29	+
	30	+`pe = price / eps_diluted (TTM)`
	31	+
	32	+Null when TTM diluted EPS ≤ 0 (a negative P/E is not meaningful).
	33	+
	34	+Inputs: `price`, `eps_diluted`
	35	+
	36	+### `forward_pe`
	37	+
	38	+`forward_pe = price / forward_eps`
	39	+
	40	+HF Market Data does not carry analyst consensus estimates, so `forward_eps` is never available
	41	+and `forward_pe` is always null with reason `no_estimates` — we do not extrapolate.
	42	+
	43	+Inputs: `forward_eps`
	44	+
	45	+### `pb`
	46	+
	47	+`pb = market_cap / total_equity`
	48	+
	49	+Null when book equity ≤ 0.
	50	+
	51	+Inputs: `market_cap`, `total_equity`
	52	+
	53	+### `ps`
	54	+
	55	+`ps = market_cap / revenue (TTM)`
	56	+
	57	+Inputs: `market_cap`, `revenue`
	58	+
	59	+### `ev_ebitda`
	60	+
	61	+`ev_ebitda = enterprise_value / ebitda (TTM)`
	62	+
	63	+Null when EBITDA ≤ 0.
	64	+
	65	+Inputs: `enterprise_value`, `ebitda`
	66	+
	67	+### `ev_sales`
	68	+
	69	+`ev_sales = enterprise_value / revenue (TTM)`
	70	+
	71	+Inputs: `enterprise_value`, `revenue`
	72	+
	73	+### `ev_fcf`
	74	+
	75	+`ev_fcf = enterprise_value / free_cash_flow (TTM)`
	76	+
	77	+Null when FCF ≤ 0.
	78	+
	79	+Inputs: `enterprise_value`, `free_cash_flow`
	80	+
	81	+### `earnings_yield`
	82	+
	83	+`earnings_yield = net_income (TTM) / market_cap`
	84	+
	85	+Inputs: `net_income`, `market_cap`
	86	+
	87	+### `fcf_yield`
	88	+
	89	+`fcf_yield = free_cash_flow (TTM) / market_cap`
	90	+
	91	+Inputs: `free_cash_flow`, `market_cap`
	92	+
	93	+### `dividend_yield`
	94	+
	95	+`dividend_yield = dividends paid (TTM, cash flow statement) / market_cap`
	96	+
	97	+Inputs: `dividends`, `market_cap`
	98	+
	99	+### `buyback_yield`
	100	+
	101	+`buyback_yield = buybacks (TTM, cash paid for repurchases) / market_cap`
	102	+
	103	+Inputs: `buybacks`, `market_cap`
	104	+
	105	+## Profitability
	106	+
	107	+### `gross_margin`
	108	+
	109	+`gross_margin = gross_profit / revenue (TTM)`
	110	+
	111	+Inputs: `gross_profit`, `revenue`
	112	+
	113	+### `operating_margin`
	114	+
	115	+`operating_margin = operating_income / revenue (TTM)`
	116	+
	117	+Inputs: `operating_income`, `revenue`
	118	+
	119	+### `net_margin`
	120	+
	121	+`net_margin = net_income / revenue (TTM)`
	122	+
	123	+Inputs: `net_income`, `revenue`
	124	+
	125	+### `ebitda_margin`
	126	+
	127	+`ebitda_margin = ebitda / revenue (TTM)`
	128	+
	129	+Inputs: `ebitda`, `revenue`
	130	+
	131	+### `fcf_margin`
	132	+
	133	+`fcf_margin = free_cash_flow / revenue (TTM)`
	134	+
	135	+Inputs: `free_cash_flow`, `revenue`
	136	+
	137	+### `roe`
	138	+
	139	+`roe = net_income (TTM) / total_equity (latest)`
	140	+
	141	+Uses the latest book equity, not the average — simpler and point-in-time consistent. Null when
	142	+equity ≤ 0.
	143	+
	144	+Inputs: `net_income`, `total_equity`
	145	+
	146	+### `roa`
	147	+
	148	+`roa = net_income (TTM) / total_assets (latest)`
	149	+
	150	+Inputs: `net_income`, `total_assets`
	151	+
	152	+### `roic`
	153	+
	154	+`roic = operating_income × (1 − tax_rate) / (total_debt + total_equity − cash_and_equivalents)`
	155	+
	156	+`tax_rate` = income_tax / pretax_income (TTM), only accepted in [0, 1]; when the effective rate is
	157	+not computable the ratio is null (`tax_rate_unavailable`) — no statutory rate is assumed. Null when
	158	+invested capital ≤ 0.
	159	+
	160	+Inputs: `operating_income`, `income_tax`, `pretax_income`, `total_debt`, `total_equity`, `cash_and_equivalents`
	161	+
	162	+## Liquidity
	163	+
	164	+### `current_ratio`
	165	+
	166	+`current_ratio = total_current_assets / total_current_liabilities`
	167	+
	168	+Inputs: `total_current_assets`, `total_current_liabilities`
	169	+
	170	+### `quick_ratio`
	171	+
	172	+`quick_ratio = (cash_and_equivalents + short_term_investments + receivables) / total_current_liabilities`
	173	+
	174	+Missing short_term_investments or receivables are treated as 0 (flagged).
	175	+
	176	+Inputs: `cash_and_equivalents`, `short_term_investments`, `receivables`, `total_current_liabilities`
	177	+
	178	+### `cash_ratio`
	179	+
	180	+`cash_ratio = (cash_and_equivalents + short_term_investments) / total_current_liabilities`
	181	+
	182	+Inputs: `cash_and_equivalents`, `short_term_investments`, `total_current_liabilities`
	183	+
	184	+## Solvency
	185	+
	186	+### `debt_to_equity`
	187	+
	188	+`debt_to_equity = total_debt / total_equity`
	189	+
	190	+Null when equity ≤ 0.
	191	+
	192	+Inputs: `total_debt`, `total_equity`
	193	+
	194	+### `debt_to_assets`
	195	+
	196	+`debt_to_assets = total_debt / total_assets`
	197	+
	198	+Inputs: `total_debt`, `total_assets`
	199	+
	200	+### `net_debt_to_ebitda`
	201	+
	202	+`net_debt_to_ebitda = net_debt / ebitda (TTM)`
	203	+
	204	+Null when EBITDA ≤ 0.
	205	+
	206	+Inputs: `net_debt`, `ebitda`
	207	+
	208	+### `interest_coverage`
	209	+
	210	+`interest_coverage = operating_income / interest_expense (TTM)`
	211	+
	212	+Null when interest expense is 0 or not reported.
	213	+
	214	+Inputs: `operating_income`, `interest_expense`
	215	+
	216	+## Efficiency
	217	+
	218	+### `asset_turnover`
	219	+
	220	+`asset_turnover = revenue (TTM) / total_assets (latest)`
	221	+
	222	+Inputs: `revenue`, `total_assets`
	223	+
	224	+### `inventory_turnover`
	225	+
	226	+`inventory_turnover = cost_of_revenue (TTM) / inventory (latest)`
	227	+
	228	+Inputs: `cost_of_revenue`, `inventory`
	229	+
	230	+### `receivables_turnover`
	231	+
	232	+`receivables_turnover = revenue (TTM) / receivables (latest)`
	233	+
	234	+Inputs: `revenue`, `receivables`
	235	+
	236	+### `days_sales_outstanding`
	237	+
	238	+`days_sales_outstanding = 365 × receivables / revenue (TTM)`
	239	+
	240	+Inputs: `receivables`, `revenue`
	241	+
	242	+### `cash_conversion_cycle`
	243	+
	244	+`cash_conversion_cycle = DSO + DIO − DPO`
	245	+
	246	+DSO = 365 × receivables / revenue · DIO = 365 × inventory / cost_of_revenue ·
	247	+DPO = 365 × accounts_payable / cost_of_revenue (all flows TTM, balances latest).
	248	+
	249	+Inputs: `receivables`, `revenue`, `inventory`, `accounts_payable`, `cost_of_revenue`
	250	+
	251	+## Growth
	252	+
	253	+### `revenue_growth_yoy`
	254	+
	255	+`revenue_growth_yoy = revenue (TTM) / revenue (TTM one year earlier) − 1`
	256	+
	257	+Inputs: `revenue`, `revenue_prev_year`
	258	+
	259	+### `revenue_growth_qoq`
	260	+
	261	+`revenue_growth_qoq = revenue (latest quarter) / revenue (previous quarter) − 1`
	262	+
	263	+Inputs: `revenue_q`, `revenue_prev_quarter`
	264	+
	265	+### `eps_growth_yoy`
	266	+
	267	+`eps_growth_yoy = eps_diluted (TTM) / eps_diluted (TTM one year earlier) − 1`
	268	+
	269	+Null when the base EPS ≤ 0.
	270	+
	271	+Inputs: `eps_diluted`, `eps_diluted_prev_year`
	272	+
	273	+### `fcf_growth_yoy`
	274	+
	275	+`fcf_growth_yoy = free_cash_flow (TTM) / free_cash_flow (TTM one year earlier) − 1`
	276	+
	277	+Inputs: `free_cash_flow`, `free_cash_flow_prev_year`
	278	+
	279	+### `revenue_cagr_3y`
	280	+
	281	+`revenue_cagr_3y = (revenue TTM / revenue TTM 3 years earlier)^(1/3) − 1`
	282	+
	283	+Inputs: `revenue`, `revenue_3y`
	284	+
	285	+### `revenue_cagr_5y`
	286	+
	287	+`revenue_cagr_5y = (revenue TTM / revenue TTM 5 years earlier)^(1/5) − 1`
	288	+
	289	+Inputs: `revenue`, `revenue_5y`
	290	+
	291	+### `revenue_cagr_10y`
	292	+
	293	+`revenue_cagr_10y = (revenue TTM / revenue TTM 10 years earlier)^(1/10) − 1`
	294	+
	295	+Inputs: `revenue`, `revenue_10y`
	296	+
	297	+### `eps_cagr_5y`
	298	+
	299	+`eps_cagr_5y = (eps_diluted TTM / eps_diluted TTM 5 years earlier)^(1/5) − 1`
	300	+
	301	+Inputs: `eps_diluted`, `eps_diluted_5y`
	302	+
	303	+## Per Share
	304	+
	305	+### `revenue_ps`
	306	+
	307	+`revenue_ps = revenue (TTM) / shares_diluted (weighted average, latest quarter)`
	308	+
	309	+Inputs: `revenue`, `shares_diluted`
	310	+
	311	+### `book_value_ps`
	312	+
	313	+`book_value_ps = total_equity / shares_outstanding`
	314	+
	315	+Inputs: `total_equity`, `shares_outstanding`
	316	+
	317	+### `fcf_ps`
	318	+
	319	+`fcf_ps = free_cash_flow (TTM) / shares_diluted`
	320	+
	321	+Inputs: `free_cash_flow`, `shares_diluted`
	322	+
	323	+### `cash_ps`
	324	+
	325	+`cash_ps = (cash_and_equivalents + short_term_investments) / shares_outstanding`
	326	+
	327	+Inputs: `cash_and_equivalents`, `short_term_investments`, `shares_outstanding`

added docs/fundamentals.md +316 −0

		`@@ -0,0 +1,316 @@`
	1	+# Fundamentals (SEC EDGAR) — architecture, schema, ingestion, point-in-time
	2	+
	3	+Chantier 7 of the v2 upgrade. Module `hfmarketdata/api/fundamentals/` (router `/v1/fundamentals`), plus
	4	+`stream/` (WebSocket `/v1/stream`) and `bulk/` (`/v1/bulk/fundamentals/{year}.parquet`). Data source: the SEC
	5	+EDGAR XBRL APIs (`companyfacts`, `submissions`), never proxied live to users. Ratio formulas:
	6	+[fundamentals-ratios.md](fundamentals-ratios.md). Stream protocol: [asyncapi.yaml](asyncapi.yaml).
	7	+
	8	+## 1. Architecture
	9	+
	10	+```
	11	+ EDGAR (data.sec.gov / www.sec.gov) ≤ 10 req/s token bucket, backoff 429/503, UA = settings.sec_user_agent
	12	+ │ companyfacts + submissions (+ MetaLinks.json of the latest 10-K, Atom feed, daily index)
	13	+ ▼
	14	+ data_root/edgar/raw/**.json.gz raw cache (replayable offline, one file per document)
	15	+ │ normalize.facts_frame
	16	+ ▼
	17	+ data_root/edgar/facts/cik={cik}/facts.parquet RAW FACTS LAKE (DuckDB) — every fact instance, versioned by (accn, filed)
	18	+ │ normalize.normalize_company (mapping.py priority lists, fiscal calendar, versions, derivation)
	19	+ ▼
	20	+ SQLite (core.db): edgar_companies · edgar_filings · fund_statements (wide, versioned) · fund_mapping ·
	21	+ fund_mapping_log · fund_coverage · fund_ingest_state · fund_latest (screener)
	22	+ │ │
	23	+ ▼ ▼
	24	+ /v1/fundamentals/* (service.py, DuckDB asof join with the price lake for ratios) Redis `filings` → /v1/stream
	25	+ data_root/bulk/fundamentals_{year}.parquet (bulk/build.py, ETag/304, quota exempt)
	26	+```
	27	+
	28	+Why two stores:
	29	+
	30	+* Parquet + DuckDB for facts — ~25 000 facts per large filer, ~200 M for the universe; columnar, cheap to
	31	+ rescan when the mapping changes (re-normalisation never re-downloads), `view=as_reported` and
	32	+ `/facts/{concept}` read it directly with predicate pushdown. Never mutated: facts are appended by filing.
	33	+* SQLite for standardized statements — needs indexes for point-in-time lookups (`ticker, period_end,
	34	+ filed_date`), small (≈ 1–2 M rows for 7 600 companies × 15 years × 3 statements × ~1.2 versions), transactional
	35	+ replace per company, and the screener table lives next to it. `coverage` JSON per row explains every null.
	36	+
	37	+## 2. Schema (SQLite, `fundamentals/models.py`)
	38	+
	39	+\| table \| key \| purpose \|
	40	+\|---\|---\|---\|
	41	+\| `edgar_companies` \| `cik` \| `ticker` (canonical = first SEC listing), `tickers` (all share classes: `["GOOGL","GOOG"]`), name, `sic`, `exchange`, `fiscal_year_end` (MMDD), `status` active/delisted, `ticker_history` JSON, `facts_updated_at`, `normalized_at` \|
	42	+\| `edgar_filings` \| `accn` \| cik, form, filed_date, period_of_report, primary_doc (full EDGAR URL), is_amendment, is_xbrl, parsed_at — index (cik, filed_date) \|
	43	+\| `fund_statements` \| id; unique (cik, statement, fiscal_year, fiscal_quarter, accn) \| wide, versioned: cik, ticker, statement (income/balance/cashflow), fiscal_year, fiscal_quarter (1–4, 0 = annual), period_start, period_end, calendar_quarter (`2024Q1`), form, accn, filed_date, derived, restated, currency, coverage JSON, mapping_version + one REAL column per account (49). Indexes: point-in-time `(ticker, period_end, filed_date)`, `(cik, statement, fiscal_year, fiscal_quarter, filed_date)`, `(statement, fiscal_quarter, calendar_quarter)` for frames \|
	44	+\| `fund_mapping` \| id; unique (version, account, taxonomy, tag) \| the prioritized mapping seeded from `mapping.py` — 183 rows (49 accounts, of which 5 computed and 2 auxiliary), version `2026.09.1` (`GET /v1/fundamentals/_mapping`) \|
	45	+\| `fund_mapping_log` \| (cik, taxonomy, tag) \| tags seen but not mapped: standard-taxonomy tags outside the mapping (`is_extension=false`, occurrences, first/last seen, sample accn) and company extensions found in the statements of the latest 10-K via `MetaLinks.json` (`is_extension=true`, `hint_account`) \|
	46	+\| `fund_coverage` \| ticker \| first/last period, quarters, annuals, filings, completeness % (overall + per statement), `missing_accounts` {account: reason, periods}, `gaps` (missing fiscal quarters), derived/restated counts, extensions logged \|
	47	+\| `fund_ingest_state` \| key (backfill/incremental/reconcile) \| last run/success, last RSS check, `lag_seconds`, companies total/done, failures + samples, `mapping_failure_rate`, requests, events published — served by `GET /v1/fundamentals/_health` \|
	48	+\| `fund_latest` \| ticker \| precomputed screener row: identity, price/price_date, TTM flows + latest balances (public accounts) + every ratio, `reasons` JSON, `shares_source` \|
	49	+
	50	+Raw facts lake columns: `cik, taxonomy, tag, unit, fy, fp, form, start, end, val, accn, filed, frame`
	51	+(de-duplicated on `(taxonomy, tag, unit, start, end, accn)` keeping the latest `filed`).
	52	+
	53	+## 3. Standard chart of accounts and mapping (`mapping.py`)
	54	+
	55	+Exactly the spec's accounts — income (16), balance (19), cash flow (12) — plus two auxiliary inputs that are
	56	+stored but not part of the public chart: `depreciation_amortization` (for `ebitda`) and `shares_outstanding`
	57	+(dei cover-page shares, all classes summed, for `market_cap`). Computed accounts (`*` in the spec): `ebitda`,
	58	+`total_debt`, `net_debt`, `working_capital`, `free_cash_flow` — never read from a tag, formula in `coverage`.
	59	+
	60	+Each account lists its tags in priority order; the first tag with a fact for the period wins and the chosen
	61	+tag + priority are written in `coverage[account]`. Examples (full table: `/v1/fundamentals/_mapping`):
	62	+
	63	+\| account \| priority list (us-gaap unless noted) \| why the fallbacks \|
	64	+\|---\|---\|---\|
	65	+\| revenue \| Revenues → RevenueFromContractWithCustomerExcludingAssessedTax → SalesRevenueNet → RevenueFromContractWithCustomerIncludingAssessedTax → SalesRevenueGoodsNet → SalesRevenueServicesNet → RevenuesNetOfInterestExpense → InterestAndDividendIncomeOperating → RegulatedAndUnregulatedOperatingRevenue → OperatingLeasesIncomeStatementLeaseRevenue → ifrs-full:Revenue \| ASC 606 (2018) replaced SalesRevenueNet; banks/utilities/REITs use their own top line \|
	66	+\| net_income \| NetIncomeLoss → NetIncomeLossAvailableToCommonStockholdersBasic → ProfitLoss → IncomeLossFromContinuingOperations → ifrs-full:ProfitLossAttributableToOwnersOfParent \| ProfitLoss includes NCI, used when the parent figure is absent \|
	67	+\| cash_and_equivalents \| CashAndCashEquivalentsAtCarryingValue → CashCashEquivalentsRestrictedCashAndRestrictedCashEquivalents → Cash → CashAndDueFromBanks → CashCashEquivalentsAndShortTermInvestments \| ASU 2016-18 presentation incl. restricted cash \|
	68	+\| short_term_debt \| DebtCurrent → LongTermDebtAndCapitalLeaseObligationsCurrent → LongTermDebtCurrent → ShortTermBorrowings → CommercialPaper → NotesPayableCurrent \| when DebtCurrent is absent the disjoint components are summed (`components: true`); Apple = CommercialPaper + LongTermDebtCurrent \|
	69	+\| sga_expense \| SellingGeneralAndAdministrativeExpense → G&A + Selling components \| split SG&A summed, flagged \|
	70	+\| capex \| PaymentsToAcquirePropertyPlantAndEquipment → PaymentsToAcquireProductiveAssets → PaymentsForCapitalImprovements → … \| positive = outflow (EDGAR convention) \|
	71	+
	72	+Accounting identities used as fallbacks (flagged `identity: true`): `gross_profit = revenue − cost_of_revenue`,
	73	+`total_liabilities = LiabilitiesAndStockholdersEquity − total_equity` (else `total_assets − total_equity`),
	74	+`operating_income = revenue − CostsAndExpenses`. Units: USD raw (not thousands), `shares`, `USD/share`; a
	75	+non-USD filer keeps its currency in `currency` and `coverage[account].currency` (20-F: ifrs-full tags are in
	76	+the lists). Extension tags (`aapl:`, `shak:`…) are logged, never guessed.
	77	+
	78	+## 4. Normalisation (`normalize.py`)
	79	+
	80	+1. Facts → frame, de-dup by `(tag, unit, start, end, accn)` keeping the latest filed.
	81	+2. Fiscal calendar. The `fy`/`fp` fields of a companyfacts fact describe the filing, not the fact
	82	+ (Apple's 10-Q for Q2 FY2025 re-reports Q2 FY2024 with `fy=2025, fp=Q2`). So the fiscal period of every fact
	83	+ is derived from its `end` against the fiscal year ends learnt from the 10-K report dates
	84	+ (`FiscalCalendar`, ±7-day snap for 52/53-week filers, MMDD extrapolation for years without a 10-K) and
	85	+ `fy`/`fp` are only used as a sanity check on the filing's own period (mismatches are counted in
	86	+ `stats.fiscal_mismatches`; 0 for the three prototypes). Apple Q2 FY2024: `end=2024-03-30`, FY end
	87	+ 2024-09-28 → 182 days → quarter 4 − round(182/91.3) = 2. `calendar_quarter` comes from `period_end`
	88	+ (an end in the first 7 days of a month belongs to the previous month: 2025-01-03 → 2024Q4).
	89	+3. Span classification of duration facts: 75–105 days = quarter, 165–195 = 6-month YTD, 255–290 = 9-month
	90	+ YTD, 340–380 = fiscal year; anything else (stub periods) is ignored.
	91	+4. Resolution per filing (accession) and period → `Resolved` values + coverage (section 3).
	92	+5. Versioning. Filings are replayed in `filed` order. A period gets a new row only when a filing changes or
	93	+ completes what was known (`restated=true` when a previously served number changed; same-accession
	94	+ reported + derived pieces are one version). A later filing that re-reports a period with a lower-priority
	95	+ concept (cash incl. restricted cash in a comparative column) does not override the better concept.
	96	+ Unique key `(cik, statement, fiscal_year, fiscal_quarter, accn)`.
	97	+6. Derivation (flagged `derived=true`, formula in `coverage[account].derived`): cash-flow statements in 10-Qs
	98	+ are year-to-date only, income statements often carry 9-month YTD facts — `Q2 = YTD6 − Q1`,
	99	+ `Q3 = YTD9 − YTD6` (or `YTD9 − Q1 − Q2`), `Q4 = FY − YTD9` or `FY − (Q1+Q2+Q3)`; only when every input
	100	+ belongs to the same fiscal year, same unit, and was known at that filing date. Weighted share counts use
	101	+ `4×FY − (Q1+Q2+Q3)` (`approx: true`); EPS Q4 = FY − ΣQ (`approx`). Nothing is derived when a quarter is missing.
	102	+7. Computed accounts (`ebitda`, `total_debt`, `net_debt`, `working_capital`, `free_cash_flow`) are attached
	103	+ to the latest version of the period with their formula; assumptions are flagged
	104	+ (`short_term_debt_assumed_zero`, `short_term_investments_assumed_zero`).
	105	+8. TTM (`ttm()`): sum of the last four consecutive fiscal quarters for flows, latest quarter for balances
	106	+ and share counts, EPS = sum of four quarterly EPS (`approx`). Windows with a hole are skipped
	107	+ (`missing_quarters` reason).
	108	+
	109	+### Point-in-time (anti look-ahead)
	110	+
	111	+Every row carries the `filed_date` of the filing that made it known. Serving "latest" = the version with the max
	112	+`filed_date` per (statement, fiscal_year, fiscal_quarter). `as_of=D` = the same selection restricted to
	113	+`filed_date ≤ D` (`normalize.select_as_of`). Consequences:
	114	+
	115	+* Apple's Q2 FY2024 income statement does not exist on 2024-05-02 and exists on 2024-05-03 (filing date).
	116	+* a 10-K/A or a later 10-K that restates a quarter creates a new version; `as_of` before the amendment returns
	117	+ the original numbers, after it the restated ones (`restated=true`).
	118	+* `/ratios?as_of=D` uses the fundamentals known at D and the last close at or before D;
	119	+ `/ratios/daily` builds one snapshot per `filed_date` and ASOF-joins it to the daily closes in DuckDB
	120	+ (`p.date >= s.valid_from`), so a 10-Q filed May 3 only affects May 3 onwards.
	121	+* `frames/{concept}?as_of=D` and the bulk files keep `accn`/`filed_date` for the same reason.
	122	+
	123	+## 5. Worked example — Apple, Q2 FY2024 (10-Q filed 2024-05-03, quarter ended 2024-03-30)
	124	+
	125	+`GET /v1/fundamentals/AAPL/statements?statement=income&period=quarterly&from=2024-03-30&to=2024-03-30`
	126	+
	127	+\| account \| value \| provenance (`coverage`) \|
	128	+\|---\|---\|---\|
	129	+\| revenue \| 90 753 000 000 \| `us-gaap:RevenueFromContractWithCustomerExcludingAssessedTax` (priority 2 — Apple does not tag `Revenues`) \|
	130	+\| cost_of_revenue \| 48 482 000 000 \| `CostOfGoodsAndServicesSold` \|
	131	+\| gross_profit \| 42 271 000 000 \| `GrossProfit` \|
	132	+\| operating_income \| 27 900 000 000 \| `OperatingIncomeLoss` \|
	133	+\| net_income \| 23 636 000 000 \| `NetIncomeLoss` \|
	134	+\| eps_diluted \| 1.53 \| `EarningsPerShareDiluted` (USD/shares) \|
	135	+\| shares_diluted \| 15 464 709 000 \| `WeightedAverageNumberOfDilutedSharesOutstanding` \|
	136	+\| ebitda \| 30 736 000 000 \| computed `operating_income + depreciation_amortization` (2 836 M from the cash-flow statement) \|
	137	+\| interest_expense, dividends_paid \| null \| `reason: no_mapped_tag` (Apple stopped tagging interest expense in FY2023) \|
	138	+
	139	+Balance sheet (same period): total_assets 337 411 000 000, cash 32 695 M, short-term investments 34 455 M,
	140	+short_term_debt 12 759 M (`CommercialPaper` + `LongTermDebtCurrent`, `components: true`), long_term_debt
	141	+91 831 M, total_debt 104 590 M (computed), total_equity 74 194 M, shares_outstanding 15 334 082 000 (cover page
	142	+2024-04-19).
	143	+
	144	+Cash flow Q2 FY2024 (`derived=true`): operating_cash_flow 22 690 000 000 = YTD6 62 585 M − Q1 39 895 M
	145	+(`derived: "YTD6-Q1"`), capex 1 996 M, free_cash_flow 20 694 M (computed).
	146	+
	147	+Q4 FY2024 (derived from the 10-K filed 2024-11-01): revenue 94 930 M = FY 391 035 M − YTD9 296 105 M.
	148	+TTM at Q2 FY2024: revenue 381 623 M, net income 100 389 M, diluted EPS 6.43 (sum of 4 quarters).
	149	+
	150	+`GET /v1/fundamentals/AAPL/ratios?as_of=2024-05-03` (real close 183.38 on 2024-05-03): market_cap =
	151	+183.38 × 15 334 082 000 = 2 812 T; pe = 183.38 / 6.43 = 28.5; pb = 37.9; gross_margin = 173 966 / 381 623 = 45.6 %;
	152	+net_margin 26.3 %; roe = 100 389 / 74 194 = 135 %; current_ratio = 128 416 / 123 822 = 1.04; debt_to_equity 1.41;
	153	+enterprise_value = market_cap + 104 590 M − 32 695 M − 34 455 M; forward_pe = null (`no_estimates`);
	154	+interest_coverage = null (`missing:interest_expense`); revenue_growth_yoy = 381 623 / 385 095 − 1 = −0.9 %.
	155	+(These numbers are asserted in `tests/test_fundamentals_unit.py::test_ratio_formulas_on_apple_q2_fy2024`
	156	+against the recorded fixture; in the test lake the price is synthetic, so the API test checks the formulas, not
	157	+the absolute valuation.)
	158	+
	159	+## 6. Prototype results (full companyfacts, 2026-09-04)
	160	+
	161	+\| company \| CIK \| facts \| filings \| statement versions \| derived rows \| restated rows \| completeness \| extensions logged \|
	162	+\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	163	+\| Apple (FYE late Sept, 52/53 weeks) \| 320193 \| 25 135 \| 70 \| 412 \| 113 \| 76 \| 87.2 % \| 3 \|
	164	+\| Microsoft (FYE June 30; 10-K carries quarterly data) \| 789019 \| 32 671 \| 68 \| 458 \| 60 \| 120 \| 85.9 % \| 4 \|
	165	+\| Shake Shack (Russell 2000; FYE last Wednesday of Dec, 2025 → Dec 31) \| 1620533 \| 18 358 \| 47 \| 266 \| 66 \| 42 \| 68.8 % \| 9 \|
	166	+
	167	+Shake Shack was chosen as the small cap because its income statement uses the company extension
	168	+`shak:OperatingMaterialsExpense` (food and paper costs) plus `us-gaap:LaborAndRelatedExpense` / `OccupancyNet`
	169	+instead of any cost-of-revenue concept: `cost_of_revenue` and `gross_profit` are therefore **null with
	170	+`no_mapped_tag`**, the extension is logged in `fund_mapping_log` with `hint_account=cost_of_revenue` and shown
	171	+in `/coverage.custom_extensions` — exactly the "log, don't guess" behaviour. Its 52/53-week calendar
	172	+(2023-12-27, 2024-12-25, then a change to 2025-12-31) and multi-class shares also exercise the calendar and
	173	+dei logic. Reconciliation against a fresh EDGAR fetch: 0 discrepancies for AAPL and SHAK.
	174	+
	175	+Latency on the test lake (TestClient, 25 runs, p95): statements 4–7 ms, ratios 7 ms, ratios/daily 1 year 30 ms
	176	+(2.5 years 51 ms), screener 3 ms, frames 3 ms, filings 3 ms, coverage 1 ms — far under the 300 ms / 1 s targets;
	177	+the screener reads a precomputed table so it stays O(rows in `fund_latest`) at universe scale.
	178	+
	179	+## 7. Ingestion
	180	+
	181	+* Universe (`ingest.sync_universe`): `company_tickers.json` + `company_tickers_exchange.json` restricted to
	182	+ the tickers present in `parquet/stock/1day/` and `parquet/etf/1day/`. One CIK with several tickers
	183	+ (GOOG/GOOGL, BRK-A/BRK-B) = one company, `tickers` lists the classes and any class resolves in the API. A
	184	+ ticker that leaves the SEC list is appended to `ticker_history` and the company becomes `delisted` (its data
	185	+ keeps being served). ETFs that are not SEC operating filers (SPY…) are simply absent →
	186	+ `FUNDAMENTALS_NOT_AVAILABLE`.
	187	+* Backfill (`scripts/edgar_backfill.py`): per CIK `companyfacts` + `submissions` (+ `MetaLinks.json` of the
	188	+ latest 10-K, 1 request) → lake → statements → coverage → screener row; manifest
	189	+ `data_root/edgar/backfill_manifest.json` makes it resumable (`ok` / `no_facts` are skipped, `error` retried;
	190	+ `--force` redoes). `--from-zip companyfacts.zip` reads the SEC bulk archive instead of the API. Ends with the
	191	+ bulk Parquet files.
	192	+* Incremental (`scripts/edgar_incremental.py`, every 2 min): Atom `getcurrent` feed per form
	193	+ (10-K, 10-Q, 8-K, 20-F — 4 requests) + today's/yesterday's daily master index as a safety net → new accessions
	194	+ of tracked CIKs → refetch companyfacts (cache bypass) → re-normalise → publish a `filing` event
	195	+ (Redis `filings` pub/sub + `filings:stream` capped at 1 000) → lag written to `fund_ingest_state`. The efts
	196	+ full-text search needs a query term and is therefore not used as a "list everything" source.
	197	+* Reconcile (`scripts/edgar_reconcile.py`): 20 random companies refetched and re-normalised in memory,
	198	+ every public account of every latest period compared to the DB (`--fix` re-ingests the differing ones);
	199	+ result in `fund_ingest_state.reconcile` and `_health`. Exit code 1 on discrepancies for alerting.
	200	+* Monitoring: `GET /v1/fundamentals/_health` (hidden from OpenAPI) — companies, statement versions,
	201	+ screener rows, last filed date, per-job lag/failures/`mapping_failure_rate` (share of null public accounts in
	202	+ `fund_latest`).
	203	+
	204	+### Running on production (M3U96b)
	205	+
	206	+```bash
	207	+ssh M3U96b
	208	+cd ~/hfmarketdata
	209	+export HFMD_DATA_ROOT=~/firstratedata # lake + edgar/ + bulk/ + state/hfmd.db ; Redis: redis://127.0.0.1:6379/0 (default)
	210	+venv/bin/pip install -r hfmarketdata/requirements.txt
	211	+
	212	+# 1) initial backfill (resumable; ~7 600 CIKs)
	213	+nohup venv/bin/python scripts/edgar_backfill.py --workers 4 > ~/edgar_backfill.log 2>&1 &
	214	+# faster first pass: download the SEC bulk archive once (~1.3 GB) and use it for companyfacts
	215	+# curl -A "$HFMD_SEC_USER_AGENT" -o /tmp/companyfacts.zip https://www.sec.gov/Archives/edgar/daily-index/xbrl/companyfacts.zip
	216	+# venv/bin/python scripts/edgar_backfill.py --from-zip /tmp/companyfacts.zip --workers 6
	217	+
	218	+# 2) incremental poller under PM2 (2-minute cycle, publishes to Redis for /v1/stream)
	219	+pm2 start venv/bin/python --name edgar-incremental --cwd ~/hfmarketdata -- scripts/edgar_incremental.py
	220	+pm2 save
	221	+
	222	+# 3) nightly reconciliation (cron 03:15) — exit code 1 = discrepancies
	223	+15 3 * * * cd ~/hfmarketdata && HFMD_DATA_ROOT=~/firstratedata venv/bin/python scripts/edgar_reconcile.py --json ~/edgar_reconcile.json >> ~/edgar_reconcile.log 2>&1
	224	+
	225	+# health
	226	+curl -s https://www.hfmarketdata.io/v1/fundamentals/_health \| jq .data.jobs
	227	+```
	228	+
	229	+The API process needs no restart: `init_db()` is idempotent and every request reads SQLite/Parquet.
	230	+
	231	+### Backfill duration estimate (~7 600 CIKs at 10 req/s)
	232	+
	233	+Requests: 2 per CIK (companyfacts + submissions) + 1 MetaLinks + ~0.3 for paginated older submissions ≈ 3.3 ×
	234	+7 600 ≈ 25 000 requests → ~42 min at the 10 req/s ceiling. Volume: companyfacts average ≈ 2–3 MB
	235	+(Apple 10 MB, small caps 1 MB) → ~20 GB downloaded, gzip-cached at ~2 GB. CPU: normalisation measured at
	236	+0.7–1.2 s per company (pandas, one core) → ~2 h single-threaded, ~35 min with `--workers 4` overlapping network
	237	+and CPU. Expected wall clock: 1 h – 1 h 30 with 4 workers (network-bound at the bucket), ≈ 45 min with
	238	+`--from-zip` (no companyfacts requests; the zip download itself takes a few minutes). Resulting sizes: facts lake
	239	+≈ 2.5 GB Parquet, SQLite ≈ 1.5 GB, bulk files ≈ 20 × 30 MB.
	240	+
	241	+## 8. Endpoints (all `format=json\|csv\|parquet`, cursor pagination, `as_of`)
	242	+
	243	+\| endpoint \| notes \|
	244	+\|---\|---\|
	245	+\| `GET /v1/fundamentals/{ticker}/statements` \| `statement=income\\|balance\\|cashflow\\|all`, `period=quarterly\\|annual\\|ttm`, `from`/`to` on period_end, `view=standardized\\|as_reported` \|
	246	+\| `GET /v1/fundamentals/{ticker}/facts/{concept}` \| standardized account → point-in-time series; raw `us-gaap:Tag` → fact instances from the lake (`CONCEPT_NOT_FOUND`) \|
	247	+\| `GET /v1/fundamentals/{ticker}/ratios` \| 45 ratios in 7 groups + `inputs`, `meta.reasons` for nulls, `period=ttm\\|annual` \|
	248	+\| `GET /v1/fundamentals/{ticker}/ratios/daily` \| daily close × fundamentals known that day (DuckDB ASOF), `fields=` \|
	249	+\| `GET /v1/fundamentals/{ticker}/filings` \| `form=` filter, EDGAR links \|
	250	+\| `GET /v1/fundamentals/{ticker}/coverage` \| completeness, missing accounts + reasons, gaps, custom extensions \|
	251	+\| `GET /v1/fundamentals/screener` \| `filters=pe<15,roe>15%,market_cap>1b,ev_ebitda=5..12,exchange=Nasdaq\\|NYSE`, `sort=fcf_yield:desc`, `columns=`; `request_cost=2`, `requires_key=true` \|
	252	+\| `GET /v1/fundamentals/frames/{concept}` \| `calendar_quarter=2024Q1` or `fiscal_year=&fiscal_quarter=`; `request_cost=2` \|
	253	+\| `GET /v1/fundamentals/_mapping` \| chart of accounts + prioritized tags (public) · `GET /_health` internal \|
	254	+\| `GET /v1/bulk/fundamentals` · `GET /v1/bulk/fundamentals/{year}.parquet` \| quota exempt, strong ETag, `If-None-Match` → 304 \|
	255	+\| `GET /v1/stream` (WebSocket) · `GET /v1/stream/info` \| see below \|
	256	+
	257	+Error codes added to `core/errors.py`: `FUNDAMENTALS_NOT_AVAILABLE` (404), `INVALID_FILTER` (400),
	258	+`CONCEPT_NOT_FOUND` (404), `STREAM_CONNECTION_LIMIT` (429).
	259	+
	260	+## 9. Stream (`stream/`)
	261	+
	262	+`GET /v1/stream` (WebSocket). Auth by `?api_key=` or `Authorization: Bearer` (keyless → JSON error then close
	263	+4001). Client `{"action":"subscribe","channel":"filings","tickers":[…]\|"all","forms":[…],"resume_token":<seq>}`;
	264	+server `hello`, `subscribed`, `filing` (`ticker, cik, form, period, filed_date, url, accn, seq, summary:{revenue,
	265	+net_income, eps_diluted, total_assets, operating_cash_flow, fiscal_year, fiscal_quarter, yoy:{…}}`), `heartbeat`
	266	+every 20 s, `pong`, `error`. Max 5 concurrent sockets per key (Redis counter `stream:conns:{principal}`, 6th →
	267	+`STREAM_CONNECTION_LIMIT`, close 4029). Buffer = Redis stream `filings:stream` (MAXLEN 1 000, ids `<seq>-0`,
	268	+`seq` from `INCR filings:seq`); `resume_token` replays everything after it. Delivery polls the stream every
	269	+250 ms per socket (also gives resume for free); the `filings` pub/sub channel is published for other consumers.
	270	+Full spec: `docs/asyncapi.yaml`.
	271	+
	272	+Authentication hook: `stream/auth.py` delegates to `accounts.security.{verify_api_key\|resolve_api_key\|
	273	+authenticate_key}(key)` when the accounts module is present; otherwise a key with the documented shape
	274	+`hfmd_live_<32 base62>` is accepted and the principal is `key:<sha256(salt+key)[:16]>`.
	275	+
	276	+Stream accounting interface (for the ratelimit module):
	277	+
	278	+```python
	279	+from stream import accounting
	280	+accounting.set_charger(fn) # fn(principal: str, rows: int) -> None ; called once per delivered `filing` message
	281	+accounting.charge(principal, rows=1) # what the socket calls (REDUCED_RATE_ROWS_PER_MESSAGE = 1)
	282	+accounting.rows_charged(principal) # default fallback counter: Redis `stream:rows:{principal}:{YYYY-MM-DD}` (24 h TTL)
	283	+```
	284	+
	285	+`principal` is the same `key:<id>` string the HTTP limiter uses, so stream rows can be folded into the rows
	286	+quota; heartbeats, hello/subscribed and errors are free.
	287	+
	288	+## 10. Tests and fixtures
	289	+
	290	+`tests/fixtures/edgar/` holds real, trimmed EDGAR documents (≈ 80 KB gzipped: companyfacts + submissions of
	291	+Apple, Microsoft, Shake Shack limited to mapped tags and filings since 2022, SHAK's MetaLinks tag list, one Atom
	292	+page, one daily index) recorded by `tests/fixtures/edgar/record.py`. `tests/conftest.py::edgar_mock` serves them
	293	+through `respx` — no test touches the network. The synthetic price lake gained the SHAK, GOOG and GOOGL tickers.
	294	+Suites: `test_fundamentals_unit.py` (mapping, calendar incl. Apple Q2 FY2024, resolution priority/units,
	295	+Q4 derivation, restatements/as_of, TTM, ratio formulas on Apple's real numbers, filter grammar, client backoff),
	296	+`test_fundamentals_api.py` (every endpoint in json/csv/parquet, envelopes, as_of, share classes),
	297	+`test_bulk.py` (ETag/304), `test_stream.py` (auth, subscribe, delivery, resume, heartbeat, connection limit,
	298	+accounting hook). `./.venv/bin/python -m pytest` → 53 passed.
	299	+
	300	+## 11. Limitations and next steps
	301	+
	302	+* Companyfacts only carries standard taxonomies: values reported exclusively through company extensions
	303	+ (SHAK cost lines, many bank/insurer line items) stay null with a reason; the `fund_mapping_log` hints are the
	304	+ input for the next mapping version (bump `MAPPING_VERSION`, re-run normalisation from the cached raw JSON).
	305	+* Dimensional facts (segments, share classes) are not in companyfacts; `shares_outstanding` sums the dei cover
	306	+ facts, which is right for market cap but not per class.
	307	+* `forward_pe` is always null (no consensus estimates). Growth/CAGR need enough history (null otherwise).
	308	+* Fiscal-year labels of Jan–Mar year ends follow the filer's convention learnt from its 10-Ks; a filer with no
	309	+ 10-K in the window falls back to the MMDD extrapolation.
	310	+* Restatement detection is value-based (tolerance 1e-6 relative / 0.5 absolute); a filer re-tagging the same
	311	+ number under a lower-priority concept does not create a version, a re-tag under a higher-priority concept does.
	312	+* 20-F/40-F filers are mapped through ifrs-full tags but were not part of the prototype; currency is kept, not
	313	+ converted.
	314	+* The ratelimit module must call `stream.accounting.set_charger` and honour `request.state.requires_key` /
	315	+ `request_cost` / `quota_exempt` set by these routes; the WebSocket itself is not rate-limited beyond the
	316	+ 5-connection cap.
	317