SPB Git
spboucher.ai

spb/firc0001_uqo

Public

N1: ledger reconciliation (gate pass / true call count, backfilled block, outage, collisions, logs, throughput)

Items: FIRC0001-EDIT-02 EDIT-03 EDIT-18 ADVE-03 ADVE-05 ADVE-07 AI_C-01 AI_C-07 DATA-02 DATA-03 DATA-14
ECON-11 ECON-12 THEO-02 THEO-05 THEO-16 EDIT-05 ADVE-06 AI_C-08 DATA-04 ECON-06 LITE-07 THEO-10 DATA-08 ADVE-12 EDIT-15.
Read-only DuckDB on M3U96b (12 s). 703,725 of 1,123,098 ledger rows pass the gate (true calls);
backfilled block 154,964 = 153,894 + 1,070 rows < 30 words; 0 ledger rows inside the 5.25 h outage gap;
304 sections / 412 collided mentions; 14 silent JSON-decode failures outside the outage plus one
anomalous post-resume batch (2,686 calls, 23 productive vs 536 expected); 149,352 reconciled from the log.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Nov 5, 2026) parent 7680606 Browse files

22 changed files +3,078 −0

22 changed files
Load all 22 files Unified Split
added code/audit/a01_ledger_reconciliation.py +374 −0
@@ -0,0 +1,374 @@
1+"""N1 - ledger reconciliation (clusters C, G, J-throughput, H-log numbers). Read-only.
2+
3+(a) _PROJECT_GATE over every ledger section (DuckDB RE2, case-insensitive) -> gate pass and the
4+ true call count by form x item x run; productive rate / mentions / USD per call.
5+(b) backfilled block (rows at min(extracted_at)), word_count < 30 rows, outage-window rows,
6+ zero-mention share per flush timestamp and per 5-minute bin.
7+(c) sections where ledger n_mentions > stored mentions (id collisions).
8+(d) run-log parser (October data/logs/spid_extract.log, June project-db/logs/extract.log).
9+(e) wall-clock / outage / effective throughput from ledger timestamps.
10+(f) backend values, confidence = 0.600 count, amount_raw formatting rule, min confidence by run.
11+"""
12+from __future__ import annotations
13+import json, re, sys, datetime as dt
14+from pathlib import Path
15+from collections import Counter, defaultdict
16+sys.path.insert(0, str(Path(__file__).resolve().parent))
17+from common import *
18+
19+GATE = (r"\b(project|initiative|program|invest|expan|construct|build|launch|develop|"
20+ r"deploy|implement|acqui|partner|facilit|capacity|capital|migrat|transform|"
21+ r"pipeline|plant|data ?cent|automat|sustainab|renewable|restructur)\w*")
22+OUTAGE_START = "2026-10-28 01:10:00" # window named in the consolidated review (THEO-05 / DATA-02)
23+OUTAGE_END = "2026-10-28 06:22:00"
24+PAPER_USD_TOTAL = 775.0 # construct.tex:35 'roughly 775 USD'
25+PAPER_USD_PER_SECTION = 0.0008 # construct.tex:35
26+JUNE_LOG_SECTIONS = 153894 # project-db/logs/extract.log FULL EXTRACT COMPLETE
27+JUNE_LOG_MENTIONS = 39930 # TOTAL mentions in DB after the June run
28+
29+r = Run("ledger")
30+con = connect(SPID_DB)
31+
32+# ---- (a) section metadata + gate flag for every section visible to the pipeline --------------
33+print("building sec_meta (regex over the parquet text, streamed) ...", flush=True)
34+con.execute(f"""
35+CREATE TEMP TABLE sec_meta AS
36+SELECT section_id, accession_number, section_name, word_count, length(text) AS nchar,
37+ regexp_matches(text, '{GATE}', 'i') AS gate, 'parsed' AS segmenter
38+FROM src_sections
39+UNION ALL
40+SELECT section_id, accession_number, section_name, word_count, length(text) AS nchar,
41+ regexp_matches(text, '{GATE}', 'i') AS gate, 'legacy_ingest' AS segmenter
42+FROM spid_sections
43+""")
44+r.notes["sec_meta_rows"] = con.execute("select count(*) from sec_meta").fetchone()[0]
45+
46+con.execute("""
47+CREATE TEMP TABLE stored AS
48+SELECT section_id, count(*) AS stored_mentions,
49+ list(project_type || ':' || left(project_name, 40)) AS stored_keys
50+FROM project_mentions GROUP BY 1
51+""")
52+tmin = con.execute("select min(extracted_at) from extracted_sections").fetchone()[0]
53+r.notes["min_extracted_at"] = str(tmin)
54+con.execute(f"""
55+CREATE TEMP TABLE ledger AS
56+SELECT e.section_id, e.accession_number, e.backend, e.n_mentions, e.extracted_at,
57+ CASE WHEN e.extracted_at = TIMESTAMP '{tmin}' THEN 'backfilled_june' ELSE 'october' END AS run,
58+ m.section_name, m.word_count, m.nchar, m.gate, m.segmenter,
59+ f.form_type, f.filing_date, f.cik, f.fiscal_year,
60+ co.in_sp500, co.in_russell3000,
61+ coalesce(s.stored_mentions, 0) AS stored_mentions, s.stored_keys
62+FROM extracted_sections e
63+LEFT JOIN sec_meta m USING (section_id)
64+LEFT JOIN src_filings f ON f.accession_number = e.accession_number
65+LEFT JOIN src_companies co ON f.cik = co.cik
66+LEFT JOIN stored s USING (section_id)
67+""")
68+n_ledger = con.execute("select count(*) from ledger").fetchone()[0]
69+n_unmatched = con.execute("select count(*) from ledger where section_name is null").fetchone()[0]
70+r.notes["ledger_rows"] = n_ledger
71+r.notes["ledger_rows_without_section_metadata"] = n_unmatched
72+
73+agg = """
74+ count(*) AS ledger_rows,
75+ count(*) FILTER (WHERE word_count < 30) AS lt30_words,
76+ count(*) FILTER (WHERE gate) AS gate_pass,
77+ count(*) FILTER (WHERE gate AND word_count >= 30) AS calls,
78+ count(*) FILTER (WHERE NOT gate AND n_mentions = 0) AS gate_rejected_zero,
79+ count(*) FILTER (WHERE NOT gate AND n_mentions > 0) AS gate_fail_but_mentions,
80+ count(*) FILTER (WHERE n_mentions > 0) AS with_mentions,
81+ sum(n_mentions) AS ledger_mentions,
82+ sum(stored_mentions) AS stored_mentions,
83+ count(DISTINCT cik) AS issuers,
84+ round(100.0 * count(*) FILTER (WHERE n_mentions > 0) / count(*), 2) AS productive_pct_ledger,
85+ round(100.0 * count(*) FILTER (WHERE n_mentions > 0) / nullif(count(*) FILTER (WHERE gate AND word_count >= 30),0), 2) AS productive_pct_calls,
86+ round(sum(n_mentions) * 1.0 / count(*), 4) AS mentions_per_ledger_row,
87+ round(sum(n_mentions) * 1.0 / nullif(count(*) FILTER (WHERE gate AND word_count >= 30),0), 4) AS mentions_per_call
88+"""
89+r.dump("gate_pass_by_form_item.csv", con, f"SELECT form_type, section_name AS item, run, {agg} FROM ledger GROUP BY 1,2,3 ORDER BY 1,2,3")
90+r.dump("gate_pass_by_form.csv", con, f"SELECT form_type, {agg} FROM ledger GROUP BY 1 ORDER BY 1")
91+r.dump("gate_pass_by_run.csv", con, f"SELECT run, {agg} FROM ledger GROUP BY 1 ORDER BY 1")
92+r.dump("gate_pass_by_form_run.csv", con, f"SELECT form_type, run, {agg} FROM ledger GROUP BY 1,2 ORDER BY 1,2")
93+tot = con.execute(f"SELECT {agg} FROM ledger").fetchone()
94+cols = ["ledger_rows","lt30_words","gate_pass","calls","gate_rejected_zero","gate_fail_but_mentions","with_mentions","ledger_mentions","stored_mentions","issuers","productive_pct_ledger","productive_pct_calls","mentions_per_ledger_row","mentions_per_call"]
95+T = dict(zip(cols, tot))
96+oct_calls = con.execute("select count(*) filter (where gate and word_count>=30) from ledger where run='october'").fetchone()[0]
97+oct_rows = con.execute("select count(*) from ledger where run='october'").fetchone()[0]
98+T["usd_per_call_paper_total_775"] = round(PAPER_USD_TOTAL / oct_calls, 6)
99+T["usd_per_october_row_paper_total_775"] = round(PAPER_USD_TOTAL / oct_rows, 6)
100+T["usd_per_call_from_0_0008_x_ledger"] = round(PAPER_USD_PER_SECTION * n_ledger / T["calls"], 6)
101+T["october_rows"] = oct_rows; T["october_calls"] = oct_calls
102+T["headline_before"] = {"sections_submitted": 1123098, "productive_pct": 15.0, "mentions_per_section": 0.27, "usd_per_section": 0.0008}
103+r.write_json("gate_pass_totals.json", T)
104+
105+# ---- (b) backfilled block, < 30 words, outage window, flush timestamps -----------------------
106+B = {}
107+B["min_extracted_at"] = str(tmin)
108+B["rows_at_min_timestamp"] = con.execute("select count(*) from ledger where run='backfilled_june'").fetchone()[0]
109+B["june_log_sections"] = JUNE_LOG_SECTIONS
110+B["gap_rows_minus_log"] = B["rows_at_min_timestamp"] - JUNE_LOG_SECTIONS
111+B["block_ledger_mentions"] = con.execute("select sum(n_mentions) from ledger where run='backfilled_june'").fetchone()[0]
112+B["block_stored_mentions"] = con.execute("select sum(stored_mentions) from ledger where run='backfilled_june'").fetchone()[0]
113+B["june_log_mentions"] = JUNE_LOG_MENTIONS
114+B["block_lt30_words"] = con.execute("select count(*) from ledger where run='backfilled_june' and word_count<30").fetchone()[0]
115+B["block_gate_fail"] = con.execute("select count(*) from ledger where run='backfilled_june' and not gate").fetchone()[0]
116+B["block_lt30_or_gate_fail"] = con.execute("select count(*) from ledger where run='backfilled_june' and (word_count<30 or not gate)").fetchone()[0]
117+B["block_ge30_and_gate"] = con.execute("select count(*) from ledger where run='backfilled_june' and word_count>=30 and gate").fetchone()[0]
118+B["october_lt30_words"] = con.execute("select count(*) from ledger where run='october' and word_count<30").fetchone()[0]
119+B["block_by_form"] = con.execute("select form_type, count(*), sum(n_mentions), count(*) filter (where word_count<30) from ledger where run='backfilled_june' group by 1 order by 1").fetchall()
120+B["block_by_segmenter"] = con.execute("select segmenter, count(*) from ledger where run='backfilled_june' group by 1").fetchall()
121+B["block_sp500_share"] = con.execute("select count(*) filter (where in_sp500), count(*) from ledger where run='backfilled_june'").fetchone()
122+B["block_ciks"] = con.execute("select count(distinct cik) from ledger where run='backfilled_june'").fetchone()[0]
123+B["outage_window"] = [OUTAGE_START, OUTAGE_END]
124+B["rows_in_outage_window"] = con.execute(f"select count(*), sum(n_mentions) from ledger where extracted_at between TIMESTAMP '{OUTAGE_START}' and TIMESTAMP '{OUTAGE_END}'").fetchone()
125+B["rows_0110_0640"] = con.execute("select count(*), sum(n_mentions) from ledger where extracted_at between TIMESTAMP '2026-10-28 01:10:00' and TIMESTAMP '2026-10-28 06:40:53'").fetchone()
126+B["rows_by_hour_1028"] = con.execute("select hour(extracted_at) h, count(*), count(*) filter (where n_mentions=0) zeros, sum(n_mentions) from ledger where run='october' and extracted_at::date = DATE '2026-10-28' group by 1 order by 1").fetchall()
127+B["rows_by_hour_1027"] = con.execute("select hour(extracted_at) h, count(*), count(*) filter (where n_mentions=0) zeros, sum(n_mentions) from ledger where run='october' and extracted_at::date = DATE '2026-10-27' group by 1 order by 1").fetchall()
128+B["paper_outage_rows_149352"] = 149352
129+r.write_json("backfill_block.json", B)
130+
131+r.dump("ledger_by_5min.csv", con, """
132+SELECT time_bucket(INTERVAL '5 minutes', extracted_at) AS bin5, count(*) AS rows, count(*) FILTER (WHERE n_mentions>0) AS with_mentions,
133+ sum(n_mentions) AS mentions, round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
134+ count(*) FILTER (WHERE gate AND word_count>=30) AS calls, count(DISTINCT extracted_at) AS flushes
135+FROM ledger WHERE run='october' GROUP BY 1 ORDER BY 1""")
136+r.dump("ledger_by_minute.csv", con, """
137+SELECT time_bucket(INTERVAL '1 minute', extracted_at) AS bin1, count(*) AS rows, count(*) FILTER (WHERE n_mentions>0) AS with_mentions,
138+ sum(n_mentions) AS mentions, round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
139+ count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
140+ round(100.0*count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30)/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0),2) AS zero_pct_among_calls
141+FROM ledger WHERE run='october' GROUP BY 1 ORDER BY 1""")
142+B["october_distinct_timestamps"], B["october_rows_check"] = con.execute("select count(distinct extracted_at), count(*) from ledger where run='october'").fetchone()
143+B["timestamp_granularity_note"] = "extracted_at is stamped per row (executemany INSERT OR REPLACE, DEFAULT current_timestamp), not per flush: distinct timestamps = rows"
144+r.dump("flush_zero_groups_after_0622.csv", con, f"""
145+SELECT time_bucket(INTERVAL '1 minute', extracted_at) AS bin1, count(*) AS rows, count(*) FILTER (WHERE n_mentions=0) AS zeros,
146+ round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
147+ count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
148+ round(100.0*count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30)/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0),2) AS zero_pct_among_calls
149+FROM ledger WHERE run='october' AND extracted_at > TIMESTAMP '{OUTAGE_END}'
150+GROUP BY 1 HAVING count(*) >= 50 AND count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30) >= 0.99*count(*) FILTER (WHERE gate AND word_count>=30) ORDER BY 1""")
151+# the same test before the outage, for comparison
152+r.dump("flush_zero_groups_before_0110.csv", con, f"""
153+SELECT time_bucket(INTERVAL '1 minute', extracted_at) AS bin1, count(*) AS rows, count(*) FILTER (WHERE n_mentions=0) AS zeros,
154+ round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
155+ count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
156+ round(100.0*count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30)/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0),2) AS zero_pct_among_calls
157+FROM ledger WHERE run='october' AND extracted_at < TIMESTAMP '{OUTAGE_START}'
158+GROUP BY 1 HAVING count(*) >= 50 AND count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30) >= 0.99*count(*) FILTER (WHERE gate AND word_count>=30) ORDER BY 1""")
159+
160+# ---- (c) id collisions: ledger n_mentions > stored ------------------------------------------
161+r.dump("ledger_collisions.csv", con, """
162+SELECT section_id, accession_number, form_type, section_name AS item, cik, run, n_mentions AS ledger_n, stored_mentions AS stored_n,
163+ n_mentions - stored_mentions AS deficit, stored_keys
164+FROM ledger WHERE n_mentions > stored_mentions ORDER BY deficit DESC, section_id""")
165+r.dump("ledger_collisions_by_form.csv", con, """
166+SELECT form_type, count(*) AS sections, sum(n_mentions - stored_mentions) AS excess, sum(n_mentions) AS ledger_mentions, sum(stored_mentions) AS stored_mentions
167+FROM ledger WHERE n_mentions > stored_mentions GROUP BY 1 ORDER BY 1""")
168+C = {}
169+C["sections_ledger_gt_stored"], C["excess_mentions"] = con.execute("select count(*), sum(n_mentions-stored_mentions) from ledger where n_mentions > stored_mentions").fetchone()
170+C["sections_ledger_lt_stored"], C["deficit_mentions"] = con.execute("select count(*), sum(stored_mentions-n_mentions) from ledger where n_mentions < stored_mentions").fetchone()
171+C["sum_ledger_n_mentions"] = con.execute("select sum(n_mentions) from ledger").fetchone()[0]
172+C["stored_project_mentions"] = con.execute("select count(*) from project_mentions").fetchone()[0]
173+C["mentions_whose_section_not_in_ledger"] = con.execute("select count(*) from project_mentions m where not exists (select 1 from extracted_sections e where e.section_id=m.section_id)").fetchone()[0]
174+C["ledger_vs_stored_by_form"] = con.execute("select form_type, sum(n_mentions), sum(stored_mentions), sum(n_mentions)-sum(stored_mentions) from ledger group by 1 order by 1").fetchall()
175+C["duplicate_key_sections_in_stored"] = con.execute("""
176+ select count(*) from (select section_id from project_mentions group by section_id, project_type, left(project_name,40) having count(*)>1)""").fetchone()[0]
177+r.write_json("collisions.json", C)
178+
179+# ---- (f) misc code-vs-data checks ---------------------------------------------------------------
180+F = {}
181+F["distinct_backend_ledger"] = [x[0] for x in con.execute("select distinct backend from extracted_sections").fetchall()]
182+F["distinct_backend_mentions"] = [x[0] for x in con.execute("select distinct backend from project_mentions").fetchall()]
183+F["confidence_exact_0600_total"] = con.execute("select count(*) from project_mentions where round(confidence,3)=0.600").fetchone()[0]
184+F["confidence_exact_0600_by_run"] = con.execute("""
185+ select l.run, count(*) from project_mentions m join ledger l using(section_id) where round(m.confidence,3)=0.600 group by 1""").fetchall()
186+F["confidence_bins_by_run"] = con.execute("""
187+ select l.run, round(m.confidence,1) c, count(*) from project_mentions m join ledger l using(section_id) group by 1,2 order by 1,2""").fetchall()
188+F["min_confidence_by_run"] = con.execute("select l.run, min(m.confidence), max(m.confidence), count(*) from project_mentions m join ledger l using(section_id) group by 1").fetchall()
189+F["mentions_below_045"] = con.execute("select count(*) from project_mentions where confidence < 0.45").fetchone()[0]
190+F["amount_raw_rule_mismatch"] = con.execute("""
191+ select count(*) from project_mentions where amount_usd is not null and amount_raw is distinct from ('$' || format('{:,.0f}', amount_usd))""").fetchone()[0]
192+F["amount_raw_rule_mismatch_rows"] = con.execute("""
193+ select mention_id, amount_usd, amount_raw from project_mentions where amount_usd is not null and amount_raw is distinct from ('$' || format('{:,.0f}', amount_usd)) limit 20""").fetchall()
194+F["amount_raw_null_with_amount"] = con.execute("select count(*) from project_mentions where amount_usd is not null and amount_raw is null").fetchone()[0]
195+F["amount_raw_notnull_without_amount"] = con.execute("select count(*) from project_mentions where amount_usd is null and amount_raw is not null").fetchone()[0]
196+F["amount_le_0"] = con.execute("select count(*) from project_mentions where amount_usd <= 0").fetchone()[0]
197+F["mentions_with_run"] = con.execute("select l.run, count(*) from project_mentions m join ledger l using(section_id) group by 1").fetchall()
198+F["mentions_backfilled_block_vs_june_log"] = {"block_stored": B["block_stored_mentions"], "june_log": JUNE_LOG_MENTIONS}
199+F["fiscal_year_null_mentions_by_run"] = con.execute("select l.run, count(*) filter (where m.fiscal_year is null), count(*) from project_mentions m join ledger l using(section_id) group by 1").fetchall()
200+r.write_json("code_vs_data_checks.json", F)
201+
202+# ---- (e) throughput from the ledger ----------------------------------------------------------
203+E = {}
204+ts = con.execute("select distinct extracted_at from ledger where run='october' order by 1").fetchall()
205+ts = [t[0] for t in ts]
206+E["october_first_flush"] = str(ts[0]); E["october_last_flush"] = str(ts[-1])
207+E["october_distinct_flush_timestamps"] = len(ts)
208+E["october_rows"] = oct_rows
209+E["october_calls"] = oct_calls
210+wall = (ts[-1] - ts[0]).total_seconds()
211+gaps = sorted(((ts[i+1]-ts[i]).total_seconds(), str(ts[i]), str(ts[i+1])) for i in range(len(ts)-1))
212+E["largest_gaps_seconds"] = gaps[-5:]
213+outage_gap = gaps[-1][0]
214+E["wall_clock_hours_first_to_last_flush"] = round(wall/3600, 3)
215+E["largest_gap_hours"] = round(outage_gap/3600, 3)
216+E["effective_hours_excluding_largest_gap"] = round((wall-outage_gap)/3600, 3)
217+E["rows_per_second_wall"] = round(oct_rows/wall, 2)
218+E["rows_per_second_effective"] = round(oct_rows/(wall-outage_gap), 2)
219+E["calls_per_second_effective"] = round(oct_calls/(wall-outage_gap), 2)
220+E["paper_claim"] = {"sections": 968134, "hours": "about twelve", "rate": "27-36 sections per second", "workers": 48}
221+E["paper_rate_implied_by_12h"] = round(968134/(12*3600), 2)
222+# segment rates: before the outage and after
223+pre = con.execute(f"select count(*), min(extracted_at), max(extracted_at) from ledger where run='october' and extracted_at < TIMESTAMP '{OUTAGE_START}'").fetchone()
224+post = con.execute(f"select count(*), min(extracted_at), max(extracted_at) from ledger where run='october' and extracted_at > TIMESTAMP '{OUTAGE_END}'").fetchone()
225+E["segment_before_outage"] = {"rows": pre[0], "from": str(pre[1]), "to": str(pre[2]), "rows_per_s": round(pre[0]/max(1,(pre[2]-pre[1]).total_seconds()),2)}
226+E["segment_after_outage"] = {"rows": post[0], "from": str(post[1]), "to": str(post[2]), "rows_per_s": round(post[0]/max(1,(post[2]-post[1]).total_seconds()),2)}
227+E["rows_by_hour_october"] = con.execute("select date_trunc('hour', extracted_at) h, count(*) from ledger where run='october' group by 1 order by 1").fetchall()
228+r.write_json("throughput.json", E)
229+
230+# ---- (d) log parsers ---------------------------------------------------------------------------
231+LINE = re.compile(r"^(\d\d):(\d\d):(\d\d) (\w+)\s+(\S+) \| (.*)$")
232+def parse_log(path: Path, start_date: dt.date):
233+ day = start_date; prev = None; out = []
234+ for raw in open(path, errors="replace"):
235+ m = LINE.match(raw.rstrip("\n"))
236+ if not m:
237+ out.append((None, None, None, raw.rstrip("\n"))); continue
238+ h, mi, s = int(m.group(1)), int(m.group(2)), int(m.group(3))
239+ t = dt.time(h, mi, s)
240+ if prev is not None and (dt.datetime.combine(day, prev) - dt.datetime.combine(day, t)).total_seconds() > 3600:
241+ day = day + dt.timedelta(days=1)
242+ prev = t
243+ out.append((dt.datetime.combine(day, t), m.group(4), m.group(5), m.group(6)))
244+ return out
245+
246+def classify(msg: str) -> str:
247+ if "retryable" in msg: return "retryable_http_429_5xx"
248+ if "Connection reset" in msg or "ConnectError" in msg or "ReadError" in msg or "Errno" in msg: return "connection_error"
249+ if "Unterminated string" in msg or "Expecting" in msg or "Extra data" in msg or "JSON" in msg: return "json_decode_error"
250+ if "KeyError" in msg: return "key_error"
251+ if "timed out" in msg.lower() or "Timeout" in msg: return "timeout"
252+ return "other"
253+
254+def summarize(path: Path, start_date: dt.date, window=None):
255+ rows = parse_log(path, start_date)
256+ S = {"path": str(path), "lines": len(rows), "first": str(next((r_[0] for r_ in rows if r_[0]), None)), "last": str(next((r_[0] for r_ in reversed(rows) if r_[0]), None))}
257+ fails = [(t, classify(msg)) for t, lvl, name, msg in rows if msg and "LLM call failed" in msg]
258+ S["llm_call_failed_total"] = len(fails)
259+ S["llm_call_failed_by_reason"] = dict(Counter(c for _, c in fails))
260+ byhour = Counter((t.strftime("%Y-%m-%d %H") if t else "?") for t, _ in fails)
261+ S["llm_call_failed_by_hour"] = dict(sorted(byhour.items()))
262+ if window:
263+ w0, w1 = window
264+ inside = [(t, c) for t, c in fails if t and w0 <= t <= w1]
265+ outside = [(t, c) for t, c in fails if not (t and w0 <= t <= w1)]
266+ S["window"] = [str(w0), str(w1)]
267+ S["llm_call_failed_inside_window"] = len(inside)
268+ S["llm_call_failed_outside_window"] = len(outside)
269+ S["outside_window_by_reason"] = dict(Counter(c for _, c in outside))
270+ S["outside_window_list"] = [(str(t), c) for t, c in outside]
271+ if fails:
272+ S["first_failure"] = str(min(t for t, _ in fails if t)); S["last_failure"] = str(max(t for t, _ in fails if t))
273+ ins = [t for t, _ in inside]
274+ if ins: S["first_failure_in_window"] = str(min(ins)); S["last_failure_in_window"] = str(max(ins))
275+ S["section_failed"] = sum(1 for t, lvl, name, msg in rows if msg and "section failed" in msg)
276+ S["batch_failed_lines"] = sum(1 for t, lvl, name, msg in rows if msg and "BATCH" in msg and "FAILED" in msg)
277+ S["pending_lines"] = [(str(t), msg) for t, lvl, name, msg in rows if msg and msg.startswith("pending:")]
278+ S["ciks_total_lines"] = [(str(t), msg) for t, lvl, name, msg in rows if msg and msg.startswith("CIKs total")]
279+ S["run_complete_lines"] = [(str(t), msg) for t, lvl, name, msg in rows if msg and ("RUN COMPLETE" in msg or "EXTRACT COMPLETE" in msg)]
280+ ext = [int(re.search(r"Extracting from (\d+) sections", msg).group(1)) for t, lvl, name, msg in rows if msg and "Extracting from" in msg]
281+ S["extracting_from_lines"] = len(ext); S["extracting_from_sum_sections"] = sum(ext)
282+ done = [(int(a), int(b)) for t, lvl, name, msg in rows for a, b in re.findall(r"Extraction complete: (\d+) mentions from (\d+) sections", msg or "")]
283+ S["extraction_complete_lines"] = len(done); S["extraction_complete_sum_sections"] = sum(b for a, b in done); S["extraction_complete_sum_mentions"] = sum(a for a, b in done)
284+ S["workers_values"] = sorted(set(re.findall(r"workers=(\d+)", "\n".join(msg or "" for *_, msg in rows))))
285+ S["batch_values"] = sorted(set(re.findall(r"batch=(\d+)", "\n".join(msg or "" for *_, msg in rows))))
286+ S["model_id_strings"] = sorted(set(re.findall(r"gpt-[\w\-\.]+", open(path, errors="replace").read())))
287+ return S, fails
288+
289+w0 = dt.datetime.fromisoformat(OUTAGE_START); w1 = dt.datetime.fromisoformat(OUTAGE_END)
290+oct_log, oct_fails = summarize(RDB / "data/logs/spid_extract.log", dt.date(2026, 10, 27), (w0, w1))
291+jun_log, jun_fails = summarize(RDB / "project-db/logs/extract.log", dt.date(2026, 6, 18), None)
292+# other October log files (small)
293+others = {}
294+for nm in ["spid.log", "spid_resume.log", "spid_post.log", "spid_interim.log"]:
295+ p = RDB / "data/logs" / nm
296+ if p.exists():
297+ txt = open(p, errors="replace").read()
298+ others[nm] = {"lines": txt.count("\n"), "head": txt[:1500]}
299+pend = RDB / "data/logs/spid_pending.json"
300+if pend.exists():
301+ try:
302+ pj = json.load(open(pend)); others["spid_pending.json"] = {k: (v if not isinstance(v, list) else v[:20]) for k, v in pj.items()} if isinstance(pj, dict) else str(pj)[:1500]
303+ except Exception as ex:
304+ others["spid_pending.json"] = str(ex)
305+# outage window measured from the log itself: the contiguous block of 'retryable' failures on 2026-10-28 between 01:00 and 07:00
306+retry_ts = [t for t, c in oct_fails if t and c == "retryable_http_429_5xx" and dt.datetime(2026,10,28,1,0) <= t <= dt.datetime(2026,10,28,7,0)]
307+oct_log["outage_window_from_log"] = [str(min(retry_ts)), str(max(retry_ts)), len(retry_ts)]
308+w0l, w1l = min(retry_ts), max(retry_ts)
309+oct_log["llm_call_failed_outside_log_window"] = [(str(t), c) for t, c in oct_fails if not (t and w0l <= t <= w1l)]
310+oct_log["silent_empty_results_outside_outage"] = len(oct_log["llm_call_failed_outside_log_window"])
311+oct_log["retryable_failures_outside_log_window"] = sum(1 for t, c in oct_fails if c == "retryable_http_429_5xx" and not (t and w0l <= t <= w1l))
312+oct_log["ledger_rows_in_log_window"] = con.execute(f"select count(*), sum(n_mentions) from ledger where extracted_at between TIMESTAMP '{w0l}' and TIMESTAMP '{w1l}'").fetchone()
313+oct_log["ledger_rows_in_largest_gap"] = con.execute(f"select count(*) from ledger where extracted_at > TIMESTAMP '{gaps[-1][1]}' and extracted_at < TIMESTAMP '{gaps[-1][2]}'").fetchone()[0]
314+pend2 = int(re.search(r"pending: (\d+)", oct_log["pending_lines"][1][1]).group(1)) if len(oct_log["pending_lines"]) > 1 else None
315+oct_log["outage_rows_reconciliation"] = {"extraction_complete_sum_sections": oct_log["extraction_complete_sum_sections"], "pending_at_second_start": pend2,
316+ "difference": (oct_log["extraction_complete_sum_sections"] - pend2) if pend2 else None, "paper_research_log_149352": 149352,
317+ "pending_at_resume": int(re.search(r"pending: (\d+)", oct_log["pending_lines"][-1][1]).group(1)),
318+ "rows_before_outage_plus_in_window": con.execute(f"select count(*) from ledger where run='october' and extracted_at < TIMESTAMP '{gaps[-1][2]}'").fetchone()[0]}
319+# batch-level productivity from the log (silent-failure test, THEO-05): mentions per section per batch
320+def batch_rows(rows_, run_):
321+ out = []
322+ for t, lvl, name, msg in rows_:
323+ m_ = re.search(r"BATCH (\d+)/(\d+) done \| batch_sec=(\d+) batch_men=(\d+)", msg or "")
324+ if m_:
325+ out.append((run_, int(m_.group(1)), int(m_.group(2)), str(t), int(m_.group(3)), int(m_.group(4)), round(int(m_.group(4))/max(1,int(m_.group(3))), 4)))
326+ return out
327+oct_rows_ = parse_log(RDB / "data/logs/spid_extract.log", dt.date(2026, 10, 27)); jun_rows_ = parse_log(RDB / "project-db/logs/extract.log", dt.date(2026, 6, 18))
328+br = batch_rows(oct_rows_, "october") + batch_rows(jun_rows_, "june")
329+import statistics
330+med_oct = statistics.median([b[6] for b in br if b[0] == "october"]); med_jun = statistics.median([b[6] for b in br if b[0] == "june"])
331+br2 = [b + (round(b[6]/(med_oct if b[0]=="october" else med_jun), 3), (b[6] < 0.25*(med_oct if b[0]=="october" else med_jun))) for b in br]
332+r.write_csv("batch_productivity_from_log.csv", ["run", "batch", "n_batches", "done_at", "sections", "mentions", "mentions_per_section", "ratio_to_median", "anomalous_lt_25pct_of_median"], br2)
333+oct_log["batch_productivity_median_mentions_per_section"] = med_oct
334+oct_log["anomalous_batches"] = [b for b in br2 if b[0] == "october" and b[8]]
335+jun_log["batch_productivity_median_mentions_per_section"] = med_jun
336+jun_log["anomalous_batches"] = [b for b in br2 if b[0] == "june" and b[8]]
337+# the first resumed batch: rows stamped between its start and end, with the expected productive count at the form x item rates of the rest of the October run
338+b1 = [b for b in br2 if b[0] == "october" and b[1] == 1 and b[2] == 81]
339+if b1:
340+ b1_end = b1[0][3]
341+ b1_start = str(dt.datetime.fromisoformat(oct_log["pending_lines"][-1][0]))
342+ con.execute(f"CREATE TEMP TABLE b1 AS SELECT * FROM ledger WHERE run='october' AND extracted_at BETWEEN TIMESTAMP '{b1_start}' AND TIMESTAMP '{b1_end}'")
343+ r.dump("resume_batch1_vs_expected.csv", con, """
344+ WITH rates AS (SELECT form_type, section_name, count(*) FILTER (WHERE n_mentions>0)*1.0/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0) p_prod, sum(n_mentions)*1.0/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0) m_per_call
345+ FROM ledger WHERE run='october' AND section_id NOT IN (SELECT section_id FROM b1) GROUP BY 1,2)
346+ SELECT b.form_type, b.section_name AS item, count(*) rows, count(*) FILTER (WHERE gate AND word_count>=30) calls, count(*) FILTER (WHERE n_mentions>0) with_mentions, sum(n_mentions) mentions,
347+ round(sum(CASE WHEN gate AND word_count>=30 THEN r.p_prod ELSE 0 END),1) expected_with_mentions, round(sum(CASE WHEN gate AND word_count>=30 THEN r.m_per_call ELSE 0 END),1) expected_mentions,
348+ count(DISTINCT b.cik) ciks
349+ FROM b1 b LEFT JOIN rates r ON b.form_type=r.form_type AND b.section_name=r.section_name GROUP BY 1,2 ORDER BY 1,2""")
350+ oct_log["resume_batch1"] = {"window": [b1_start, b1_end], "log_sections": b1[0][4], "log_mentions": b1[0][5],
351+ "ledger_rows": con.execute("select count(*) from b1").fetchone()[0],
352+ "ledger_calls": con.execute("select count(*) from b1 where gate and word_count>=30").fetchone()[0],
353+ "ledger_with_mentions": con.execute("select count(*) from b1 where n_mentions>0").fetchone()[0],
354+ "ledger_mentions": con.execute("select sum(n_mentions) from b1").fetchone()[0],
355+ "ciks": con.execute("select list(distinct cik order by cik) from b1").fetchone()[0],
356+ "expected_with_mentions": con.execute("""WITH rates AS (SELECT form_type, section_name, count(*) FILTER (WHERE n_mentions>0)*1.0/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0) p FROM ledger WHERE run='october' AND section_id NOT IN (SELECT section_id FROM b1) GROUP BY 1,2)
357+ SELECT round(sum(CASE WHEN gate AND word_count>=30 THEN r.p ELSE 0 END),1) FROM b1 b LEFT JOIN rates r ON b.form_type=r.form_type AND b.section_name=r.section_name""").fetchone()[0],
358+ "failed_call_lines_in_window": sum(1 for t, c in oct_fails if t and dt.datetime.fromisoformat(b1_start) <= t <= dt.datetime.fromisoformat(b1_end))}
359+r.write_json("log_summary.json", {"october": oct_log, "june": jun_log, "other_logs": others,
360+ "june_log_date_assumed": "2026-06-18 (file mtime; the log carries time-of-day only)"})
361+r.write_csv("log_failures_by_hour.csv", ["run", "hour", "reason", "n"],
362+ [("october", h, c, n) for (h, c), n in sorted(Counter((t.strftime("%Y-%m-%d %H") if t else "?", c) for t, c in oct_fails).items())] +
363+ [("june", h, c, n) for (h, c), n in sorted(Counter((t.strftime("%Y-%m-%d %H") if t else "?", c) for t, c in jun_fails).items())])
364+
365+# ---- Table 3 replacement input ---------------------------------------------------------------
366+r.dump("table3_ledger_vs_stored.csv", con, """
367+SELECT form_type, count(*) AS ledger_rows, count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
368+ count(*) FILTER (WHERE n_mentions>0) AS with_mentions, sum(n_mentions) AS returned_by_model_before_dedup,
369+ sum(stored_mentions) AS stored_mentions, count(DISTINCT cik) AS issuers,
370+ count(DISTINCT cik) FILTER (WHERE n_mentions>0) AS issuers_with_mentions
371+FROM ledger GROUP BY 1 ORDER BY 1""")
372+r.notes["issuers_total_distinct_in_ledger"] = con.execute("select count(distinct cik) from ledger").fetchone()[0]
373+r.notes["issuers_with_filings"] = con.execute("select count(distinct cik) from src_filings").fetchone()[0]
374+r.finish()
added results/ledger/backfill_block.json +132 −0
@@ -0,0 +1,132 @@
1+{
2+ "min_extracted_at": "2026-10-27 13:12:32.140400",
3+ "rows_at_min_timestamp": 154964,
4+ "june_log_sections": 153894,
5+ "gap_rows_minus_log": 1070,
6+ "block_ledger_mentions": 39930,
7+ "block_stored_mentions": 39930,
8+ "june_log_mentions": 39930,
9+ "block_lt30_words": 1070,
10+ "block_gate_fail": 59666,
11+ "block_lt30_or_gate_fail": 59678,
12+ "block_ge30_and_gate": 95286,
13+ "october_lt30_words": 0,
14+ "block_by_form": [
15+ [
16+ "10-K",
17+ 27164,
18+ 15787,
19+ 0
20+ ],
21+ [
22+ "10-Q",
23+ 23873,
24+ 5817,
25+ 1069
26+ ],
27+ [
28+ "8-K",
29+ 103927,
30+ 18326,
31+ 1
32+ ]
33+ ],
34+ "block_by_segmenter": [
35+ [
36+ "parsed",
37+ 127800
38+ ],
39+ [
40+ "legacy_ingest",
41+ 27164
42+ ]
43+ ],
44+ "block_sp500_share": [
45+ 154964,
46+ 154964
47+ ],
48+ "block_ciks": 500,
49+ "outage_window": [
50+ "2026-10-28 01:10:00",
51+ "2026-10-28 06:22:00"
52+ ],
53+ "rows_in_outage_window": [
54+ 504,
55+ 1011
56+ ],
57+ "rows_0110_0640": [
58+ 5273,
59+ 1041
60+ ],
61+ "rows_by_hour_1028": [
62+ [
63+ 0,
64+ 107883,
65+ 92099,
66+ 30321
67+ ],
68+ [
69+ 1,
70+ 16609,
71+ 12953,
72+ 7061
73+ ],
74+ [
75+ 6,
76+ 35147,
77+ 31638,
78+ 7092
79+ ],
80+ [
81+ 7,
82+ 148277,
83+ 125987,
84+ 41887
85+ ],
86+ [
87+ 8,
88+ 114988,
89+ 98154,
90+ 32310
91+ ],
92+ [
93+ 9,
94+ 130711,
95+ 111241,
96+ 36466
97+ ],
98+ [
99+ 10,
100+ 93253,
101+ 77492,
102+ 29917
103+ ],
104+ [
105+ 11,
106+ 22048,
107+ 17903,
108+ 7103
109+ ]
110+ ],
111+ "rows_by_hour_1027": [
112+ [
113+ 21,
114+ 56405,
115+ 48434,
116+ 14344
117+ ],
118+ [
119+ 22,
120+ 126305,
121+ 107801,
122+ 33512
123+ ],
124+ [
125+ 23,
126+ 116508,
127+ 100609,
128+ 28534
129+ ]
130+ ],
131+ "paper_outage_rows_149352": 149352
132+}
\ No newline at end of file
added results/ledger/batch_productivity_from_log.csv +176 −0
@@ -0,0 +1,176 @@
1+run,batch,n_batches,done_at,sections,mentions,mentions_per_section,ratio_to_median,anomalous_lt_25pct_of_median
2+october,1,125,2026-10-27 21:41:00,8596,1902,0.2213,0.795,False
3+october,2,125,2026-10-27 21:44:12,8723,2721,0.3119,1.12,False
4+october,3,125,2026-10-27 21:46:33,9571,2116,0.2211,0.794,False
5+october,4,125,2026-10-27 21:49:32,9102,2562,0.2815,1.011,False
6+october,5,125,2026-10-27 21:52:48,8408,1951,0.232,0.833,False
7+october,6,125,2026-10-27 21:58:36,8984,2089,0.2325,0.835,False
8+october,7,125,2026-10-27 22:03:29,8954,2939,0.3282,1.179,False
9+october,8,125,2026-10-27 22:08:22,8299,2793,0.3365,1.209,False
10+october,9,125,2026-10-27 22:12:28,8588,2371,0.2761,0.992,False
11+october,10,125,2026-10-27 22:17:45,10363,2742,0.2646,0.95,False
12+october,11,125,2026-10-27 22:22:29,7837,2227,0.2842,1.021,False
13+october,12,125,2026-10-27 22:25:14,9610,2085,0.217,0.779,False
14+october,13,125,2026-10-27 22:28:24,8549,2478,0.2899,1.041,False
15+october,14,125,2026-10-27 22:31:08,7599,1906,0.2508,0.901,False
16+october,15,125,2026-10-27 22:35:00,9094,2520,0.2771,0.995,False
17+october,16,125,2026-10-27 22:37:41,8437,2367,0.2805,1.008,False
18+october,17,125,2026-10-27 22:40:13,8233,2272,0.276,0.991,False
19+october,18,125,2026-10-27 22:45:53,9705,2410,0.2483,0.892,False
20+october,19,125,2026-10-27 22:51:46,10801,2329,0.2156,0.774,False
21+october,20,125,2026-10-27 22:57:49,10564,2575,0.2438,0.876,False
22+october,21,125,2026-10-27 23:03:05,10945,2124,0.1941,0.697,False
23+october,22,125,2026-10-27 23:08:21,8674,2074,0.2391,0.859,False
24+october,23,125,2026-10-27 23:13:38,9460,2412,0.255,0.916,False
25+october,24,125,2026-10-27 23:17:51,10644,2469,0.232,0.833,False
26+october,25,125,2026-10-27 23:23:56,9128,2503,0.2742,0.985,False
27+october,26,125,2026-10-27 23:29:07,9330,2779,0.2979,1.07,False
28+october,27,125,2026-10-27 23:37:16,10209,2387,0.2338,0.84,False
29+october,28,125,2026-10-27 23:41:09,10831,2241,0.2069,0.743,False
30+october,29,125,2026-10-27 23:44:30,9162,2785,0.304,1.092,False
31+october,30,125,2026-10-27 23:49:41,9476,2326,0.2455,0.882,False
32+october,31,125,2026-10-27 23:54:22,10573,2314,0.2189,0.786,False
33+october,32,125,2026-10-27 23:58:21,9135,2116,0.2316,0.832,False
34+october,33,125,2026-10-28 00:04:07,9893,3113,0.3147,1.13,False
35+october,34,125,2026-10-28 00:08:14,9143,2574,0.2815,1.011,False
36+october,35,125,2026-10-28 00:15:29,10740,3197,0.2977,1.069,False
37+october,36,125,2026-10-28 00:20:57,11636,3241,0.2785,1.0,False
38+october,37,125,2026-10-28 00:27:11,9545,2754,0.2885,1.036,False
39+october,38,125,2026-10-28 00:35:17,9948,2921,0.2936,1.055,False
40+october,39,125,2026-10-28 00:42:28,11222,3109,0.277,0.995,False
41+october,40,125,2026-10-28 00:46:24,9764,2819,0.2887,1.037,False
42+october,41,125,2026-10-28 00:52:03,9792,2612,0.2667,0.958,False
43+october,42,125,2026-10-28 00:57:42,11237,2485,0.2211,0.794,False
44+october,43,125,2026-10-28 01:00:46,8950,2712,0.303,1.088,False
45+october,44,125,2026-10-28 01:05:51,8611,2897,0.3364,1.208,False
46+october,45,125,2026-10-28 01:09:50,10332,2442,0.2364,0.849,False
47+october,46,125,2026-10-28 01:25:38,10260,1011,0.0985,0.354,False
48+october,47,125,2026-10-28 01:44:26,9550,0,0.0,0.0,True
49+october,48,125,2026-10-28 02:02:20,8726,0,0.0,0.0,True
50+october,49,125,2026-10-28 02:25:08,10693,0,0.0,0.0,True
51+october,50,125,2026-10-28 02:45:05,9784,0,0.0,0.0,True
52+october,51,125,2026-10-28 03:06:35,10260,0,0.0,0.0,True
53+october,52,125,2026-10-28 03:24:01,9081,0,0.0,0.0,True
54+october,53,125,2026-10-28 03:47:33,10765,0,0.0,0.0,True
55+october,54,125,2026-10-28 04:05:52,8367,0,0.0,0.0,True
56+october,55,125,2026-10-28 04:29:31,10903,0,0.0,0.0,True
57+october,56,125,2026-10-28 04:47:54,9042,0,0.0,0.0,True
58+october,57,125,2026-10-28 05:07:03,9047,0,0.0,0.0,True
59+october,58,125,2026-10-28 05:24:38,8672,0,0.0,0.0,True
60+october,59,125,2026-10-28 05:45:11,10171,0,0.0,0.0,True
61+october,60,125,2026-10-28 06:05:10,9337,0,0.0,0.0,True
62+october,1,81,2026-10-28 06:40:53,5198,34,0.0065,0.023,True
63+october,2,81,2026-10-28 06:46:05,9756,1931,0.1979,0.711,False
64+october,3,81,2026-10-28 06:51:56,9550,2749,0.2879,1.034,False
65+october,4,81,2026-10-28 06:58:08,8726,1877,0.2151,0.773,False
66+october,5,81,2026-10-28 07:03:04,10693,3043,0.2846,1.022,False
67+october,6,81,2026-10-28 07:07:54,9784,2259,0.2309,0.829,False
68+october,7,81,2026-10-28 07:13:48,10260,2884,0.2811,1.01,False
69+october,8,81,2026-10-28 07:16:26,9081,2827,0.3113,1.118,False
70+october,9,81,2026-10-28 07:20:31,10765,3159,0.2935,1.054,False
71+october,10,81,2026-10-28 07:23:33,8367,2630,0.3143,1.129,False
72+october,11,81,2026-10-28 07:26:57,10903,3031,0.278,0.999,False
73+october,12,81,2026-10-28 07:30:15,9042,2387,0.264,0.948,False
74+october,13,81,2026-10-28 07:33:28,9047,2270,0.2509,0.901,False
75+october,14,81,2026-10-28 07:36:25,8672,2850,0.3286,1.18,False
76+october,15,81,2026-10-28 07:42:11,10171,3576,0.3516,1.263,False
77+october,16,81,2026-10-28 07:45:36,9337,2977,0.3188,1.145,False
78+october,17,81,2026-10-28 07:48:50,10088,2603,0.258,0.927,False
79+october,18,81,2026-10-28 07:52:13,10182,2359,0.2317,0.832,False
80+october,19,81,2026-10-28 07:56:34,9101,2533,0.2783,1.0,False
81+october,20,81,2026-10-28 08:02:06,10155,2515,0.2477,0.89,False
82+october,21,81,2026-10-28 08:08:53,9509,2811,0.2956,1.062,False
83+october,22,81,2026-10-28 08:12:58,7641,2316,0.3031,1.089,False
84+october,23,81,2026-10-28 08:16:15,10020,3346,0.3339,1.199,False
85+october,24,81,2026-10-28 08:21:04,11340,3469,0.3059,1.099,False
86+october,25,81,2026-10-28 08:25:46,9397,2494,0.2654,0.953,False
87+october,26,81,2026-10-28 08:29:08,8219,2316,0.2818,1.012,False
88+october,27,81,2026-10-28 08:31:50,8764,2560,0.2921,1.049,False
89+october,28,81,2026-10-28 08:37:51,10213,2003,0.1961,0.704,False
90+october,29,81,2026-10-28 08:42:17,9579,2587,0.2701,0.97,False
91+october,30,81,2026-10-28 08:48:43,8960,2706,0.302,1.085,False
92+october,31,81,2026-10-28 08:54:30,8143,2185,0.2683,0.964,False
93+october,32,81,2026-10-28 09:00:11,8517,2245,0.2636,0.947,False
94+october,33,81,2026-10-28 09:03:35,8343,1974,0.2366,0.85,False
95+october,34,81,2026-10-28 09:07:52,9979,2354,0.2359,0.847,False
96+october,35,81,2026-10-28 09:10:23,6747,2056,0.3047,1.094,False
97+october,36,81,2026-10-28 09:13:53,8212,2188,0.2664,0.957,False
98+october,37,81,2026-10-28 09:18:52,8843,2341,0.2647,0.951,False
99+october,38,81,2026-10-28 09:23:12,8215,2785,0.339,1.218,False
100+october,39,81,2026-10-28 09:25:11,6103,1826,0.2992,1.075,False
101+october,40,81,2026-10-28 09:28:10,7599,2260,0.2974,1.068,False
102+october,41,81,2026-10-28 09:31:25,7284,1748,0.24,0.862,False
103+october,42,81,2026-10-28 09:34:27,7932,2443,0.308,1.106,False
104+october,43,81,2026-10-28 09:38:08,7207,2017,0.2799,1.005,False
105+october,44,81,2026-10-28 09:42:49,7914,2579,0.3259,1.171,False
106+october,45,81,2026-10-28 09:46:04,6854,1646,0.2402,0.863,False
107+october,46,81,2026-10-28 09:50:02,6570,1791,0.2726,0.979,False
108+october,47,81,2026-10-28 09:53:03,7197,1793,0.2491,0.895,False
109+october,48,81,2026-10-28 09:55:57,6578,1801,0.2738,0.983,False
110+october,49,81,2026-10-28 09:58:07,5730,1619,0.2825,1.015,False
111+october,50,81,2026-10-28 10:01:00,5955,2388,0.401,1.44,False
112+october,51,81,2026-10-28 10:03:55,5620,1704,0.3032,1.089,False
113+october,52,81,2026-10-28 10:07:22,5253,1400,0.2665,0.957,False
114+october,53,81,2026-10-28 10:10:27,5477,2024,0.3695,1.327,False
115+october,54,81,2026-10-28 10:16:01,5702,2058,0.3609,1.296,False
116+october,55,81,2026-10-28 10:19:52,4770,1385,0.2904,1.043,False
117+october,56,81,2026-10-28 10:22:27,5305,1509,0.2844,1.022,False
118+october,57,81,2026-10-28 10:24:37,5088,1528,0.3003,1.079,False
119+october,58,81,2026-10-28 10:26:47,5977,1825,0.3053,1.097,False
120+october,59,81,2026-10-28 10:28:56,4767,1441,0.3023,1.086,False
121+october,60,81,2026-10-28 10:32:08,4142,1324,0.3197,1.148,False
122+october,61,81,2026-10-28 10:36:09,4916,1681,0.3419,1.228,False
123+october,62,81,2026-10-28 10:38:19,4104,1178,0.287,1.031,False
124+october,63,81,2026-10-28 10:39:44,3576,1313,0.3672,1.319,False
125+october,64,81,2026-10-28 10:42:00,3981,1041,0.2615,0.939,False
126+october,65,81,2026-10-28 10:45:01,4153,1156,0.2784,1.0,False
127+october,66,81,2026-10-28 10:47:13,3751,1651,0.4401,1.581,False
128+october,67,81,2026-10-28 10:50:35,4014,1428,0.3558,1.278,False
129+october,68,81,2026-10-28 10:54:21,3813,1189,0.3118,1.12,False
130+october,69,81,2026-10-28 10:57:53,3788,1195,0.3155,1.133,False
131+october,70,81,2026-10-28 11:00:19,3557,1092,0.307,1.103,False
132+october,71,81,2026-10-28 11:02:03,3352,929,0.2771,0.995,False
133+october,72,81,2026-10-28 11:04:10,2644,782,0.2958,1.062,False
134+october,73,81,2026-10-28 11:06:07,2468,880,0.3566,1.281,False
135+october,74,81,2026-10-28 11:08:58,2751,924,0.3359,1.207,False
136+october,75,81,2026-10-28 11:10:53,2431,938,0.3858,1.386,False
137+october,76,81,2026-10-28 11:12:55,1925,550,0.2857,1.026,False
138+october,77,81,2026-10-28 11:14:03,1636,575,0.3515,1.263,False
139+october,78,81,2026-10-28 11:14:41,1352,421,0.3114,1.119,False
140+october,79,81,2026-10-28 11:15:07,962,291,0.3025,1.087,False
141+october,80,81,2026-10-28 11:15:32,499,150,0.3006,1.08,False
142+october,81,81,2026-10-28 11:15:49,208,72,0.3462,1.244,False
143+june,1,34,2026-06-18 08:31:49,4271,997,0.2334,0.895,False
144+june,2,34,2026-06-18 08:36:05,4831,1118,0.2314,0.888,False
145+june,3,34,2026-06-18 08:40:41,4476,1326,0.2962,1.136,False
146+june,4,34,2026-06-18 08:44:50,4379,1155,0.2638,1.012,False
147+june,5,34,2026-06-18 08:48:37,4022,901,0.224,0.859,False
148+june,6,34,2026-06-18 08:53:37,6315,1178,0.1865,0.715,False
149+june,7,34,2026-06-18 08:56:53,4460,835,0.1872,0.718,False
150+june,8,34,2026-06-18 09:03:36,6573,1848,0.2812,1.079,False
151+june,9,34,2026-06-18 09:09:39,6314,1484,0.235,0.901,False
152+june,10,34,2026-06-18 09:13:35,4324,932,0.2155,0.827,False
153+june,11,34,2026-06-18 09:17:47,4619,1225,0.2652,1.017,False
154+june,12,34,2026-06-18 09:24:11,6177,1750,0.2833,1.087,False
155+june,13,34,2026-06-18 09:29:01,4633,1302,0.281,1.078,False
156+june,14,34,2026-06-18 09:32:59,4464,944,0.2115,0.811,False
157+june,15,34,2026-06-18 09:38:18,4469,1619,0.3623,1.39,False
158+june,16,34,2026-06-18 09:43:23,4734,1188,0.251,0.963,False
159+june,17,34,2026-06-18 09:47:38,4186,1028,0.2456,0.942,False
160+june,18,34,2026-06-18 09:54:22,6003,1721,0.2867,1.1,False
161+june,19,34,2026-06-18 09:59:29,4939,1243,0.2517,0.965,False
162+june,20,34,2026-06-18 10:04:17,4784,964,0.2015,0.773,False
163+june,21,34,2026-06-18 10:09:03,4083,1181,0.2892,1.109,False
164+june,22,34,2026-06-18 10:13:49,4535,1183,0.2609,1.001,False
165+june,23,34,2026-06-18 10:20:06,4998,1557,0.3115,1.195,False
166+june,24,34,2026-06-18 10:26:33,5451,1587,0.2911,1.117,False
167+june,25,34,2026-06-18 10:32:52,4960,1495,0.3014,1.156,False
168+june,26,34,2026-06-18 10:39:52,5185,1796,0.3464,1.329,False
169+june,27,34,2026-06-18 10:44:31,4780,977,0.2044,0.784,False
170+june,28,34,2026-06-18 10:50:19,5934,1410,0.2376,0.911,False
171+june,29,34,2026-06-18 10:54:16,3938,1026,0.2605,0.999,False
172+june,30,34,2026-06-18 10:58:30,4361,947,0.2172,0.833,False
173+june,31,34,2026-06-18 11:02:06,3019,850,0.2816,1.08,False
174+june,32,34,2026-06-18 11:05:01,2293,624,0.2721,1.044,False
175+june,33,34,2026-06-18 11:06:28,1268,323,0.2547,0.977,False
176+june,34,34,2026-06-18 11:06:44,116,47,0.4052,1.554,False
added results/ledger/code_vs_data_checks.json +228 −0
@@ -0,0 +1,228 @@
1+{
2+ "distinct_backend_ledger": [
3+ "llm"
4+ ],
5+ "distinct_backend_mentions": [
6+ "llm"
7+ ],
8+ "confidence_exact_0600_total": 3278,
9+ "confidence_exact_0600_by_run": [
10+ [
11+ "october",
12+ 2974
13+ ],
14+ [
15+ "backfilled_june",
16+ 304
17+ ]
18+ ],
19+ "confidence_bins_by_run": [
20+ [
21+ "backfilled_june",
22+ 0.5,
23+ 106
24+ ],
25+ [
26+ "backfilled_june",
27+ 0.6,
28+ 304
29+ ],
30+ [
31+ "backfilled_june",
32+ 0.7,
33+ 2239
34+ ],
35+ [
36+ "backfilled_june",
37+ 0.8,
38+ 8594
39+ ],
40+ [
41+ "backfilled_june",
42+ 0.9,
43+ 14459
44+ ],
45+ [
46+ "backfilled_june",
47+ 1.0,
48+ 14228
49+ ],
50+ [
51+ "october",
52+ 0.5,
53+ 1505
54+ ],
55+ [
56+ "october",
57+ 0.6,
58+ 2974
59+ ],
60+ [
61+ "october",
62+ 0.7,
63+ 19408
64+ ],
65+ [
66+ "october",
67+ 0.8,
68+ 63686
69+ ],
70+ [
71+ "october",
72+ 0.9,
73+ 86372
74+ ],
75+ [
76+ "october",
77+ 1.0,
78+ 94190
79+ ]
80+ ],
81+ "min_confidence_by_run": [
82+ [
83+ "backfilled_june",
84+ 0.5,
85+ 1.0,
86+ 39930
87+ ],
88+ [
89+ "october",
90+ 0.5,
91+ 1.0,
92+ 268135
93+ ]
94+ ],
95+ "mentions_below_045": 0,
96+ "amount_raw_rule_mismatch": 584,
97+ "amount_raw_rule_mismatch_rows": [
98+ [
99+ "734023d0520612d5f402",
100+ 0.0,
101+ null
102+ ],
103+ [
104+ "db4673180782ab45b10b",
105+ 0.0,
106+ null
107+ ],
108+ [
109+ "3c295851c31a0c3e2ec0",
110+ 0.0,
111+ null
112+ ],
113+ [
114+ "63c97357d350022be19b",
115+ 0.0,
116+ null
117+ ],
118+ [
119+ "5b38aa9e9b23848af4e7",
120+ 0.0,
121+ null
122+ ],
123+ [
124+ "0baf489b918ca6a2e565",
125+ 0.0,
126+ null
127+ ],
128+ [
129+ "93537b10c9c3d43fdca3",
130+ 0.0,
131+ null
132+ ],
133+ [
134+ "b043bc1551282d61c4f1",
135+ 0.0,
136+ null
137+ ],
138+ [
139+ "e0a74132c0790e0f8e39",
140+ 0.0,
141+ null
142+ ],
143+ [
144+ "e81dd46e05627eaebb0c",
145+ 0.0,
146+ null
147+ ],
148+ [
149+ "2bc7b4eabca7ddeec540",
150+ 0.0,
151+ null
152+ ],
153+ [
154+ "768d20d20fe24ea2abb2",
155+ 0.0,
156+ null
157+ ],
158+ [
159+ "a8b9c386fa4bb91053a9",
160+ 0.0,
161+ null
162+ ],
163+ [
164+ "1f31eb05dda96af09770",
165+ 0.0,
166+ null
167+ ],
168+ [
169+ "1ea04bf0a611d3e6e8e1",
170+ 0.0,
171+ null
172+ ],
173+ [
174+ "36d0a293409cb18eb1b7",
175+ 0.0,
176+ null
177+ ],
178+ [
179+ "8c55318a560189f234a6",
180+ 0.0,
181+ null
182+ ],
183+ [
184+ "7232cc9db4b89c6d2ed5",
185+ 0.0,
186+ null
187+ ],
188+ [
189+ "a87788f9c89f1bb3d6c6",
190+ 0.0,
191+ null
192+ ],
193+ [
194+ "4d97915cd606d985efc5",
195+ 0.0,
196+ null
197+ ]
198+ ],
199+ "amount_raw_null_with_amount": 584,
200+ "amount_raw_notnull_without_amount": 0,
201+ "amount_le_0": 608,
202+ "mentions_with_run": [
203+ [
204+ "backfilled_june",
205+ 39930
206+ ],
207+ [
208+ "october",
209+ 268135
210+ ]
211+ ],
212+ "mentions_backfilled_block_vs_june_log": {
213+ "block_stored": 39930,
214+ "june_log": 39930
215+ },
216+ "fiscal_year_null_mentions_by_run": [
217+ [
218+ "backfilled_june",
219+ 39930,
220+ 39930
221+ ],
222+ [
223+ "october",
224+ 28897,
225+ 268135
226+ ]
227+ ]
228+}
\ No newline at end of file
added results/ledger/collisions.json +30 −0
@@ -0,0 +1,30 @@
1+{
2+ "sections_ledger_gt_stored": 304,
3+ "excess_mentions": 412,
4+ "sections_ledger_lt_stored": 0,
5+ "deficit_mentions": null,
6+ "sum_ledger_n_mentions": 308477,
7+ "stored_project_mentions": 308065,
8+ "mentions_whose_section_not_in_ledger": 0,
9+ "ledger_vs_stored_by_form": [
10+ [
11+ "10-K",
12+ 81622,
13+ 81495,
14+ 127
15+ ],
16+ [
17+ "10-Q",
18+ 152261,
19+ 152004,
20+ 257
21+ ],
22+ [
23+ "8-K",
24+ 74594,
25+ 74566,
26+ 28
27+ ]
28+ ],
29+ "duplicate_key_sections_in_stored": 0
30+}
\ No newline at end of file
added results/ledger/flush_zero_groups_after_0622.csv +2 −0
@@ -0,0 +1,2 @@
1+bin1,rows,zeros,zero_pct,calls,zero_pct_among_calls
2+2026-10-28 06:40:00,5198,5173,99.52,2954,99.15
added results/ledger/flush_zero_groups_before_0110.csv +1 −0
@@ -0,0 +1 @@
1+bin1,rows,zeros,zero_pct,calls,zero_pct_among_calls
added results/ledger/gate_pass_by_form.csv +4 −0
@@ -0,0 +1,4 @@
1+form_type,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2+10-K,143704,0,135310,135310,8394,0,30877,81622,81495,2464,21.49,22.82,0.568,0.6032
3+10-Q,578793,1069,363985,363973,214808,0,75705,152261,152004,2494,13.08,20.8,0.2631,0.4183
4+8-K,400601,1,204442,204442,196159,0,61648,74594,74566,2499,15.39,30.15,0.1862,0.3649
added results/ledger/gate_pass_by_form_item.csv +29 −0
@@ -0,0 +1,29 @@
1+form_type,item,run,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2+10-K,business,backfilled_june,5545,0,5542,5542,3,0,2185,5934,5934,460,39.4,39.43,1.0702,1.0707
3+10-K,business,october,23740,0,23576,23576,164,0,8383,24558,24521,2095,35.31,35.56,1.0345,1.0417
4+10-K,full_body,backfilled_june,1230,0,1229,1229,1,0,316,824,824,230,25.69,25.71,0.6699,0.6705
5+10-K,md_and_a,backfilled_june,5198,0,5096,5096,102,0,2505,6703,6703,459,48.19,49.16,1.2895,1.3153
6+10-K,md_and_a,october,23721,0,22954,22954,767,0,11779,32396,32318,2074,49.66,51.32,1.3657,1.4113
7+10-K,properties,backfilled_june,5084,0,4897,4897,187,0,870,1721,1721,444,17.11,17.77,0.3385,0.3514
8+10-K,properties,october,23152,0,21043,21043,2109,0,3013,6158,6148,2035,13.01,14.32,0.266,0.2926
9+10-K,quantitative_disclosures,backfilled_june,4502,0,3992,3992,510,0,71,110,110,405,1.58,1.78,0.0244,0.0276
10+10-K,quantitative_disclosures,october,22374,0,18040,18040,4334,0,284,421,420,2041,1.27,1.57,0.0188,0.0233
11+10-K,risk_factors,backfilled_june,5605,0,5580,5580,25,0,257,495,495,469,4.59,4.61,0.0883,0.0887
12+10-K,risk_factors,october,23553,0,23361,23361,192,0,1214,2302,2301,2079,5.15,5.2,0.0977,0.0985
13+10-Q,controls_and_procedures,backfilled_june,3567,1,797,797,2770,0,280,338,338,82,7.85,35.13,0.0948,0.4241
14+10-Q,controls_and_procedures,october,88826,0,20116,20116,68710,0,5323,6744,6741,2478,5.99,26.46,0.0759,0.3353
15+10-Q,financial_statements,backfilled_june,3594,0,3507,3507,87,0,743,1183,1183,82,20.67,21.19,0.3292,0.3373
16+10-Q,financial_statements,october,88288,0,86599,86599,1689,0,18266,28976,28897,2462,20.69,21.09,0.3282,0.3346
17+10-Q,full_document,october,241,0,241,241,0,0,38,107,107,12,15.77,15.77,0.444,0.444
18+10-Q,legal_proceedings,backfilled_june,3024,405,956,944,2068,0,18,20,20,82,0.6,1.91,0.0066,0.0212
19+10-Q,legal_proceedings,october,63974,0,19087,19087,44887,0,623,825,825,2251,0.97,3.26,0.0129,0.0432
20+10-Q,md_and_a,backfilled_june,3606,4,3599,3599,7,0,1563,3791,3791,83,43.34,43.43,1.0513,1.0533
21+10-Q,md_and_a,october,87765,0,86670,86670,1095,0,38224,95514,95376,2465,43.55,44.1,1.0883,1.102
22+10-Q,quantitative_disclosures,backfilled_june,3394,234,1662,1662,1732,0,37,51,51,81,1.09,2.23,0.015,0.0307
23+10-Q,quantitative_disclosures,october,79744,0,45058,45058,34686,0,553,757,749,2394,0.69,1.23,0.0095,0.0168
24+10-Q,risk_factors,backfilled_june,3344,425,1200,1200,2144,0,106,150,150,83,3.17,8.83,0.0449,0.125
25+10-Q,risk_factors,october,69216,0,32626,32626,36590,0,2891,4270,4270,2408,4.18,8.86,0.0617,0.1309
26+10-Q,unregistered_sales,backfilled_june,3344,0,2881,2881,463,0,222,284,284,83,6.64,7.71,0.0849,0.0986
27+10-Q,unregistered_sales,october,76866,0,58986,58986,17880,0,6818,9251,9222,2444,8.87,11.56,0.1204,0.1568
28+8-K,full_body,backfilled_june,103927,1,54360,54360,49567,0,15234,18326,18326,497,14.66,28.02,0.1763,0.3371
29+8-K,full_body,october,296674,0,150082,150082,146592,0,46414,56268,56240,2499,15.64,30.93,0.1897,0.3749
added results/ledger/gate_pass_by_form_run.csv +7 −0
@@ -0,0 +1,7 @@
1+form_type,run,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2+10-K,backfilled_june,27164,0,26336,26336,828,0,6204,15787,15787,497,22.84,23.56,0.5812,0.5994
3+10-K,october,116540,0,108974,108974,7566,0,24673,65835,65708,2101,21.17,22.64,0.5649,0.6041
4+10-Q,backfilled_june,23873,1069,14602,14590,9271,0,2969,5817,5817,84,12.44,20.35,0.2437,0.3987
5+10-Q,october,554920,0,349383,349383,205537,0,72736,146444,146187,2491,13.11,20.82,0.2639,0.4192
6+8-K,backfilled_june,103927,1,54360,54360,49567,0,15234,18326,18326,497,14.66,28.02,0.1763,0.3371
7+8-K,october,296674,0,150082,150082,146592,0,46414,56268,56240,2499,15.64,30.93,0.1897,0.3749
added results/ledger/gate_pass_by_run.csv +3 −0
@@ -0,0 +1,3 @@
1+run,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2+backfilled_june,154964,1070,95298,95286,59666,0,24407,39930,39930,500,15.75,25.61,0.2577,0.4191
3+october,968134,0,608439,608439,359695,0,143823,268547,268135,2499,14.86,23.64,0.2774,0.4414
added results/ledger/gate_pass_totals.json +27 −0
@@ -0,0 +1,27 @@
1+{
2+ "ledger_rows": 1123098,
3+ "lt30_words": 1070,
4+ "gate_pass": 703737,
5+ "calls": 703725,
6+ "gate_rejected_zero": 419361,
7+ "gate_fail_but_mentions": 0,
8+ "with_mentions": 168230,
9+ "ledger_mentions": 308477,
10+ "stored_mentions": 308065,
11+ "issuers": 2499,
12+ "productive_pct_ledger": 14.98,
13+ "productive_pct_calls": 23.91,
14+ "mentions_per_ledger_row": 0.2747,
15+ "mentions_per_call": 0.4383,
16+ "usd_per_call_paper_total_775": 0.001274,
17+ "usd_per_october_row_paper_total_775": 0.000801,
18+ "usd_per_call_from_0_0008_x_ledger": 0.001277,
19+ "october_rows": 968134,
20+ "october_calls": 608439,
21+ "headline_before": {
22+ "sections_submitted": 1123098,
23+ "productive_pct": 15.0,
24+ "mentions_per_section": 0.27,
25+ "usd_per_section": 0.0008
26+ }
27+}
\ No newline at end of file
added results/ledger/ledger_by_5min.csv +103 −0
@@ -0,0 +1,103 @@
1+bin5,rows,with_mentions,mentions,zero_pct,calls,flushes
2+2026-10-27 21:35:00,4350,609,1000,86.0,2908,4350
3+2026-10-27 21:40:00,14971,2246,4123,85.0,9278,14971
4+2026-10-27 21:45:00,16678,2317,4178,86.11,10174,16678
5+2026-10-27 21:50:00,12278,1643,2951,86.62,7438,12278
6+2026-10-27 21:55:00,8128,1156,2092,85.78,4731,8128
7+2026-10-27 22:00:00,7568,1251,2436,83.47,4736,7568
8+2026-10-27 22:05:00,8226,1321,2794,83.94,5159,8226
9+2026-10-27 22:10:00,12652,1770,3371,86.01,7682,12652
10+2026-10-27 22:15:00,6291,1012,1741,83.91,3959,6291
11+2026-10-27 22:20:00,13126,1895,3228,85.56,8137,13126
12+2026-10-27 22:25:00,15316,2276,4063,85.14,9166,15316
13+2026-10-27 22:30:00,12444,1786,3301,85.65,7539,12444
14+2026-10-27 22:35:00,16234,2561,4495,84.22,9837,16234
15+2026-10-27 22:40:00,8706,1277,2272,85.33,5610,8706
16+2026-10-27 22:45:00,8695,1095,1908,87.41,5337,8695
17+2026-10-27 22:50:00,7557,1051,1827,86.09,4643,7557
18+2026-10-27 22:55:00,9490,1209,2076,87.26,5474,9490
19+2026-10-27 23:00:00,10337,1296,2126,87.46,5849,10337
20+2026-10-27 23:05:00,8797,1166,2072,86.75,5161,8797
21+2026-10-27 23:10:00,11704,1724,2917,85.27,6758,11704
22+2026-10-27 23:15:00,9736,1309,2468,86.56,5873,9736
23+2026-10-27 23:20:00,5649,889,1499,84.26,3657,5649
24+2026-10-27 23:25:00,9272,1466,2778,84.19,5531,9272
25+2026-10-27 23:30:00,6374,746,1503,88.3,3769,6374
26+2026-10-27 23:35:00,9361,1115,2027,88.09,5877,9361
27+2026-10-27 23:40:00,14467,2186,3883,84.89,9223,14467
28+2026-10-27 23:45:00,9476,1277,2326,86.52,5957,9476
29+2026-10-27 23:50:00,10573,1330,2314,87.42,6185,10573
30+2026-10-27 23:55:00,10762,1395,2621,87.04,7042,10762
31+2026-10-28 00:00:00,8266,1260,2608,84.76,5571,8266
32+2026-10-28 00:05:00,10799,1580,3074,85.37,7176,10799
33+2026-10-28 00:10:00,8422,1348,2507,83.99,5542,8422
34+2026-10-28 00:15:00,10262,1461,2786,85.76,6392,10262
35+2026-10-28 00:20:00,8922,1232,2647,86.19,5590,8922
36+2026-10-28 00:25:00,5251,897,1754,82.92,3301,5251
37+2026-10-28 00:30:00,5786,796,1500,86.24,3666,5786
38+2026-10-28 00:35:00,7134,1000,1923,85.98,4397,7134
39+2026-10-28 00:40:00,12516,1940,3607,84.5,8081,12516
40+2026-10-28 00:45:00,7679,1248,2318,83.75,4851,7679
41+2026-10-28 00:50:00,11588,1443,2612,87.55,7517,11588
42+2026-10-28 00:55:00,11258,1579,2985,85.97,7198,11258
43+2026-10-28 01:00:00,8677,1420,2841,83.63,5338,8677
44+2026-10-28 01:05:00,7428,1732,3209,76.68,4781,7428
45+2026-10-28 01:10:00,253,253,500,0.0,253,253
46+2026-10-28 01:20:00,245,245,500,0.0,245,245
47+2026-10-28 01:25:00,6,6,11,0.0,6,6
48+2026-10-28 06:40:00,10458,488,1034,95.33,5998,10458
49+2026-10-28 06:45:00,11183,1488,2933,86.69,6736,11183
50+2026-10-28 06:50:00,4578,563,1248,87.7,2767,4578
51+2026-10-28 06:55:00,8928,970,1877,89.14,5656,8928
52+2026-10-28 07:00:00,13065,1846,3544,85.87,8199,13065
53+2026-10-28 07:05:00,5495,737,1257,86.59,3531,5495
54+2026-10-28 07:10:00,13148,1932,3884,85.31,8222,13148
55+2026-10-28 07:15:00,14377,2207,4336,84.65,9039,14377
56+2026-10-28 07:20:00,14323,2356,4280,83.55,9481,14323
57+2026-10-28 07:25:00,15262,2242,4034,85.31,9867,15262
58+2026-10-28 07:30:00,13643,1950,3661,85.71,8627,13643
59+2026-10-28 07:35:00,11317,1918,3845,83.05,6959,11317
60+2026-10-28 07:40:00,12049,2131,4081,82.31,7761,12049
61+2026-10-28 07:45:00,14266,1947,3575,86.35,8570,14266
62+2026-10-28 07:50:00,13271,1874,3360,85.88,8687,13271
63+2026-10-28 07:55:00,8061,1150,2030,85.73,5247,8061
64+2026-10-28 08:00:00,8671,1273,2519,85.32,5375,8671
65+2026-10-28 08:05:00,6292,971,1807,84.57,4000,6292
66+2026-10-28 08:10:00,13917,2218,4325,84.06,8464,13917
67+2026-10-28 08:15:00,10201,1720,3341,83.14,6351,10201
68+2026-10-28 08:20:00,12910,1818,3468,85.92,8165,12910
69+2026-10-28 08:25:00,12030,1728,3307,85.64,7847,12030
70+2026-10-28 08:30:00,11092,1597,3062,85.6,7157,11092
71+2026-10-28 08:35:00,9716,1161,2001,88.05,6025,9716
72+2026-10-28 08:40:00,8927,1327,2590,85.13,5581,8927
73+2026-10-28 08:45:00,5340,774,1703,85.51,3383,5340
74+2026-10-28 08:50:00,8143,1123,2185,86.21,5043,8143
75+2026-10-28 08:55:00,7749,1124,2002,85.49,4697,7749
76+2026-10-28 09:00:00,12882,1830,3218,85.79,8257,12882
77+2026-10-28 09:05:00,11277,1510,2855,86.61,7136,11277
78+2026-10-28 09:10:00,9890,1466,2742,85.18,6635,9890
79+2026-10-28 09:15:00,8843,1304,2341,85.25,5743,8843
80+2026-10-28 09:20:00,13311,2171,4295,83.69,8526,13311
81+2026-10-28 09:25:00,10741,1668,3076,84.47,7093,10741
82+2026-10-28 09:30:00,13081,1893,3691,85.53,8399,13081
83+2026-10-28 09:35:00,8869,1402,2517,84.19,6010,8869
84+2026-10-28 09:40:00,10958,1661,3082,84.84,7112,10958
85+2026-10-28 09:45:00,7764,1113,2146,85.66,5017,7764
86+2026-10-28 09:50:00,12713,1842,3387,85.51,8209,12713
87+2026-10-28 09:55:00,10382,1610,3116,84.49,6761,10382
88+2026-10-28 10:00:00,8939,1599,3090,82.11,5860,8939
89+2026-10-28 10:05:00,9496,1498,2900,84.22,5904,9496
90+2026-10-28 10:10:00,5551,942,2028,83.03,3609,5551
91+2026-10-28 10:15:00,6155,1042,1939,83.07,3883,6155
92+2026-10-28 10:20:00,10393,1756,3037,83.1,7032,10393
93+2026-10-28 10:25:00,10744,1686,3266,84.31,6518,10744
94+2026-10-28 10:30:00,5861,970,1825,83.45,3687,5861
95+2026-10-28 10:35:00,10877,1896,3671,82.57,6864,10877
96+2026-10-28 10:40:00,7599,1184,2043,84.42,4755,7599
97+2026-10-28 10:45:00,7156,1365,2806,80.93,4518,7156
98+2026-10-28 10:50:00,4957,918,1616,81.48,3274,4957
99+2026-10-28 10:55:00,5525,905,1696,83.62,3602,5525
100+2026-10-28 11:00:00,7816,1336,2302,82.91,5008,7816
101+2026-10-28 11:05:00,5886,1129,1990,80.82,3669,5886
102+2026-10-28 11:10:00,6677,1369,2298,79.5,4261,6677
103+2026-10-28 11:15:00,1669,311,513,81.37,1067,1669
added results/ledger/ledger_by_minute.csv +443 −0
@@ -0,0 +1,443 @@
1+bin1,rows,with_mentions,mentions,zero_pct,calls,zero_pct_among_calls
2+2026-10-27 21:35:00,7,0,0,100.0,6,100.0
3+2026-10-27 21:38:00,2144,296,500,86.19,1469,79.85
4+2026-10-27 21:39:00,2199,313,500,85.77,1433,78.16
5+2026-10-27 21:40:00,4187,467,855,88.85,2617,82.16
6+2026-10-27 21:41:00,1657,303,548,81.71,1026,70.47
7+2026-10-27 21:42:00,2874,536,1003,81.35,1800,70.22
8+2026-10-27 21:43:00,3524,527,1001,85.05,2171,75.73
9+2026-10-27 21:44:00,2729,413,716,84.87,1664,75.18
10+2026-10-27 21:45:00,5114,616,1003,87.95,3133,80.34
11+2026-10-27 21:46:00,2462,363,613,85.26,1514,76.02
12+2026-10-27 21:47:00,3715,488,1001,86.86,2245,78.26
13+2026-10-27 21:48:00,3411,540,1002,84.17,2115,74.47
14+2026-10-27 21:49:00,1976,310,559,84.31,1167,73.44
15+2026-10-27 21:50:00,2235,270,501,87.92,1368,80.26
16+2026-10-27 21:51:00,1943,275,503,85.85,1227,77.59
17+2026-10-27 21:52:00,4230,582,947,86.24,2727,78.66
18+2026-10-27 21:53:00,2082,265,500,87.27,1150,76.96
19+2026-10-27 21:54:00,1788,251,500,85.96,966,74.02
20+2026-10-27 21:56:00,2270,284,501,87.49,1322,78.52
21+2026-10-27 21:57:00,2641,286,502,89.17,1469,80.53
22+2026-10-27 21:58:00,203,55,86,72.91,132,58.33
23+2026-10-27 21:59:00,3014,531,1003,82.38,1808,70.63
24+2026-10-27 22:00:00,1548,229,503,85.21,964,76.24
25+2026-10-27 22:01:00,1763,268,500,84.8,1134,76.37
26+2026-10-27 22:02:00,1455,271,500,81.37,895,69.72
27+2026-10-27 22:03:00,1174,224,433,80.92,724,69.06
28+2026-10-27 22:04:00,1628,259,500,84.09,1019,74.58
29+2026-10-27 22:05:00,1166,231,501,80.19,733,68.49
30+2026-10-27 22:06:00,1384,232,500,83.24,885,73.79
31+2026-10-27 22:07:00,1567,222,500,85.83,985,77.46
32+2026-10-27 22:08:00,2554,408,792,84.03,1623,74.86
33+2026-10-27 22:09:00,1555,228,501,85.34,933,75.56
34+2026-10-27 22:10:00,1546,240,500,84.48,919,73.88
35+2026-10-27 22:11:00,1690,253,500,85.03,965,73.78
36+2026-10-27 22:12:00,3797,459,870,87.91,2252,79.62
37+2026-10-27 22:13:00,3832,551,1001,85.62,2435,77.37
38+2026-10-27 22:14:00,1787,267,500,85.06,1111,75.97
39+2026-10-27 22:15:00,2052,288,504,85.96,1281,77.52
40+2026-10-27 22:17:00,2692,459,737,82.95,1732,73.5
41+2026-10-27 22:19:00,1547,265,500,82.87,946,71.99
42+2026-10-27 22:20:00,1512,256,500,83.07,957,73.25
43+2026-10-27 22:21:00,2044,313,500,84.69,1329,76.45
44+2026-10-27 22:22:00,2734,437,727,84.02,1672,73.86
45+2026-10-27 22:23:00,4515,590,1001,86.93,2684,78.02
46+2026-10-27 22:24:00,2321,299,500,87.12,1495,80.0
47+2026-10-27 22:25:00,4612,643,1084,86.06,2762,76.72
48+2026-10-27 22:26:00,1702,281,500,83.49,933,69.88
49+2026-10-27 22:27:00,3514,539,1005,84.66,1998,73.02
50+2026-10-27 22:28:00,1495,250,473,83.28,852,70.66
51+2026-10-27 22:29:00,3993,563,1001,85.9,2621,78.52
52+2026-10-27 22:30:00,2134,272,501,87.25,1362,80.03
53+2026-10-27 22:31:00,1472,210,404,85.73,931,77.44
54+2026-10-27 22:32:00,1929,285,501,85.23,1073,73.44
55+2026-10-27 22:33:00,3466,509,956,85.31,2067,75.37
56+2026-10-27 22:34:00,3443,510,939,85.19,2106,75.78
57+2026-10-27 22:35:00,1994,340,624,82.95,1134,70.02
58+2026-10-27 22:36:00,3233,591,1005,81.72,1979,70.14
59+2026-10-27 22:37:00,3466,503,862,85.49,2116,76.23
60+2026-10-27 22:38:00,3909,558,1003,85.73,2374,76.5
61+2026-10-27 22:39:00,3632,569,1001,84.33,2234,74.53
62+2026-10-27 22:40:00,692,137,268,80.2,431,68.21
63+2026-10-27 22:41:00,3870,573,1001,85.19,2495,77.03
64+2026-10-27 22:43:00,1884,273,503,85.51,1191,77.08
65+2026-10-27 22:44:00,2260,294,500,86.99,1493,80.31
66+2026-10-27 22:45:00,1691,249,406,85.27,1107,77.51
67+2026-10-27 22:47:00,2688,270,500,89.96,1553,82.61
68+2026-10-27 22:48:00,2309,298,501,87.09,1369,78.23
69+2026-10-27 22:49:00,2007,278,501,86.15,1308,78.75
70+2026-10-27 22:50:00,2564,334,500,86.97,1648,79.73
71+2026-10-27 22:51:00,1233,200,327,83.78,812,75.37
72+2026-10-27 22:52:00,1985,274,500,86.2,1151,76.19
73+2026-10-27 22:53:00,1775,243,500,86.31,1032,76.45
74+2026-10-27 22:55:00,1832,258,500,85.92,1039,75.17
75+2026-10-27 22:56:00,2164,289,500,86.65,1290,77.6
76+2026-10-27 22:57:00,2808,346,575,87.68,1690,79.53
77+2026-10-27 22:59:00,2686,316,501,88.24,1455,78.28
78+2026-10-27 23:01:00,2624,294,504,88.8,1375,78.62
79+2026-10-27 23:02:00,5339,628,1002,88.24,3058,79.46
80+2026-10-27 23:03:00,296,72,117,75.68,182,60.44
81+2026-10-27 23:04:00,2078,302,503,85.47,1234,75.53
82+2026-10-27 23:05:00,1824,266,500,85.42,1059,74.88
83+2026-10-27 23:06:00,2241,279,500,87.55,1366,79.58
84+2026-10-27 23:08:00,2531,345,571,86.37,1489,76.83
85+2026-10-27 23:09:00,2201,276,501,87.46,1247,77.87
86+2026-10-27 23:11:00,3501,573,1000,83.63,2069,72.31
87+2026-10-27 23:12:00,1857,300,500,83.84,1082,72.27
88+2026-10-27 23:13:00,1901,272,411,85.69,1092,75.09
89+2026-10-27 23:14:00,4445,579,1006,86.97,2515,76.98
90+2026-10-27 23:15:00,2407,296,500,87.7,1387,78.66
91+2026-10-27 23:16:00,2101,271,501,87.1,1190,77.23
92+2026-10-27 23:17:00,1691,262,462,84.51,983,73.35
93+2026-10-27 23:18:00,1829,240,505,86.88,1176,79.59
94+2026-10-27 23:19:00,1708,240,500,85.95,1137,78.89
95+2026-10-27 23:20:00,1980,271,500,86.31,1276,78.76
96+2026-10-27 23:22:00,1847,290,502,84.3,1198,75.79
97+2026-10-27 23:23:00,1764,327,496,81.46,1153,71.64
98+2026-10-27 23:24:00,58,1,1,98.28,30,96.67
99+2026-10-27 23:25:00,3107,552,999,82.23,1831,69.85
100+2026-10-27 23:26:00,1718,278,500,83.82,1011,72.5
101+2026-10-27 23:27:00,1887,241,501,87.23,1151,79.06
102+2026-10-27 23:28:00,1800,259,503,85.61,1064,75.66
103+2026-10-27 23:29:00,760,136,275,82.11,474,71.31
104+2026-10-27 23:30:00,2193,250,500,88.6,1279,80.45
105+2026-10-27 23:32:00,1885,237,500,87.43,1088,78.22
106+2026-10-27 23:34:00,2296,259,503,88.72,1402,81.53
107+2026-10-27 23:35:00,2048,284,501,86.13,1312,78.35
108+2026-10-27 23:37:00,1787,196,383,89.03,1135,82.73
109+2026-10-27 23:38:00,2690,266,504,90.11,1631,83.69
110+2026-10-27 23:39:00,2836,369,639,86.99,1799,79.49
111+2026-10-27 23:40:00,4061,508,866,87.49,2615,80.57
112+2026-10-27 23:41:00,3135,428,732,86.35,1925,77.77
113+2026-10-27 23:42:00,2950,552,1003,81.29,1889,70.78
114+2026-10-27 23:43:00,2594,411,768,84.16,1653,75.14
115+2026-10-27 23:44:00,1727,287,514,83.38,1141,74.85
116+2026-10-27 23:45:00,2554,293,502,88.53,1584,81.5
117+2026-10-27 23:46:00,1996,263,500,86.82,1271,79.31
118+2026-10-27 23:47:00,2021,282,500,86.05,1326,78.73
119+2026-10-27 23:48:00,1754,265,500,84.89,1071,75.26
120+2026-10-27 23:49:00,1151,174,324,84.88,705,75.32
121+2026-10-27 23:50:00,2587,298,500,88.48,1424,79.07
122+2026-10-27 23:51:00,2283,300,500,86.86,1312,77.13
123+2026-10-27 23:53:00,3398,442,786,86.99,2092,78.87
124+2026-10-27 23:54:00,2305,290,528,87.42,1357,78.63
125+2026-10-27 23:55:00,2597,273,500,89.49,1674,83.69
126+2026-10-27 23:56:00,1873,285,502,84.78,1301,78.09
127+2026-10-27 23:57:00,2268,266,501,88.27,1505,82.33
128+2026-10-27 23:58:00,2397,337,613,85.94,1520,77.83
129+2026-10-27 23:59:00,1627,234,505,85.62,1042,77.54
130+2026-10-28 00:00:00,1258,218,501,82.67,837,73.95
131+2026-10-28 00:01:00,1674,239,500,85.72,1111,78.49
132+2026-10-28 00:02:00,1426,207,500,85.48,945,78.1
133+2026-10-28 00:03:00,1762,254,501,85.58,1222,79.21
134+2026-10-28 00:04:00,2146,342,606,84.06,1456,76.51
135+2026-10-28 00:05:00,1952,270,502,86.17,1337,79.81
136+2026-10-28 00:06:00,3863,512,1001,86.75,2632,80.55
137+2026-10-28 00:07:00,1556,244,501,84.32,1005,75.72
138+2026-10-28 00:08:00,1772,315,570,82.22,1183,73.37
139+2026-10-28 00:09:00,1656,239,500,85.57,1019,76.55
140+2026-10-28 00:10:00,1637,258,500,84.24,1039,75.17
141+2026-10-28 00:11:00,1596,247,500,84.52,1012,75.59
142+2026-10-28 00:12:00,1449,275,501,81.02,984,72.05
143+2026-10-28 00:13:00,2033,303,504,85.1,1401,78.37
144+2026-10-28 00:14:00,1707,265,502,84.48,1106,76.04
145+2026-10-28 00:15:00,662,99,190,85.05,450,78.0
146+2026-10-28 00:16:00,2143,278,503,87.03,1287,78.4
147+2026-10-28 00:17:00,1717,263,502,84.68,1080,75.65
148+2026-10-28 00:18:00,3511,521,1005,85.16,2204,76.36
149+2026-10-28 00:19:00,2229,300,586,86.54,1371,78.12
150+2026-10-28 00:20:00,2036,328,645,83.89,1285,74.47
151+2026-10-28 00:21:00,1534,225,501,85.33,957,76.49
152+2026-10-28 00:22:00,1546,205,501,86.74,970,78.87
153+2026-10-28 00:23:00,1807,231,500,87.22,1097,78.94
154+2026-10-28 00:24:00,1999,243,500,87.84,1281,81.03
155+2026-10-28 00:26:00,1717,274,502,84.04,1094,74.95
156+2026-10-28 00:27:00,942,143,250,84.82,615,76.75
157+2026-10-28 00:28:00,1325,247,501,81.36,807,69.39
158+2026-10-28 00:29:00,1267,233,501,81.61,785,70.32
159+2026-10-28 00:30:00,1502,244,500,83.75,941,74.07
160+2026-10-28 00:32:00,1955,279,500,85.73,1239,77.48
161+2026-10-28 00:34:00,2329,273,500,88.28,1486,81.63
162+2026-10-28 00:35:00,1570,202,419,87.13,1012,80.04
163+2026-10-28 00:36:00,1802,239,500,86.74,1038,76.97
164+2026-10-28 00:37:00,1810,280,503,84.53,1109,74.75
165+2026-10-28 00:39:00,1952,279,501,85.71,1238,77.46
166+2026-10-28 00:40:00,3525,532,1000,84.91,2377,77.62
167+2026-10-28 00:42:00,2133,356,605,83.31,1437,75.23
168+2026-10-28 00:43:00,3359,523,1001,84.43,2021,74.12
169+2026-10-28 00:44:00,3499,529,1001,84.88,2246,76.45
170+2026-10-28 00:45:00,1822,283,500,84.47,1166,75.73
171+2026-10-28 00:46:00,1084,172,317,84.13,692,75.14
172+2026-10-28 00:47:00,1708,270,500,84.19,1054,74.38
173+2026-10-28 00:48:00,1627,265,500,83.71,1005,73.63
174+2026-10-28 00:49:00,1438,258,501,82.06,934,72.38
175+2026-10-28 00:51:00,4632,538,1000,88.39,2975,81.92
176+2026-10-28 00:52:00,387,50,111,87.08,249,79.92
177+2026-10-28 00:53:00,2511,299,500,88.09,1569,80.94
178+2026-10-28 00:54:00,4058,556,1001,86.3,2724,79.59
179+2026-10-28 00:56:00,2179,300,500,86.23,1456,79.4
180+2026-10-28 00:57:00,2489,307,484,87.67,1633,81.2
181+2026-10-28 00:58:00,3000,497,1000,83.43,1825,72.77
182+2026-10-28 00:59:00,3590,475,1001,86.77,2284,79.2
183+2026-10-28 01:00:00,2360,373,711,84.19,1457,74.4
184+2026-10-28 01:01:00,1747,263,500,84.95,1083,75.72
185+2026-10-28 01:02:00,1106,239,501,78.39,676,64.64
186+2026-10-28 01:03:00,3096,480,1002,84.5,1890,74.6
187+2026-10-28 01:04:00,368,65,127,82.34,232,71.98
188+2026-10-28 01:05:00,2294,346,767,84.92,1393,75.16
189+2026-10-28 01:07:00,2155,287,500,86.68,1237,76.8
190+2026-10-28 01:08:00,2452,572,1000,76.67,1624,64.78
191+2026-10-28 01:09:00,527,527,942,0.0,527,0.0
192+2026-10-28 01:13:00,240,240,465,0.0,240,0.0
193+2026-10-28 01:14:00,13,13,35,0.0,13,0.0
194+2026-10-28 01:20:00,80,80,173,0.0,80,0.0
195+2026-10-28 01:21:00,165,165,327,0.0,165,0.0
196+2026-10-28 01:25:00,6,6,11,0.0,6,0.0
197+2026-10-28 06:40:00,5198,25,34,99.52,2954,99.15
198+2026-10-28 06:43:00,5260,463,1000,91.2,3044,84.79
199+2026-10-28 06:45:00,2630,258,503,90.19,1591,83.78
200+2026-10-28 06:46:00,3503,484,928,86.18,2122,77.19
201+2026-10-28 06:47:00,1631,257,501,84.24,977,73.69
202+2026-10-28 06:48:00,1789,256,500,85.69,1079,76.27
203+2026-10-28 06:49:00,1630,233,501,85.71,967,75.9
204+2026-10-28 06:51:00,2863,346,747,87.91,1700,79.65
205+2026-10-28 06:53:00,1715,217,501,87.35,1067,79.66
206+2026-10-28 06:55:00,2316,242,500,89.55,1443,83.23
207+2026-10-28 06:57:00,2681,265,500,90.12,1696,84.38
208+2026-10-28 06:58:00,2014,209,376,89.62,1244,83.2
209+2026-10-28 06:59:00,1917,254,501,86.75,1273,80.05
210+2026-10-28 07:00:00,1615,225,502,86.07,1031,78.18
211+2026-10-28 07:01:00,3110,456,1001,85.34,1990,77.09
212+2026-10-28 07:02:00,3432,479,796,86.04,2208,78.31
213+2026-10-28 07:03:00,619,146,243,76.41,431,66.13
214+2026-10-28 07:04:00,4289,540,1002,87.41,2539,78.73
215+2026-10-28 07:05:00,2091,277,503,86.75,1344,79.39
216+2026-10-28 07:07:00,3404,460,754,86.49,2187,78.97
217+2026-10-28 07:10:00,2690,272,500,89.89,1744,84.4
218+2026-10-28 07:11:00,2403,301,502,87.47,1525,80.26
219+2026-10-28 07:12:00,2931,442,1006,84.92,1828,75.82
220+2026-10-28 07:13:00,2236,401,876,82.07,1428,71.92
221+2026-10-28 07:14:00,2888,516,1000,82.13,1697,69.59
222+2026-10-28 07:15:00,3392,512,1001,84.91,2002,74.43
223+2026-10-28 07:16:00,2801,428,826,84.72,1594,73.15
224+2026-10-28 07:17:00,4376,753,1504,82.79,2957,74.54
225+2026-10-28 07:18:00,1822,258,502,85.84,1187,78.26
226+2026-10-28 07:19:00,1986,256,503,87.11,1299,80.29
227+2026-10-28 07:20:00,2581,372,650,85.59,1742,78.65
228+2026-10-28 07:21:00,1746,279,502,84.02,1168,76.11
229+2026-10-28 07:22:00,2813,506,1000,82.01,1923,73.69
230+2026-10-28 07:23:00,3808,615,1128,83.85,2457,74.97
231+2026-10-28 07:24:00,3375,584,1000,82.7,2191,73.35
232+2026-10-28 07:25:00,3553,572,1000,83.9,2429,76.45
233+2026-10-28 07:26:00,3975,572,1031,85.61,2614,78.12
234+2026-10-28 07:27:00,1857,277,502,85.08,1164,76.2
235+2026-10-28 07:28:00,4056,554,1001,86.34,2569,78.44
236+2026-10-28 07:29:00,1821,267,500,85.34,1091,75.53
237+2026-10-28 07:30:00,1308,199,384,84.79,790,74.81
238+2026-10-28 07:31:00,2409,289,501,88.0,1573,81.63
239+2026-10-28 07:32:00,3901,551,1000,85.88,2522,78.15
240+2026-10-28 07:33:00,2737,420,769,84.65,1739,75.85
241+2026-10-28 07:34:00,3288,491,1007,85.07,2003,75.49
242+2026-10-28 07:35:00,3134,474,1001,84.88,1914,75.24
243+2026-10-28 07:36:00,2250,439,842,80.49,1426,69.21
244+2026-10-28 07:37:00,2909,505,1000,82.64,1775,71.55
245+2026-10-28 07:38:00,1315,247,502,81.22,823,69.99
246+2026-10-28 07:39:00,1709,253,500,85.2,1021,75.22
247+2026-10-28 07:40:00,1416,285,501,79.87,911,68.72
248+2026-10-28 07:41:00,1431,265,501,81.48,853,68.93
249+2026-10-28 07:42:00,3341,572,1072,82.88,2159,73.51
250+2026-10-28 07:43:00,3121,513,1001,83.56,2038,74.83
251+2026-10-28 07:44:00,2740,496,1006,81.9,1800,72.44
252+2026-10-28 07:45:00,1526,243,470,84.08,968,74.9
253+2026-10-28 07:46:00,2450,308,500,87.43,1435,78.54
254+2026-10-28 07:47:00,3348,531,1005,84.14,1996,73.4
255+2026-10-28 07:48:00,4290,559,1098,86.97,2494,77.59
256+2026-10-28 07:49:00,2652,306,502,88.46,1677,81.75
257+2026-10-28 07:50:00,3861,575,1002,85.11,2574,77.66
258+2026-10-28 07:51:00,2237,275,500,87.71,1405,80.43
259+2026-10-28 07:52:00,3627,476,855,86.88,2354,79.78
260+2026-10-28 07:53:00,1536,265,503,82.75,1005,73.63
261+2026-10-28 07:54:00,2010,283,500,85.92,1349,79.02
262+2026-10-28 07:55:00,1810,266,500,85.3,1181,77.48
263+2026-10-28 07:56:00,1550,281,530,81.87,1025,72.59
264+2026-10-28 07:58:00,2512,307,500,87.78,1625,81.11
265+2026-10-28 07:59:00,2189,296,500,86.48,1416,79.1
266+2026-10-28 08:00:00,1771,253,501,85.71,1140,77.81
267+2026-10-28 08:01:00,1738,254,500,85.39,1071,76.28
268+2026-10-28 08:02:00,3743,536,1018,85.68,2301,76.71
269+2026-10-28 08:03:00,1419,230,500,83.79,863,73.35
270+2026-10-28 08:05:00,1648,268,500,83.74,1022,73.78
271+2026-10-28 08:06:00,1600,249,500,84.44,1042,76.1
272+2026-10-28 08:07:00,1811,267,500,85.26,1150,76.78
273+2026-10-28 08:08:00,1233,187,307,84.83,786,76.21
274+2026-10-28 08:10:00,1685,265,500,84.27,976,72.85
275+2026-10-28 08:11:00,1761,260,502,85.24,1053,75.31
276+2026-10-28 08:12:00,4195,642,1314,84.7,2587,75.18
277+2026-10-28 08:13:00,3205,542,1006,83.09,1951,72.22
278+2026-10-28 08:14:00,3071,509,1003,83.43,1897,73.17
279+2026-10-28 08:15:00,2920,517,1000,82.29,1790,71.12
280+2026-10-28 08:16:00,2174,430,837,80.22,1377,68.77
281+2026-10-28 08:17:00,1620,258,503,84.07,980,73.67
282+2026-10-28 08:18:00,1573,239,500,84.81,996,76.0
283+2026-10-28 08:19:00,1914,276,501,85.58,1208,77.15
284+2026-10-28 08:20:00,3582,559,1000,84.39,2243,75.08
285+2026-10-28 08:21:00,3016,347,671,88.49,1842,81.16
286+2026-10-28 08:22:00,3039,431,794,85.82,1938,77.76
287+2026-10-28 08:23:00,1630,239,500,85.34,1090,78.07
288+2026-10-28 08:24:00,1643,242,503,85.27,1052,77.0
289+2026-10-28 08:25:00,1370,236,491,82.77,919,74.32
290+2026-10-28 08:26:00,1991,276,500,86.14,1278,78.4
291+2026-10-28 08:27:00,2027,267,500,86.83,1233,78.35
292+2026-10-28 08:28:00,3413,492,1003,85.58,2247,78.1
293+2026-10-28 08:29:00,3229,457,813,85.85,2170,78.94
294+2026-10-28 08:30:00,3190,479,1001,84.98,2160,77.82
295+2026-10-28 08:31:00,3133,522,1059,83.34,2063,74.7
296+2026-10-28 08:32:00,2288,287,450,87.46,1380,79.2
297+2026-10-28 08:33:00,350,32,51,90.86,220,85.45
298+2026-10-28 08:34:00,2131,277,501,87.0,1334,79.24
299+2026-10-28 08:35:00,1193,129,230,89.19,784,83.55
300+2026-10-28 08:36:00,1797,157,272,91.26,1099,85.71
301+2026-10-28 08:37:00,2454,289,499,88.22,1497,80.69
302+2026-10-28 08:38:00,2091,308,500,85.27,1317,76.61
303+2026-10-28 08:39:00,2181,278,500,87.25,1328,79.07
304+2026-10-28 08:40:00,3341,515,1002,84.59,2112,75.62
305+2026-10-28 08:42:00,1966,306,585,84.44,1220,74.92
306+2026-10-28 08:43:00,2357,287,503,87.82,1465,80.41
307+2026-10-28 08:44:00,1263,219,500,82.66,784,72.07
308+2026-10-28 08:45:00,1265,210,500,83.4,808,74.01
309+2026-10-28 08:46:00,1671,243,500,85.46,1070,77.29
310+2026-10-28 08:48:00,2404,321,703,86.65,1505,78.67
311+2026-10-28 08:50:00,1928,255,501,86.77,1175,78.3
312+2026-10-28 08:52:00,1552,245,502,84.21,949,74.18
313+2026-10-28 08:53:00,2176,262,502,87.96,1341,80.46
314+2026-10-28 08:54:00,2487,361,680,85.48,1578,77.12
315+2026-10-28 08:55:00,1955,308,500,84.25,1187,74.05
316+2026-10-28 08:56:00,1764,266,500,84.92,1069,75.12
317+2026-10-28 08:58:00,2039,298,501,85.38,1268,76.5
318+2026-10-28 08:59:00,1991,252,501,87.34,1173,78.52
319+2026-10-28 09:00:00,768,106,243,86.2,489,78.32
320+2026-10-28 09:01:00,1576,316,500,79.95,1040,69.62
321+2026-10-28 09:02:00,4391,582,1000,86.75,2836,79.48
322+2026-10-28 09:03:00,2376,274,474,88.47,1454,81.16
323+2026-10-28 09:04:00,3771,552,1001,85.36,2438,77.36
324+2026-10-28 09:05:00,2287,280,500,87.76,1439,80.54
325+2026-10-28 09:06:00,2025,293,500,85.53,1248,76.52
326+2026-10-28 09:07:00,1896,216,353,88.61,1144,81.12
327+2026-10-28 09:08:00,1940,243,502,87.47,1243,80.45
328+2026-10-28 09:09:00,3129,478,1000,84.72,2062,76.82
329+2026-10-28 09:10:00,1711,274,554,83.99,1094,74.95
330+2026-10-28 09:11:00,3366,523,1001,84.46,2238,76.63
331+2026-10-28 09:12:00,2007,269,502,86.6,1418,81.03
332+2026-10-28 09:13:00,2806,400,685,85.74,1885,78.78
333+2026-10-28 09:15:00,1786,273,500,84.71,1094,75.05
334+2026-10-28 09:16:00,1936,237,501,87.76,1249,81.02
335+2026-10-28 09:17:00,2398,282,502,88.24,1534,81.62
336+2026-10-28 09:18:00,2723,512,838,81.2,1866,72.56
337+2026-10-28 09:20:00,1544,274,500,82.25,965,71.61
338+2026-10-28 09:21:00,1895,385,738,79.68,1208,68.13
339+2026-10-28 09:22:00,4158,642,1262,84.56,2599,75.3
340+2026-10-28 09:23:00,2339,386,785,83.5,1519,74.59
341+2026-10-28 09:24:00,3375,484,1010,85.66,2235,78.34
342+2026-10-28 09:25:00,2853,458,816,83.95,1969,76.74
343+2026-10-28 09:26:00,1640,259,500,84.21,1140,77.28
344+2026-10-28 09:27:00,3412,514,1003,84.94,2280,77.46
345+2026-10-28 09:28:00,701,145,257,79.32,470,69.15
346+2026-10-28 09:29:00,2135,292,500,86.32,1234,76.34
347+2026-10-28 09:30:00,4042,548,1002,86.44,2534,78.37
348+2026-10-28 09:31:00,1107,164,246,85.19,758,78.36
349+2026-10-28 09:32:00,3346,534,1000,84.04,2156,75.23
350+2026-10-28 09:33:00,3328,464,1000,86.06,2126,78.17
351+2026-10-28 09:34:00,1258,183,443,85.45,825,77.82
352+2026-10-28 09:35:00,3230,547,1004,83.07,2154,74.61
353+2026-10-28 09:36:00,2027,303,500,85.05,1342,77.42
354+2026-10-28 09:37:00,1949,266,504,86.35,1323,79.89
355+2026-10-28 09:38:00,1,1,9,0.0,1,0.0
356+2026-10-28 09:39:00,1662,285,500,82.85,1190,76.05
357+2026-10-28 09:40:00,1364,274,502,79.91,938,70.79
358+2026-10-28 09:41:00,1501,270,500,82.01,1051,74.31
359+2026-10-28 09:42:00,3387,563,1077,83.38,2194,74.34
360+2026-10-28 09:44:00,4706,554,1003,88.23,2929,81.09
361+2026-10-28 09:45:00,1667,288,501,82.72,1101,73.84
362+2026-10-28 09:46:00,481,81,142,83.16,314,74.2
363+2026-10-28 09:47:00,1859,261,501,85.96,1188,78.03
364+2026-10-28 09:48:00,2045,242,500,88.17,1300,81.38
365+2026-10-28 09:49:00,1712,241,502,85.92,1114,78.37
366+2026-10-28 09:50:00,954,144,288,84.91,640,77.5
367+2026-10-28 09:51:00,2343,315,500,86.56,1456,78.37
368+2026-10-28 09:52:00,4092,551,1000,86.53,2524,78.17
369+2026-10-28 09:53:00,2366,453,796,80.85,1578,71.29
370+2026-10-28 09:54:00,2958,379,803,87.19,2011,81.15
371+2026-10-28 09:55:00,2016,253,495,87.45,1325,80.91
372+2026-10-28 09:56:00,1930,302,500,84.35,1259,76.01
373+2026-10-28 09:57:00,3550,521,1001,85.32,2255,76.9
374+2026-10-28 09:58:00,250,54,118,78.4,164,67.07
375+2026-10-28 09:59:00,2636,480,1002,81.79,1758,72.7
376+2026-10-28 10:00:00,2778,541,1132,80.53,1783,69.66
377+2026-10-28 10:01:00,2672,434,757,83.76,1833,76.32
378+2026-10-28 10:02:00,1443,261,500,81.91,915,71.48
379+2026-10-28 10:03:00,2046,363,701,82.26,1329,72.69
380+2026-10-28 10:05:00,2158,263,502,87.81,1300,79.77
381+2026-10-28 10:06:00,1729,252,500,85.43,1037,75.7
382+2026-10-28 10:07:00,1366,188,398,86.24,875,78.51
383+2026-10-28 10:08:00,2960,535,1000,81.93,1859,71.22
384+2026-10-28 10:09:00,1283,260,500,79.73,833,68.79
385+2026-10-28 10:10:00,1234,257,524,79.17,821,68.7
386+2026-10-28 10:12:00,1933,261,500,86.5,1288,79.74
387+2026-10-28 10:13:00,1230,205,502,83.33,775,73.55
388+2026-10-28 10:14:00,1154,219,502,81.02,725,69.79
389+2026-10-28 10:15:00,1282,232,500,81.9,783,70.37
390+2026-10-28 10:16:00,103,27,54,73.79,81,66.67
391+2026-10-28 10:17:00,1641,188,330,88.54,1040,81.92
392+2026-10-28 10:18:00,2170,390,676,82.03,1357,71.26
393+2026-10-28 10:19:00,959,205,379,78.62,622,67.04
394+2026-10-28 10:20:00,1900,318,501,83.26,1279,75.14
395+2026-10-28 10:21:00,1569,274,500,82.54,1035,73.53
396+2026-10-28 10:22:00,1836,297,508,83.82,1263,76.48
397+2026-10-28 10:23:00,1781,287,500,83.89,1211,76.3
398+2026-10-28 10:24:00,3307,580,1028,82.46,2244,74.15
399+2026-10-28 10:25:00,3271,506,1002,84.53,1960,74.18
400+2026-10-28 10:26:00,2706,442,823,83.67,1677,73.64
401+2026-10-28 10:27:00,1873,268,502,85.69,1180,77.29
402+2026-10-28 10:28:00,2894,470,939,83.76,1701,72.37
403+2026-10-28 10:30:00,1424,258,500,81.88,944,72.67
404+2026-10-28 10:31:00,1762,264,500,85.02,1067,75.26
405+2026-10-28 10:32:00,956,182,324,80.96,597,69.51
406+2026-10-28 10:33:00,1719,266,501,84.53,1079,75.35
407+2026-10-28 10:35:00,2108,356,769,83.11,1272,72.01
408+2026-10-28 10:36:00,1089,218,411,79.98,676,67.75
409+2026-10-28 10:37:00,2163,314,504,85.48,1381,77.26
410+2026-10-28 10:38:00,3278,622,1174,81.03,2139,70.92
411+2026-10-28 10:39:00,2239,386,813,82.76,1396,72.35
412+2026-10-28 10:40:00,1705,270,500,84.16,1139,76.29
413+2026-10-28 10:41:00,2168,325,500,85.01,1463,77.79
414+2026-10-28 10:42:00,108,27,41,75.0,93,70.97
415+2026-10-28 10:43:00,1788,277,500,84.51,1033,73.18
416+2026-10-28 10:44:00,1830,285,502,84.43,1027,72.25
417+2026-10-28 10:45:00,1708,339,654,80.15,1074,68.44
418+2026-10-28 10:46:00,2240,437,1002,80.49,1389,68.54
419+2026-10-28 10:47:00,338,65,149,80.77,204,68.14
420+2026-10-28 10:48:00,1456,277,500,80.98,988,71.96
421+2026-10-28 10:49:00,1414,247,501,82.53,863,71.38
422+2026-10-28 10:50:00,1144,253,427,77.88,743,65.95
423+2026-10-28 10:52:00,1620,296,501,81.73,1080,72.59
424+2026-10-28 10:53:00,1600,270,500,83.13,1048,74.24
425+2026-10-28 10:54:00,593,99,188,83.31,403,75.43
426+2026-10-28 10:56:00,1832,264,500,85.59,1199,77.98
427+2026-10-28 10:57:00,1956,357,695,81.75,1223,70.81
428+2026-10-28 10:59:00,1737,284,501,83.65,1180,75.93
429+2026-10-28 11:00:00,3579,621,1066,82.65,2315,73.17
430+2026-10-28 11:01:00,26,11,25,57.69,15,26.67
431+2026-10-28 11:02:00,1567,248,429,84.17,985,74.82
432+2026-10-28 11:03:00,1817,293,500,83.87,1155,74.63
433+2026-10-28 11:04:00,827,163,282,80.29,538,69.7
434+2026-10-28 11:05:00,1560,273,503,82.5,961,71.59
435+2026-10-28 11:06:00,908,214,377,76.43,560,61.79
436+2026-10-28 11:07:00,1585,292,501,81.58,990,70.51
437+2026-10-28 11:08:00,1166,245,423,78.99,747,67.2
438+2026-10-28 11:09:00,667,105,186,84.26,411,74.45
439+2026-10-28 11:10:00,1764,463,752,73.75,1183,60.86
440+2026-10-28 11:12:00,1925,331,550,82.81,1175,71.83
441+2026-10-28 11:13:00,610,110,199,81.97,383,71.28
442+2026-10-28 11:14:00,2378,465,797,80.45,1520,69.41
443+2026-10-28 11:15:00,1669,311,513,81.37,1067,70.85
added results/ledger/ledger_collisions.csv +305 −0
Large diff (305 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/ledger_collisions_by_form.csv +4 −0
Large diff (4 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/log_failures_by_hour.csv +17 −0
Large diff (17 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/log_summary.json +1021 −0
Large diff (1,022 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/manifest.json +49 −0
Large diff (50 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/resume_batch1_vs_expected.csv +14 −0
Large diff (14 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/table3_ledger_vs_stored.csv +4 −0
Large diff (4 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/throughput.json +105 −0
Large diff (107 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme
↑↓ navigate ↵ open esc close / or ⌘K anywhere