SPB Git

spb/firc0001_uqo

Public

N1: ledger reconciliation (gate pass / true call count, backfilled block, outage, collisions, logs, throughput)

Items: FIRC0001-EDIT-02 EDIT-03 EDIT-18 ADVE-03 ADVE-05 ADVE-07 AI_C-01 AI_C-07 DATA-02 DATA-03 DATA-14
ECON-11 ECON-12 THEO-02 THEO-05 THEO-16 EDIT-05 ADVE-06 AI_C-08 DATA-04 ECON-06 LITE-07 THEO-10 DATA-08 ADVE-12 EDIT-15.
Read-only DuckDB on M3U96b (12 s). 703,725 of 1,123,098 ledger rows pass the gate (true calls);
backfilled block 154,964 = 153,894 + 1,070 rows < 30 words; 0 ledger rows inside the 5.25 h outage gap;
304 sections / 412 collided mentions; 14 silent JSON-decode failures outside the outage plus one
anomalous post-resume batch (2,686 calls, 23 productive vs 536 expected); 149,352 reconciled from the log.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed (Nov 5, 2026) parent 7680606 Browse files

22 changed files +3,078 −0

added code/audit/a01_ledger_reconciliation.py +374 −0
@@ -0,0 +1,374 @@
1 +"""N1 - ledger reconciliation (clusters C, G, J-throughput, H-log numbers). Read-only.
2 +
3 +(a) _PROJECT_GATE over every ledger section (DuckDB RE2, case-insensitive) -> gate pass and the
4 + true call count by form x item x run; productive rate / mentions / USD per call.
5 +(b) backfilled block (rows at min(extracted_at)), word_count < 30 rows, outage-window rows,
6 + zero-mention share per flush timestamp and per 5-minute bin.
7 +(c) sections where ledger n_mentions > stored mentions (id collisions).
8 +(d) run-log parser (October data/logs/spid_extract.log, June project-db/logs/extract.log).
9 +(e) wall-clock / outage / effective throughput from ledger timestamps.
10 +(f) backend values, confidence = 0.600 count, amount_raw formatting rule, min confidence by run.
11 +"""
12 +from __future__ import annotations
13 +import json, re, sys, datetime as dt
14 +from pathlib import Path
15 +from collections import Counter, defaultdict
16 +sys.path.insert(0, str(Path(__file__).resolve().parent))
17 +from common import *
18 +
19 +GATE = (r"\b(project|initiative|program|invest|expan|construct|build|launch|develop|"
20 + r"deploy|implement|acqui|partner|facilit|capacity|capital|migrat|transform|"
21 + r"pipeline|plant|data ?cent|automat|sustainab|renewable|restructur)\w*")
22 +OUTAGE_START = "2026-10-28 01:10:00" # window named in the consolidated review (THEO-05 / DATA-02)
23 +OUTAGE_END = "2026-10-28 06:22:00"
24 +PAPER_USD_TOTAL = 775.0 # construct.tex:35 'roughly 775 USD'
25 +PAPER_USD_PER_SECTION = 0.0008 # construct.tex:35
26 +JUNE_LOG_SECTIONS = 153894 # project-db/logs/extract.log FULL EXTRACT COMPLETE
27 +JUNE_LOG_MENTIONS = 39930 # TOTAL mentions in DB after the June run
28 +
29 +r = Run("ledger")
30 +con = connect(SPID_DB)
31 +
32 +# ---- (a) section metadata + gate flag for every section visible to the pipeline --------------
33 +print("building sec_meta (regex over the parquet text, streamed) ...", flush=True)
34 +con.execute(f"""
35 +CREATE TEMP TABLE sec_meta AS
36 +SELECT section_id, accession_number, section_name, word_count, length(text) AS nchar,
37 + regexp_matches(text, '{GATE}', 'i') AS gate, 'parsed' AS segmenter
38 +FROM src_sections
39 +UNION ALL
40 +SELECT section_id, accession_number, section_name, word_count, length(text) AS nchar,
41 + regexp_matches(text, '{GATE}', 'i') AS gate, 'legacy_ingest' AS segmenter
42 +FROM spid_sections
43 +""")
44 +r.notes["sec_meta_rows"] = con.execute("select count(*) from sec_meta").fetchone()[0]
45 +
46 +con.execute("""
47 +CREATE TEMP TABLE stored AS
48 +SELECT section_id, count(*) AS stored_mentions,
49 + list(project_type || ':' || left(project_name, 40)) AS stored_keys
50 +FROM project_mentions GROUP BY 1
51 +""")
52 +tmin = con.execute("select min(extracted_at) from extracted_sections").fetchone()[0]
53 +r.notes["min_extracted_at"] = str(tmin)
54 +con.execute(f"""
55 +CREATE TEMP TABLE ledger AS
56 +SELECT e.section_id, e.accession_number, e.backend, e.n_mentions, e.extracted_at,
57 + CASE WHEN e.extracted_at = TIMESTAMP '{tmin}' THEN 'backfilled_june' ELSE 'october' END AS run,
58 + m.section_name, m.word_count, m.nchar, m.gate, m.segmenter,
59 + f.form_type, f.filing_date, f.cik, f.fiscal_year,
60 + co.in_sp500, co.in_russell3000,
61 + coalesce(s.stored_mentions, 0) AS stored_mentions, s.stored_keys
62 +FROM extracted_sections e
63 +LEFT JOIN sec_meta m USING (section_id)
64 +LEFT JOIN src_filings f ON f.accession_number = e.accession_number
65 +LEFT JOIN src_companies co ON f.cik = co.cik
66 +LEFT JOIN stored s USING (section_id)
67 +""")
68 +n_ledger = con.execute("select count(*) from ledger").fetchone()[0]
69 +n_unmatched = con.execute("select count(*) from ledger where section_name is null").fetchone()[0]
70 +r.notes["ledger_rows"] = n_ledger
71 +r.notes["ledger_rows_without_section_metadata"] = n_unmatched
72 +
73 +agg = """
74 + count(*) AS ledger_rows,
75 + count(*) FILTER (WHERE word_count < 30) AS lt30_words,
76 + count(*) FILTER (WHERE gate) AS gate_pass,
77 + count(*) FILTER (WHERE gate AND word_count >= 30) AS calls,
78 + count(*) FILTER (WHERE NOT gate AND n_mentions = 0) AS gate_rejected_zero,
79 + count(*) FILTER (WHERE NOT gate AND n_mentions > 0) AS gate_fail_but_mentions,
80 + count(*) FILTER (WHERE n_mentions > 0) AS with_mentions,
81 + sum(n_mentions) AS ledger_mentions,
82 + sum(stored_mentions) AS stored_mentions,
83 + count(DISTINCT cik) AS issuers,
84 + round(100.0 * count(*) FILTER (WHERE n_mentions > 0) / count(*), 2) AS productive_pct_ledger,
85 + round(100.0 * count(*) FILTER (WHERE n_mentions > 0) / nullif(count(*) FILTER (WHERE gate AND word_count >= 30),0), 2) AS productive_pct_calls,
86 + round(sum(n_mentions) * 1.0 / count(*), 4) AS mentions_per_ledger_row,
87 + round(sum(n_mentions) * 1.0 / nullif(count(*) FILTER (WHERE gate AND word_count >= 30),0), 4) AS mentions_per_call
88 +"""
89 +r.dump("gate_pass_by_form_item.csv", con, f"SELECT form_type, section_name AS item, run, {agg} FROM ledger GROUP BY 1,2,3 ORDER BY 1,2,3")
90 +r.dump("gate_pass_by_form.csv", con, f"SELECT form_type, {agg} FROM ledger GROUP BY 1 ORDER BY 1")
91 +r.dump("gate_pass_by_run.csv", con, f"SELECT run, {agg} FROM ledger GROUP BY 1 ORDER BY 1")
92 +r.dump("gate_pass_by_form_run.csv", con, f"SELECT form_type, run, {agg} FROM ledger GROUP BY 1,2 ORDER BY 1,2")
93 +tot = con.execute(f"SELECT {agg} FROM ledger").fetchone()
94 +cols = ["ledger_rows","lt30_words","gate_pass","calls","gate_rejected_zero","gate_fail_but_mentions","with_mentions","ledger_mentions","stored_mentions","issuers","productive_pct_ledger","productive_pct_calls","mentions_per_ledger_row","mentions_per_call"]
95 +T = dict(zip(cols, tot))
96 +oct_calls = con.execute("select count(*) filter (where gate and word_count>=30) from ledger where run='october'").fetchone()[0]
97 +oct_rows = con.execute("select count(*) from ledger where run='october'").fetchone()[0]
98 +T["usd_per_call_paper_total_775"] = round(PAPER_USD_TOTAL / oct_calls, 6)
99 +T["usd_per_october_row_paper_total_775"] = round(PAPER_USD_TOTAL / oct_rows, 6)
100 +T["usd_per_call_from_0_0008_x_ledger"] = round(PAPER_USD_PER_SECTION * n_ledger / T["calls"], 6)
101 +T["october_rows"] = oct_rows; T["october_calls"] = oct_calls
102 +T["headline_before"] = {"sections_submitted": 1123098, "productive_pct": 15.0, "mentions_per_section": 0.27, "usd_per_section": 0.0008}
103 +r.write_json("gate_pass_totals.json", T)
104 +
105 +# ---- (b) backfilled block, < 30 words, outage window, flush timestamps -----------------------
106 +B = {}
107 +B["min_extracted_at"] = str(tmin)
108 +B["rows_at_min_timestamp"] = con.execute("select count(*) from ledger where run='backfilled_june'").fetchone()[0]
109 +B["june_log_sections"] = JUNE_LOG_SECTIONS
110 +B["gap_rows_minus_log"] = B["rows_at_min_timestamp"] - JUNE_LOG_SECTIONS
111 +B["block_ledger_mentions"] = con.execute("select sum(n_mentions) from ledger where run='backfilled_june'").fetchone()[0]
112 +B["block_stored_mentions"] = con.execute("select sum(stored_mentions) from ledger where run='backfilled_june'").fetchone()[0]
113 +B["june_log_mentions"] = JUNE_LOG_MENTIONS
114 +B["block_lt30_words"] = con.execute("select count(*) from ledger where run='backfilled_june' and word_count<30").fetchone()[0]
115 +B["block_gate_fail"] = con.execute("select count(*) from ledger where run='backfilled_june' and not gate").fetchone()[0]
116 +B["block_lt30_or_gate_fail"] = con.execute("select count(*) from ledger where run='backfilled_june' and (word_count<30 or not gate)").fetchone()[0]
117 +B["block_ge30_and_gate"] = con.execute("select count(*) from ledger where run='backfilled_june' and word_count>=30 and gate").fetchone()[0]
118 +B["october_lt30_words"] = con.execute("select count(*) from ledger where run='october' and word_count<30").fetchone()[0]
119 +B["block_by_form"] = con.execute("select form_type, count(*), sum(n_mentions), count(*) filter (where word_count<30) from ledger where run='backfilled_june' group by 1 order by 1").fetchall()
120 +B["block_by_segmenter"] = con.execute("select segmenter, count(*) from ledger where run='backfilled_june' group by 1").fetchall()
121 +B["block_sp500_share"] = con.execute("select count(*) filter (where in_sp500), count(*) from ledger where run='backfilled_june'").fetchone()
122 +B["block_ciks"] = con.execute("select count(distinct cik) from ledger where run='backfilled_june'").fetchone()[0]
123 +B["outage_window"] = [OUTAGE_START, OUTAGE_END]
124 +B["rows_in_outage_window"] = con.execute(f"select count(*), sum(n_mentions) from ledger where extracted_at between TIMESTAMP '{OUTAGE_START}' and TIMESTAMP '{OUTAGE_END}'").fetchone()
125 +B["rows_0110_0640"] = con.execute("select count(*), sum(n_mentions) from ledger where extracted_at between TIMESTAMP '2026-10-28 01:10:00' and TIMESTAMP '2026-10-28 06:40:53'").fetchone()
126 +B["rows_by_hour_1028"] = con.execute("select hour(extracted_at) h, count(*), count(*) filter (where n_mentions=0) zeros, sum(n_mentions) from ledger where run='october' and extracted_at::date = DATE '2026-10-28' group by 1 order by 1").fetchall()
127 +B["rows_by_hour_1027"] = con.execute("select hour(extracted_at) h, count(*), count(*) filter (where n_mentions=0) zeros, sum(n_mentions) from ledger where run='october' and extracted_at::date = DATE '2026-10-27' group by 1 order by 1").fetchall()
128 +B["paper_outage_rows_149352"] = 149352
129 +r.write_json("backfill_block.json", B)
130 +
131 +r.dump("ledger_by_5min.csv", con, """
132 +SELECT time_bucket(INTERVAL '5 minutes', extracted_at) AS bin5, count(*) AS rows, count(*) FILTER (WHERE n_mentions>0) AS with_mentions,
133 + sum(n_mentions) AS mentions, round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
134 + count(*) FILTER (WHERE gate AND word_count>=30) AS calls, count(DISTINCT extracted_at) AS flushes
135 +FROM ledger WHERE run='october' GROUP BY 1 ORDER BY 1""")
136 +r.dump("ledger_by_minute.csv", con, """
137 +SELECT time_bucket(INTERVAL '1 minute', extracted_at) AS bin1, count(*) AS rows, count(*) FILTER (WHERE n_mentions>0) AS with_mentions,
138 + sum(n_mentions) AS mentions, round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
139 + count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
140 + round(100.0*count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30)/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0),2) AS zero_pct_among_calls
141 +FROM ledger WHERE run='october' GROUP BY 1 ORDER BY 1""")
142 +B["october_distinct_timestamps"], B["october_rows_check"] = con.execute("select count(distinct extracted_at), count(*) from ledger where run='october'").fetchone()
143 +B["timestamp_granularity_note"] = "extracted_at is stamped per row (executemany INSERT OR REPLACE, DEFAULT current_timestamp), not per flush: distinct timestamps = rows"
144 +r.dump("flush_zero_groups_after_0622.csv", con, f"""
145 +SELECT time_bucket(INTERVAL '1 minute', extracted_at) AS bin1, count(*) AS rows, count(*) FILTER (WHERE n_mentions=0) AS zeros,
146 + round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
147 + count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
148 + round(100.0*count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30)/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0),2) AS zero_pct_among_calls
149 +FROM ledger WHERE run='october' AND extracted_at > TIMESTAMP '{OUTAGE_END}'
150 +GROUP BY 1 HAVING count(*) >= 50 AND count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30) >= 0.99*count(*) FILTER (WHERE gate AND word_count>=30) ORDER BY 1""")
151 +# the same test before the outage, for comparison
152 +r.dump("flush_zero_groups_before_0110.csv", con, f"""
153 +SELECT time_bucket(INTERVAL '1 minute', extracted_at) AS bin1, count(*) AS rows, count(*) FILTER (WHERE n_mentions=0) AS zeros,
154 + round(100.0*count(*) FILTER (WHERE n_mentions=0)/count(*),2) AS zero_pct,
155 + count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
156 + round(100.0*count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30)/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0),2) AS zero_pct_among_calls
157 +FROM ledger WHERE run='october' AND extracted_at < TIMESTAMP '{OUTAGE_START}'
158 +GROUP BY 1 HAVING count(*) >= 50 AND count(*) FILTER (WHERE n_mentions=0 AND gate AND word_count>=30) >= 0.99*count(*) FILTER (WHERE gate AND word_count>=30) ORDER BY 1""")
159 +
160 +# ---- (c) id collisions: ledger n_mentions > stored ------------------------------------------
161 +r.dump("ledger_collisions.csv", con, """
162 +SELECT section_id, accession_number, form_type, section_name AS item, cik, run, n_mentions AS ledger_n, stored_mentions AS stored_n,
163 + n_mentions - stored_mentions AS deficit, stored_keys
164 +FROM ledger WHERE n_mentions > stored_mentions ORDER BY deficit DESC, section_id""")
165 +r.dump("ledger_collisions_by_form.csv", con, """
166 +SELECT form_type, count(*) AS sections, sum(n_mentions - stored_mentions) AS excess, sum(n_mentions) AS ledger_mentions, sum(stored_mentions) AS stored_mentions
167 +FROM ledger WHERE n_mentions > stored_mentions GROUP BY 1 ORDER BY 1""")
168 +C = {}
169 +C["sections_ledger_gt_stored"], C["excess_mentions"] = con.execute("select count(*), sum(n_mentions-stored_mentions) from ledger where n_mentions > stored_mentions").fetchone()
170 +C["sections_ledger_lt_stored"], C["deficit_mentions"] = con.execute("select count(*), sum(stored_mentions-n_mentions) from ledger where n_mentions < stored_mentions").fetchone()
171 +C["sum_ledger_n_mentions"] = con.execute("select sum(n_mentions) from ledger").fetchone()[0]
172 +C["stored_project_mentions"] = con.execute("select count(*) from project_mentions").fetchone()[0]
173 +C["mentions_whose_section_not_in_ledger"] = con.execute("select count(*) from project_mentions m where not exists (select 1 from extracted_sections e where e.section_id=m.section_id)").fetchone()[0]
174 +C["ledger_vs_stored_by_form"] = con.execute("select form_type, sum(n_mentions), sum(stored_mentions), sum(n_mentions)-sum(stored_mentions) from ledger group by 1 order by 1").fetchall()
175 +C["duplicate_key_sections_in_stored"] = con.execute("""
176 + select count(*) from (select section_id from project_mentions group by section_id, project_type, left(project_name,40) having count(*)>1)""").fetchone()[0]
177 +r.write_json("collisions.json", C)
178 +
179 +# ---- (f) misc code-vs-data checks ---------------------------------------------------------------
180 +F = {}
181 +F["distinct_backend_ledger"] = [x[0] for x in con.execute("select distinct backend from extracted_sections").fetchall()]
182 +F["distinct_backend_mentions"] = [x[0] for x in con.execute("select distinct backend from project_mentions").fetchall()]
183 +F["confidence_exact_0600_total"] = con.execute("select count(*) from project_mentions where round(confidence,3)=0.600").fetchone()[0]
184 +F["confidence_exact_0600_by_run"] = con.execute("""
185 + select l.run, count(*) from project_mentions m join ledger l using(section_id) where round(m.confidence,3)=0.600 group by 1""").fetchall()
186 +F["confidence_bins_by_run"] = con.execute("""
187 + select l.run, round(m.confidence,1) c, count(*) from project_mentions m join ledger l using(section_id) group by 1,2 order by 1,2""").fetchall()
188 +F["min_confidence_by_run"] = con.execute("select l.run, min(m.confidence), max(m.confidence), count(*) from project_mentions m join ledger l using(section_id) group by 1").fetchall()
189 +F["mentions_below_045"] = con.execute("select count(*) from project_mentions where confidence < 0.45").fetchone()[0]
190 +F["amount_raw_rule_mismatch"] = con.execute("""
191 + select count(*) from project_mentions where amount_usd is not null and amount_raw is distinct from ('$' || format('{:,.0f}', amount_usd))""").fetchone()[0]
192 +F["amount_raw_rule_mismatch_rows"] = con.execute("""
193 + select mention_id, amount_usd, amount_raw from project_mentions where amount_usd is not null and amount_raw is distinct from ('$' || format('{:,.0f}', amount_usd)) limit 20""").fetchall()
194 +F["amount_raw_null_with_amount"] = con.execute("select count(*) from project_mentions where amount_usd is not null and amount_raw is null").fetchone()[0]
195 +F["amount_raw_notnull_without_amount"] = con.execute("select count(*) from project_mentions where amount_usd is null and amount_raw is not null").fetchone()[0]
196 +F["amount_le_0"] = con.execute("select count(*) from project_mentions where amount_usd <= 0").fetchone()[0]
197 +F["mentions_with_run"] = con.execute("select l.run, count(*) from project_mentions m join ledger l using(section_id) group by 1").fetchall()
198 +F["mentions_backfilled_block_vs_june_log"] = {"block_stored": B["block_stored_mentions"], "june_log": JUNE_LOG_MENTIONS}
199 +F["fiscal_year_null_mentions_by_run"] = con.execute("select l.run, count(*) filter (where m.fiscal_year is null), count(*) from project_mentions m join ledger l using(section_id) group by 1").fetchall()
200 +r.write_json("code_vs_data_checks.json", F)
201 +
202 +# ---- (e) throughput from the ledger ----------------------------------------------------------
203 +E = {}
204 +ts = con.execute("select distinct extracted_at from ledger where run='october' order by 1").fetchall()
205 +ts = [t[0] for t in ts]
206 +E["october_first_flush"] = str(ts[0]); E["october_last_flush"] = str(ts[-1])
207 +E["october_distinct_flush_timestamps"] = len(ts)
208 +E["october_rows"] = oct_rows
209 +E["october_calls"] = oct_calls
210 +wall = (ts[-1] - ts[0]).total_seconds()
211 +gaps = sorted(((ts[i+1]-ts[i]).total_seconds(), str(ts[i]), str(ts[i+1])) for i in range(len(ts)-1))
212 +E["largest_gaps_seconds"] = gaps[-5:]
213 +outage_gap = gaps[-1][0]
214 +E["wall_clock_hours_first_to_last_flush"] = round(wall/3600, 3)
215 +E["largest_gap_hours"] = round(outage_gap/3600, 3)
216 +E["effective_hours_excluding_largest_gap"] = round((wall-outage_gap)/3600, 3)
217 +E["rows_per_second_wall"] = round(oct_rows/wall, 2)
218 +E["rows_per_second_effective"] = round(oct_rows/(wall-outage_gap), 2)
219 +E["calls_per_second_effective"] = round(oct_calls/(wall-outage_gap), 2)
220 +E["paper_claim"] = {"sections": 968134, "hours": "about twelve", "rate": "27-36 sections per second", "workers": 48}
221 +E["paper_rate_implied_by_12h"] = round(968134/(12*3600), 2)
222 +# segment rates: before the outage and after
223 +pre = con.execute(f"select count(*), min(extracted_at), max(extracted_at) from ledger where run='october' and extracted_at < TIMESTAMP '{OUTAGE_START}'").fetchone()
224 +post = con.execute(f"select count(*), min(extracted_at), max(extracted_at) from ledger where run='october' and extracted_at > TIMESTAMP '{OUTAGE_END}'").fetchone()
225 +E["segment_before_outage"] = {"rows": pre[0], "from": str(pre[1]), "to": str(pre[2]), "rows_per_s": round(pre[0]/max(1,(pre[2]-pre[1]).total_seconds()),2)}
226 +E["segment_after_outage"] = {"rows": post[0], "from": str(post[1]), "to": str(post[2]), "rows_per_s": round(post[0]/max(1,(post[2]-post[1]).total_seconds()),2)}
227 +E["rows_by_hour_october"] = con.execute("select date_trunc('hour', extracted_at) h, count(*) from ledger where run='october' group by 1 order by 1").fetchall()
228 +r.write_json("throughput.json", E)
229 +
230 +# ---- (d) log parsers ---------------------------------------------------------------------------
231 +LINE = re.compile(r"^(\d\d):(\d\d):(\d\d) (\w+)\s+(\S+) \| (.*)$")
232 +def parse_log(path: Path, start_date: dt.date):
233 + day = start_date; prev = None; out = []
234 + for raw in open(path, errors="replace"):
235 + m = LINE.match(raw.rstrip("\n"))
236 + if not m:
237 + out.append((None, None, None, raw.rstrip("\n"))); continue
238 + h, mi, s = int(m.group(1)), int(m.group(2)), int(m.group(3))
239 + t = dt.time(h, mi, s)
240 + if prev is not None and (dt.datetime.combine(day, prev) - dt.datetime.combine(day, t)).total_seconds() > 3600:
241 + day = day + dt.timedelta(days=1)
242 + prev = t
243 + out.append((dt.datetime.combine(day, t), m.group(4), m.group(5), m.group(6)))
244 + return out
245 +
246 +def classify(msg: str) -> str:
247 + if "retryable" in msg: return "retryable_http_429_5xx"
248 + if "Connection reset" in msg or "ConnectError" in msg or "ReadError" in msg or "Errno" in msg: return "connection_error"
249 + if "Unterminated string" in msg or "Expecting" in msg or "Extra data" in msg or "JSON" in msg: return "json_decode_error"
250 + if "KeyError" in msg: return "key_error"
251 + if "timed out" in msg.lower() or "Timeout" in msg: return "timeout"
252 + return "other"
253 +
254 +def summarize(path: Path, start_date: dt.date, window=None):
255 + rows = parse_log(path, start_date)
256 + S = {"path": str(path), "lines": len(rows), "first": str(next((r_[0] for r_ in rows if r_[0]), None)), "last": str(next((r_[0] for r_ in reversed(rows) if r_[0]), None))}
257 + fails = [(t, classify(msg)) for t, lvl, name, msg in rows if msg and "LLM call failed" in msg]
258 + S["llm_call_failed_total"] = len(fails)
259 + S["llm_call_failed_by_reason"] = dict(Counter(c for _, c in fails))
260 + byhour = Counter((t.strftime("%Y-%m-%d %H") if t else "?") for t, _ in fails)
261 + S["llm_call_failed_by_hour"] = dict(sorted(byhour.items()))
262 + if window:
263 + w0, w1 = window
264 + inside = [(t, c) for t, c in fails if t and w0 <= t <= w1]
265 + outside = [(t, c) for t, c in fails if not (t and w0 <= t <= w1)]
266 + S["window"] = [str(w0), str(w1)]
267 + S["llm_call_failed_inside_window"] = len(inside)
268 + S["llm_call_failed_outside_window"] = len(outside)
269 + S["outside_window_by_reason"] = dict(Counter(c for _, c in outside))
270 + S["outside_window_list"] = [(str(t), c) for t, c in outside]
271 + if fails:
272 + S["first_failure"] = str(min(t for t, _ in fails if t)); S["last_failure"] = str(max(t for t, _ in fails if t))
273 + ins = [t for t, _ in inside]
274 + if ins: S["first_failure_in_window"] = str(min(ins)); S["last_failure_in_window"] = str(max(ins))
275 + S["section_failed"] = sum(1 for t, lvl, name, msg in rows if msg and "section failed" in msg)
276 + S["batch_failed_lines"] = sum(1 for t, lvl, name, msg in rows if msg and "BATCH" in msg and "FAILED" in msg)
277 + S["pending_lines"] = [(str(t), msg) for t, lvl, name, msg in rows if msg and msg.startswith("pending:")]
278 + S["ciks_total_lines"] = [(str(t), msg) for t, lvl, name, msg in rows if msg and msg.startswith("CIKs total")]
279 + S["run_complete_lines"] = [(str(t), msg) for t, lvl, name, msg in rows if msg and ("RUN COMPLETE" in msg or "EXTRACT COMPLETE" in msg)]
280 + ext = [int(re.search(r"Extracting from (\d+) sections", msg).group(1)) for t, lvl, name, msg in rows if msg and "Extracting from" in msg]
281 + S["extracting_from_lines"] = len(ext); S["extracting_from_sum_sections"] = sum(ext)
282 + done = [(int(a), int(b)) for t, lvl, name, msg in rows for a, b in re.findall(r"Extraction complete: (\d+) mentions from (\d+) sections", msg or "")]
283 + S["extraction_complete_lines"] = len(done); S["extraction_complete_sum_sections"] = sum(b for a, b in done); S["extraction_complete_sum_mentions"] = sum(a for a, b in done)
284 + S["workers_values"] = sorted(set(re.findall(r"workers=(\d+)", "\n".join(msg or "" for *_, msg in rows))))
285 + S["batch_values"] = sorted(set(re.findall(r"batch=(\d+)", "\n".join(msg or "" for *_, msg in rows))))
286 + S["model_id_strings"] = sorted(set(re.findall(r"gpt-[\w\-\.]+", open(path, errors="replace").read())))
287 + return S, fails
288 +
289 +w0 = dt.datetime.fromisoformat(OUTAGE_START); w1 = dt.datetime.fromisoformat(OUTAGE_END)
290 +oct_log, oct_fails = summarize(RDB / "data/logs/spid_extract.log", dt.date(2026, 10, 27), (w0, w1))
291 +jun_log, jun_fails = summarize(RDB / "project-db/logs/extract.log", dt.date(2026, 6, 18), None)
292 +# other October log files (small)
293 +others = {}
294 +for nm in ["spid.log", "spid_resume.log", "spid_post.log", "spid_interim.log"]:
295 + p = RDB / "data/logs" / nm
296 + if p.exists():
297 + txt = open(p, errors="replace").read()
298 + others[nm] = {"lines": txt.count("\n"), "head": txt[:1500]}
299 +pend = RDB / "data/logs/spid_pending.json"
300 +if pend.exists():
301 + try:
302 + pj = json.load(open(pend)); others["spid_pending.json"] = {k: (v if not isinstance(v, list) else v[:20]) for k, v in pj.items()} if isinstance(pj, dict) else str(pj)[:1500]
303 + except Exception as ex:
304 + others["spid_pending.json"] = str(ex)
305 +# outage window measured from the log itself: the contiguous block of 'retryable' failures on 2026-10-28 between 01:00 and 07:00
306 +retry_ts = [t for t, c in oct_fails if t and c == "retryable_http_429_5xx" and dt.datetime(2026,10,28,1,0) <= t <= dt.datetime(2026,10,28,7,0)]
307 +oct_log["outage_window_from_log"] = [str(min(retry_ts)), str(max(retry_ts)), len(retry_ts)]
308 +w0l, w1l = min(retry_ts), max(retry_ts)
309 +oct_log["llm_call_failed_outside_log_window"] = [(str(t), c) for t, c in oct_fails if not (t and w0l <= t <= w1l)]
310 +oct_log["silent_empty_results_outside_outage"] = len(oct_log["llm_call_failed_outside_log_window"])
311 +oct_log["retryable_failures_outside_log_window"] = sum(1 for t, c in oct_fails if c == "retryable_http_429_5xx" and not (t and w0l <= t <= w1l))
312 +oct_log["ledger_rows_in_log_window"] = con.execute(f"select count(*), sum(n_mentions) from ledger where extracted_at between TIMESTAMP '{w0l}' and TIMESTAMP '{w1l}'").fetchone()
313 +oct_log["ledger_rows_in_largest_gap"] = con.execute(f"select count(*) from ledger where extracted_at > TIMESTAMP '{gaps[-1][1]}' and extracted_at < TIMESTAMP '{gaps[-1][2]}'").fetchone()[0]
314 +pend2 = int(re.search(r"pending: (\d+)", oct_log["pending_lines"][1][1]).group(1)) if len(oct_log["pending_lines"]) > 1 else None
315 +oct_log["outage_rows_reconciliation"] = {"extraction_complete_sum_sections": oct_log["extraction_complete_sum_sections"], "pending_at_second_start": pend2,
316 + "difference": (oct_log["extraction_complete_sum_sections"] - pend2) if pend2 else None, "paper_research_log_149352": 149352,
317 + "pending_at_resume": int(re.search(r"pending: (\d+)", oct_log["pending_lines"][-1][1]).group(1)),
318 + "rows_before_outage_plus_in_window": con.execute(f"select count(*) from ledger where run='october' and extracted_at < TIMESTAMP '{gaps[-1][2]}'").fetchone()[0]}
319 +# batch-level productivity from the log (silent-failure test, THEO-05): mentions per section per batch
320 +def batch_rows(rows_, run_):
321 + out = []
322 + for t, lvl, name, msg in rows_:
323 + m_ = re.search(r"BATCH (\d+)/(\d+) done \| batch_sec=(\d+) batch_men=(\d+)", msg or "")
324 + if m_:
325 + out.append((run_, int(m_.group(1)), int(m_.group(2)), str(t), int(m_.group(3)), int(m_.group(4)), round(int(m_.group(4))/max(1,int(m_.group(3))), 4)))
326 + return out
327 +oct_rows_ = parse_log(RDB / "data/logs/spid_extract.log", dt.date(2026, 10, 27)); jun_rows_ = parse_log(RDB / "project-db/logs/extract.log", dt.date(2026, 6, 18))
328 +br = batch_rows(oct_rows_, "october") + batch_rows(jun_rows_, "june")
329 +import statistics
330 +med_oct = statistics.median([b[6] for b in br if b[0] == "october"]); med_jun = statistics.median([b[6] for b in br if b[0] == "june"])
331 +br2 = [b + (round(b[6]/(med_oct if b[0]=="october" else med_jun), 3), (b[6] < 0.25*(med_oct if b[0]=="october" else med_jun))) for b in br]
332 +r.write_csv("batch_productivity_from_log.csv", ["run", "batch", "n_batches", "done_at", "sections", "mentions", "mentions_per_section", "ratio_to_median", "anomalous_lt_25pct_of_median"], br2)
333 +oct_log["batch_productivity_median_mentions_per_section"] = med_oct
334 +oct_log["anomalous_batches"] = [b for b in br2 if b[0] == "october" and b[8]]
335 +jun_log["batch_productivity_median_mentions_per_section"] = med_jun
336 +jun_log["anomalous_batches"] = [b for b in br2 if b[0] == "june" and b[8]]
337 +# the first resumed batch: rows stamped between its start and end, with the expected productive count at the form x item rates of the rest of the October run
338 +b1 = [b for b in br2 if b[0] == "october" and b[1] == 1 and b[2] == 81]
339 +if b1:
340 + b1_end = b1[0][3]
341 + b1_start = str(dt.datetime.fromisoformat(oct_log["pending_lines"][-1][0]))
342 + con.execute(f"CREATE TEMP TABLE b1 AS SELECT * FROM ledger WHERE run='october' AND extracted_at BETWEEN TIMESTAMP '{b1_start}' AND TIMESTAMP '{b1_end}'")
343 + r.dump("resume_batch1_vs_expected.csv", con, """
344 + WITH rates AS (SELECT form_type, section_name, count(*) FILTER (WHERE n_mentions>0)*1.0/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0) p_prod, sum(n_mentions)*1.0/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0) m_per_call
345 + FROM ledger WHERE run='october' AND section_id NOT IN (SELECT section_id FROM b1) GROUP BY 1,2)
346 + SELECT b.form_type, b.section_name AS item, count(*) rows, count(*) FILTER (WHERE gate AND word_count>=30) calls, count(*) FILTER (WHERE n_mentions>0) with_mentions, sum(n_mentions) mentions,
347 + round(sum(CASE WHEN gate AND word_count>=30 THEN r.p_prod ELSE 0 END),1) expected_with_mentions, round(sum(CASE WHEN gate AND word_count>=30 THEN r.m_per_call ELSE 0 END),1) expected_mentions,
348 + count(DISTINCT b.cik) ciks
349 + FROM b1 b LEFT JOIN rates r ON b.form_type=r.form_type AND b.section_name=r.section_name GROUP BY 1,2 ORDER BY 1,2""")
350 + oct_log["resume_batch1"] = {"window": [b1_start, b1_end], "log_sections": b1[0][4], "log_mentions": b1[0][5],
351 + "ledger_rows": con.execute("select count(*) from b1").fetchone()[0],
352 + "ledger_calls": con.execute("select count(*) from b1 where gate and word_count>=30").fetchone()[0],
353 + "ledger_with_mentions": con.execute("select count(*) from b1 where n_mentions>0").fetchone()[0],
354 + "ledger_mentions": con.execute("select sum(n_mentions) from b1").fetchone()[0],
355 + "ciks": con.execute("select list(distinct cik order by cik) from b1").fetchone()[0],
356 + "expected_with_mentions": con.execute("""WITH rates AS (SELECT form_type, section_name, count(*) FILTER (WHERE n_mentions>0)*1.0/nullif(count(*) FILTER (WHERE gate AND word_count>=30),0) p FROM ledger WHERE run='october' AND section_id NOT IN (SELECT section_id FROM b1) GROUP BY 1,2)
357 + SELECT round(sum(CASE WHEN gate AND word_count>=30 THEN r.p ELSE 0 END),1) FROM b1 b LEFT JOIN rates r ON b.form_type=r.form_type AND b.section_name=r.section_name""").fetchone()[0],
358 + "failed_call_lines_in_window": sum(1 for t, c in oct_fails if t and dt.datetime.fromisoformat(b1_start) <= t <= dt.datetime.fromisoformat(b1_end))}
359 +r.write_json("log_summary.json", {"october": oct_log, "june": jun_log, "other_logs": others,
360 + "june_log_date_assumed": "2026-06-18 (file mtime; the log carries time-of-day only)"})
361 +r.write_csv("log_failures_by_hour.csv", ["run", "hour", "reason", "n"],
362 + [("october", h, c, n) for (h, c), n in sorted(Counter((t.strftime("%Y-%m-%d %H") if t else "?", c) for t, c in oct_fails).items())] +
363 + [("june", h, c, n) for (h, c), n in sorted(Counter((t.strftime("%Y-%m-%d %H") if t else "?", c) for t, c in jun_fails).items())])
364 +
365 +# ---- Table 3 replacement input ---------------------------------------------------------------
366 +r.dump("table3_ledger_vs_stored.csv", con, """
367 +SELECT form_type, count(*) AS ledger_rows, count(*) FILTER (WHERE gate AND word_count>=30) AS calls,
368 + count(*) FILTER (WHERE n_mentions>0) AS with_mentions, sum(n_mentions) AS returned_by_model_before_dedup,
369 + sum(stored_mentions) AS stored_mentions, count(DISTINCT cik) AS issuers,
370 + count(DISTINCT cik) FILTER (WHERE n_mentions>0) AS issuers_with_mentions
371 +FROM ledger GROUP BY 1 ORDER BY 1""")
372 +r.notes["issuers_total_distinct_in_ledger"] = con.execute("select count(distinct cik) from ledger").fetchone()[0]
373 +r.notes["issuers_with_filings"] = con.execute("select count(distinct cik) from src_filings").fetchone()[0]
374 +r.finish()
added results/ledger/backfill_block.json +132 −0
@@ -0,0 +1,132 @@
1 +{
2 + "min_extracted_at": "2026-10-27 13:12:32.140400",
3 + "rows_at_min_timestamp": 154964,
4 + "june_log_sections": 153894,
5 + "gap_rows_minus_log": 1070,
6 + "block_ledger_mentions": 39930,
7 + "block_stored_mentions": 39930,
8 + "june_log_mentions": 39930,
9 + "block_lt30_words": 1070,
10 + "block_gate_fail": 59666,
11 + "block_lt30_or_gate_fail": 59678,
12 + "block_ge30_and_gate": 95286,
13 + "october_lt30_words": 0,
14 + "block_by_form": [
15 + [
16 + "10-K",
17 + 27164,
18 + 15787,
19 + 0
20 + ],
21 + [
22 + "10-Q",
23 + 23873,
24 + 5817,
25 + 1069
26 + ],
27 + [
28 + "8-K",
29 + 103927,
30 + 18326,
31 + 1
32 + ]
33 + ],
34 + "block_by_segmenter": [
35 + [
36 + "parsed",
37 + 127800
38 + ],
39 + [
40 + "legacy_ingest",
41 + 27164
42 + ]
43 + ],
44 + "block_sp500_share": [
45 + 154964,
46 + 154964
47 + ],
48 + "block_ciks": 500,
49 + "outage_window": [
50 + "2026-10-28 01:10:00",
51 + "2026-10-28 06:22:00"
52 + ],
53 + "rows_in_outage_window": [
54 + 504,
55 + 1011
56 + ],
57 + "rows_0110_0640": [
58 + 5273,
59 + 1041
60 + ],
61 + "rows_by_hour_1028": [
62 + [
63 + 0,
64 + 107883,
65 + 92099,
66 + 30321
67 + ],
68 + [
69 + 1,
70 + 16609,
71 + 12953,
72 + 7061
73 + ],
74 + [
75 + 6,
76 + 35147,
77 + 31638,
78 + 7092
79 + ],
80 + [
81 + 7,
82 + 148277,
83 + 125987,
84 + 41887
85 + ],
86 + [
87 + 8,
88 + 114988,
89 + 98154,
90 + 32310
91 + ],
92 + [
93 + 9,
94 + 130711,
95 + 111241,
96 + 36466
97 + ],
98 + [
99 + 10,
100 + 93253,
101 + 77492,
102 + 29917
103 + ],
104 + [
105 + 11,
106 + 22048,
107 + 17903,
108 + 7103
109 + ]
110 + ],
111 + "rows_by_hour_1027": [
112 + [
113 + 21,
114 + 56405,
115 + 48434,
116 + 14344
117 + ],
118 + [
119 + 22,
120 + 126305,
121 + 107801,
122 + 33512
123 + ],
124 + [
125 + 23,
126 + 116508,
127 + 100609,
128 + 28534
129 + ]
130 + ],
131 + "paper_outage_rows_149352": 149352
132 +}
\ No newline at end of file
added results/ledger/batch_productivity_from_log.csv +176 −0
@@ -0,0 +1,176 @@
1 +run,batch,n_batches,done_at,sections,mentions,mentions_per_section,ratio_to_median,anomalous_lt_25pct_of_median
2 +october,1,125,2026-10-27 21:41:00,8596,1902,0.2213,0.795,False
3 +october,2,125,2026-10-27 21:44:12,8723,2721,0.3119,1.12,False
4 +october,3,125,2026-10-27 21:46:33,9571,2116,0.2211,0.794,False
5 +october,4,125,2026-10-27 21:49:32,9102,2562,0.2815,1.011,False
6 +october,5,125,2026-10-27 21:52:48,8408,1951,0.232,0.833,False
7 +october,6,125,2026-10-27 21:58:36,8984,2089,0.2325,0.835,False
8 +october,7,125,2026-10-27 22:03:29,8954,2939,0.3282,1.179,False
9 +october,8,125,2026-10-27 22:08:22,8299,2793,0.3365,1.209,False
10 +october,9,125,2026-10-27 22:12:28,8588,2371,0.2761,0.992,False
11 +october,10,125,2026-10-27 22:17:45,10363,2742,0.2646,0.95,False
12 +october,11,125,2026-10-27 22:22:29,7837,2227,0.2842,1.021,False
13 +october,12,125,2026-10-27 22:25:14,9610,2085,0.217,0.779,False
14 +october,13,125,2026-10-27 22:28:24,8549,2478,0.2899,1.041,False
15 +october,14,125,2026-10-27 22:31:08,7599,1906,0.2508,0.901,False
16 +october,15,125,2026-10-27 22:35:00,9094,2520,0.2771,0.995,False
17 +october,16,125,2026-10-27 22:37:41,8437,2367,0.2805,1.008,False
18 +october,17,125,2026-10-27 22:40:13,8233,2272,0.276,0.991,False
19 +october,18,125,2026-10-27 22:45:53,9705,2410,0.2483,0.892,False
20 +october,19,125,2026-10-27 22:51:46,10801,2329,0.2156,0.774,False
21 +october,20,125,2026-10-27 22:57:49,10564,2575,0.2438,0.876,False
22 +october,21,125,2026-10-27 23:03:05,10945,2124,0.1941,0.697,False
23 +october,22,125,2026-10-27 23:08:21,8674,2074,0.2391,0.859,False
24 +october,23,125,2026-10-27 23:13:38,9460,2412,0.255,0.916,False
25 +october,24,125,2026-10-27 23:17:51,10644,2469,0.232,0.833,False
26 +october,25,125,2026-10-27 23:23:56,9128,2503,0.2742,0.985,False
27 +october,26,125,2026-10-27 23:29:07,9330,2779,0.2979,1.07,False
28 +october,27,125,2026-10-27 23:37:16,10209,2387,0.2338,0.84,False
29 +october,28,125,2026-10-27 23:41:09,10831,2241,0.2069,0.743,False
30 +october,29,125,2026-10-27 23:44:30,9162,2785,0.304,1.092,False
31 +october,30,125,2026-10-27 23:49:41,9476,2326,0.2455,0.882,False
32 +october,31,125,2026-10-27 23:54:22,10573,2314,0.2189,0.786,False
33 +october,32,125,2026-10-27 23:58:21,9135,2116,0.2316,0.832,False
34 +october,33,125,2026-10-28 00:04:07,9893,3113,0.3147,1.13,False
35 +october,34,125,2026-10-28 00:08:14,9143,2574,0.2815,1.011,False
36 +october,35,125,2026-10-28 00:15:29,10740,3197,0.2977,1.069,False
37 +october,36,125,2026-10-28 00:20:57,11636,3241,0.2785,1.0,False
38 +october,37,125,2026-10-28 00:27:11,9545,2754,0.2885,1.036,False
39 +october,38,125,2026-10-28 00:35:17,9948,2921,0.2936,1.055,False
40 +october,39,125,2026-10-28 00:42:28,11222,3109,0.277,0.995,False
41 +october,40,125,2026-10-28 00:46:24,9764,2819,0.2887,1.037,False
42 +october,41,125,2026-10-28 00:52:03,9792,2612,0.2667,0.958,False
43 +october,42,125,2026-10-28 00:57:42,11237,2485,0.2211,0.794,False
44 +october,43,125,2026-10-28 01:00:46,8950,2712,0.303,1.088,False
45 +october,44,125,2026-10-28 01:05:51,8611,2897,0.3364,1.208,False
46 +october,45,125,2026-10-28 01:09:50,10332,2442,0.2364,0.849,False
47 +october,46,125,2026-10-28 01:25:38,10260,1011,0.0985,0.354,False
48 +october,47,125,2026-10-28 01:44:26,9550,0,0.0,0.0,True
49 +october,48,125,2026-10-28 02:02:20,8726,0,0.0,0.0,True
50 +october,49,125,2026-10-28 02:25:08,10693,0,0.0,0.0,True
51 +october,50,125,2026-10-28 02:45:05,9784,0,0.0,0.0,True
52 +october,51,125,2026-10-28 03:06:35,10260,0,0.0,0.0,True
53 +october,52,125,2026-10-28 03:24:01,9081,0,0.0,0.0,True
54 +october,53,125,2026-10-28 03:47:33,10765,0,0.0,0.0,True
55 +october,54,125,2026-10-28 04:05:52,8367,0,0.0,0.0,True
56 +october,55,125,2026-10-28 04:29:31,10903,0,0.0,0.0,True
57 +october,56,125,2026-10-28 04:47:54,9042,0,0.0,0.0,True
58 +october,57,125,2026-10-28 05:07:03,9047,0,0.0,0.0,True
59 +october,58,125,2026-10-28 05:24:38,8672,0,0.0,0.0,True
60 +october,59,125,2026-10-28 05:45:11,10171,0,0.0,0.0,True
61 +october,60,125,2026-10-28 06:05:10,9337,0,0.0,0.0,True
62 +october,1,81,2026-10-28 06:40:53,5198,34,0.0065,0.023,True
63 +october,2,81,2026-10-28 06:46:05,9756,1931,0.1979,0.711,False
64 +october,3,81,2026-10-28 06:51:56,9550,2749,0.2879,1.034,False
65 +october,4,81,2026-10-28 06:58:08,8726,1877,0.2151,0.773,False
66 +october,5,81,2026-10-28 07:03:04,10693,3043,0.2846,1.022,False
67 +october,6,81,2026-10-28 07:07:54,9784,2259,0.2309,0.829,False
68 +october,7,81,2026-10-28 07:13:48,10260,2884,0.2811,1.01,False
69 +october,8,81,2026-10-28 07:16:26,9081,2827,0.3113,1.118,False
70 +october,9,81,2026-10-28 07:20:31,10765,3159,0.2935,1.054,False
71 +october,10,81,2026-10-28 07:23:33,8367,2630,0.3143,1.129,False
72 +october,11,81,2026-10-28 07:26:57,10903,3031,0.278,0.999,False
73 +october,12,81,2026-10-28 07:30:15,9042,2387,0.264,0.948,False
74 +october,13,81,2026-10-28 07:33:28,9047,2270,0.2509,0.901,False
75 +october,14,81,2026-10-28 07:36:25,8672,2850,0.3286,1.18,False
76 +october,15,81,2026-10-28 07:42:11,10171,3576,0.3516,1.263,False
77 +october,16,81,2026-10-28 07:45:36,9337,2977,0.3188,1.145,False
78 +october,17,81,2026-10-28 07:48:50,10088,2603,0.258,0.927,False
79 +october,18,81,2026-10-28 07:52:13,10182,2359,0.2317,0.832,False
80 +october,19,81,2026-10-28 07:56:34,9101,2533,0.2783,1.0,False
81 +october,20,81,2026-10-28 08:02:06,10155,2515,0.2477,0.89,False
82 +october,21,81,2026-10-28 08:08:53,9509,2811,0.2956,1.062,False
83 +october,22,81,2026-10-28 08:12:58,7641,2316,0.3031,1.089,False
84 +october,23,81,2026-10-28 08:16:15,10020,3346,0.3339,1.199,False
85 +october,24,81,2026-10-28 08:21:04,11340,3469,0.3059,1.099,False
86 +october,25,81,2026-10-28 08:25:46,9397,2494,0.2654,0.953,False
87 +october,26,81,2026-10-28 08:29:08,8219,2316,0.2818,1.012,False
88 +october,27,81,2026-10-28 08:31:50,8764,2560,0.2921,1.049,False
89 +october,28,81,2026-10-28 08:37:51,10213,2003,0.1961,0.704,False
90 +october,29,81,2026-10-28 08:42:17,9579,2587,0.2701,0.97,False
91 +october,30,81,2026-10-28 08:48:43,8960,2706,0.302,1.085,False
92 +october,31,81,2026-10-28 08:54:30,8143,2185,0.2683,0.964,False
93 +october,32,81,2026-10-28 09:00:11,8517,2245,0.2636,0.947,False
94 +october,33,81,2026-10-28 09:03:35,8343,1974,0.2366,0.85,False
95 +october,34,81,2026-10-28 09:07:52,9979,2354,0.2359,0.847,False
96 +october,35,81,2026-10-28 09:10:23,6747,2056,0.3047,1.094,False
97 +october,36,81,2026-10-28 09:13:53,8212,2188,0.2664,0.957,False
98 +october,37,81,2026-10-28 09:18:52,8843,2341,0.2647,0.951,False
99 +october,38,81,2026-10-28 09:23:12,8215,2785,0.339,1.218,False
100 +october,39,81,2026-10-28 09:25:11,6103,1826,0.2992,1.075,False
101 +october,40,81,2026-10-28 09:28:10,7599,2260,0.2974,1.068,False
102 +october,41,81,2026-10-28 09:31:25,7284,1748,0.24,0.862,False
103 +october,42,81,2026-10-28 09:34:27,7932,2443,0.308,1.106,False
104 +october,43,81,2026-10-28 09:38:08,7207,2017,0.2799,1.005,False
105 +october,44,81,2026-10-28 09:42:49,7914,2579,0.3259,1.171,False
106 +october,45,81,2026-10-28 09:46:04,6854,1646,0.2402,0.863,False
107 +october,46,81,2026-10-28 09:50:02,6570,1791,0.2726,0.979,False
108 +october,47,81,2026-10-28 09:53:03,7197,1793,0.2491,0.895,False
109 +october,48,81,2026-10-28 09:55:57,6578,1801,0.2738,0.983,False
110 +october,49,81,2026-10-28 09:58:07,5730,1619,0.2825,1.015,False
111 +october,50,81,2026-10-28 10:01:00,5955,2388,0.401,1.44,False
112 +october,51,81,2026-10-28 10:03:55,5620,1704,0.3032,1.089,False
113 +october,52,81,2026-10-28 10:07:22,5253,1400,0.2665,0.957,False
114 +october,53,81,2026-10-28 10:10:27,5477,2024,0.3695,1.327,False
115 +october,54,81,2026-10-28 10:16:01,5702,2058,0.3609,1.296,False
116 +october,55,81,2026-10-28 10:19:52,4770,1385,0.2904,1.043,False
117 +october,56,81,2026-10-28 10:22:27,5305,1509,0.2844,1.022,False
118 +october,57,81,2026-10-28 10:24:37,5088,1528,0.3003,1.079,False
119 +october,58,81,2026-10-28 10:26:47,5977,1825,0.3053,1.097,False
120 +october,59,81,2026-10-28 10:28:56,4767,1441,0.3023,1.086,False
121 +october,60,81,2026-10-28 10:32:08,4142,1324,0.3197,1.148,False
122 +october,61,81,2026-10-28 10:36:09,4916,1681,0.3419,1.228,False
123 +october,62,81,2026-10-28 10:38:19,4104,1178,0.287,1.031,False
124 +october,63,81,2026-10-28 10:39:44,3576,1313,0.3672,1.319,False
125 +october,64,81,2026-10-28 10:42:00,3981,1041,0.2615,0.939,False
126 +october,65,81,2026-10-28 10:45:01,4153,1156,0.2784,1.0,False
127 +october,66,81,2026-10-28 10:47:13,3751,1651,0.4401,1.581,False
128 +october,67,81,2026-10-28 10:50:35,4014,1428,0.3558,1.278,False
129 +october,68,81,2026-10-28 10:54:21,3813,1189,0.3118,1.12,False
130 +october,69,81,2026-10-28 10:57:53,3788,1195,0.3155,1.133,False
131 +october,70,81,2026-10-28 11:00:19,3557,1092,0.307,1.103,False
132 +october,71,81,2026-10-28 11:02:03,3352,929,0.2771,0.995,False
133 +october,72,81,2026-10-28 11:04:10,2644,782,0.2958,1.062,False
134 +october,73,81,2026-10-28 11:06:07,2468,880,0.3566,1.281,False
135 +october,74,81,2026-10-28 11:08:58,2751,924,0.3359,1.207,False
136 +october,75,81,2026-10-28 11:10:53,2431,938,0.3858,1.386,False
137 +october,76,81,2026-10-28 11:12:55,1925,550,0.2857,1.026,False
138 +october,77,81,2026-10-28 11:14:03,1636,575,0.3515,1.263,False
139 +october,78,81,2026-10-28 11:14:41,1352,421,0.3114,1.119,False
140 +october,79,81,2026-10-28 11:15:07,962,291,0.3025,1.087,False
141 +october,80,81,2026-10-28 11:15:32,499,150,0.3006,1.08,False
142 +october,81,81,2026-10-28 11:15:49,208,72,0.3462,1.244,False
143 +june,1,34,2026-06-18 08:31:49,4271,997,0.2334,0.895,False
144 +june,2,34,2026-06-18 08:36:05,4831,1118,0.2314,0.888,False
145 +june,3,34,2026-06-18 08:40:41,4476,1326,0.2962,1.136,False
146 +june,4,34,2026-06-18 08:44:50,4379,1155,0.2638,1.012,False
147 +june,5,34,2026-06-18 08:48:37,4022,901,0.224,0.859,False
148 +june,6,34,2026-06-18 08:53:37,6315,1178,0.1865,0.715,False
149 +june,7,34,2026-06-18 08:56:53,4460,835,0.1872,0.718,False
150 +june,8,34,2026-06-18 09:03:36,6573,1848,0.2812,1.079,False
151 +june,9,34,2026-06-18 09:09:39,6314,1484,0.235,0.901,False
152 +june,10,34,2026-06-18 09:13:35,4324,932,0.2155,0.827,False
153 +june,11,34,2026-06-18 09:17:47,4619,1225,0.2652,1.017,False
154 +june,12,34,2026-06-18 09:24:11,6177,1750,0.2833,1.087,False
155 +june,13,34,2026-06-18 09:29:01,4633,1302,0.281,1.078,False
156 +june,14,34,2026-06-18 09:32:59,4464,944,0.2115,0.811,False
157 +june,15,34,2026-06-18 09:38:18,4469,1619,0.3623,1.39,False
158 +june,16,34,2026-06-18 09:43:23,4734,1188,0.251,0.963,False
159 +june,17,34,2026-06-18 09:47:38,4186,1028,0.2456,0.942,False
160 +june,18,34,2026-06-18 09:54:22,6003,1721,0.2867,1.1,False
161 +june,19,34,2026-06-18 09:59:29,4939,1243,0.2517,0.965,False
162 +june,20,34,2026-06-18 10:04:17,4784,964,0.2015,0.773,False
163 +june,21,34,2026-06-18 10:09:03,4083,1181,0.2892,1.109,False
164 +june,22,34,2026-06-18 10:13:49,4535,1183,0.2609,1.001,False
165 +june,23,34,2026-06-18 10:20:06,4998,1557,0.3115,1.195,False
166 +june,24,34,2026-06-18 10:26:33,5451,1587,0.2911,1.117,False
167 +june,25,34,2026-06-18 10:32:52,4960,1495,0.3014,1.156,False
168 +june,26,34,2026-06-18 10:39:52,5185,1796,0.3464,1.329,False
169 +june,27,34,2026-06-18 10:44:31,4780,977,0.2044,0.784,False
170 +june,28,34,2026-06-18 10:50:19,5934,1410,0.2376,0.911,False
171 +june,29,34,2026-06-18 10:54:16,3938,1026,0.2605,0.999,False
172 +june,30,34,2026-06-18 10:58:30,4361,947,0.2172,0.833,False
173 +june,31,34,2026-06-18 11:02:06,3019,850,0.2816,1.08,False
174 +june,32,34,2026-06-18 11:05:01,2293,624,0.2721,1.044,False
175 +june,33,34,2026-06-18 11:06:28,1268,323,0.2547,0.977,False
176 +june,34,34,2026-06-18 11:06:44,116,47,0.4052,1.554,False
added results/ledger/code_vs_data_checks.json +228 −0
@@ -0,0 +1,228 @@
1 +{
2 + "distinct_backend_ledger": [
3 + "llm"
4 + ],
5 + "distinct_backend_mentions": [
6 + "llm"
7 + ],
8 + "confidence_exact_0600_total": 3278,
9 + "confidence_exact_0600_by_run": [
10 + [
11 + "october",
12 + 2974
13 + ],
14 + [
15 + "backfilled_june",
16 + 304
17 + ]
18 + ],
19 + "confidence_bins_by_run": [
20 + [
21 + "backfilled_june",
22 + 0.5,
23 + 106
24 + ],
25 + [
26 + "backfilled_june",
27 + 0.6,
28 + 304
29 + ],
30 + [
31 + "backfilled_june",
32 + 0.7,
33 + 2239
34 + ],
35 + [
36 + "backfilled_june",
37 + 0.8,
38 + 8594
39 + ],
40 + [
41 + "backfilled_june",
42 + 0.9,
43 + 14459
44 + ],
45 + [
46 + "backfilled_june",
47 + 1.0,
48 + 14228
49 + ],
50 + [
51 + "october",
52 + 0.5,
53 + 1505
54 + ],
55 + [
56 + "october",
57 + 0.6,
58 + 2974
59 + ],
60 + [
61 + "october",
62 + 0.7,
63 + 19408
64 + ],
65 + [
66 + "october",
67 + 0.8,
68 + 63686
69 + ],
70 + [
71 + "october",
72 + 0.9,
73 + 86372
74 + ],
75 + [
76 + "october",
77 + 1.0,
78 + 94190
79 + ]
80 + ],
81 + "min_confidence_by_run": [
82 + [
83 + "backfilled_june",
84 + 0.5,
85 + 1.0,
86 + 39930
87 + ],
88 + [
89 + "october",
90 + 0.5,
91 + 1.0,
92 + 268135
93 + ]
94 + ],
95 + "mentions_below_045": 0,
96 + "amount_raw_rule_mismatch": 584,
97 + "amount_raw_rule_mismatch_rows": [
98 + [
99 + "734023d0520612d5f402",
100 + 0.0,
101 + null
102 + ],
103 + [
104 + "db4673180782ab45b10b",
105 + 0.0,
106 + null
107 + ],
108 + [
109 + "3c295851c31a0c3e2ec0",
110 + 0.0,
111 + null
112 + ],
113 + [
114 + "63c97357d350022be19b",
115 + 0.0,
116 + null
117 + ],
118 + [
119 + "5b38aa9e9b23848af4e7",
120 + 0.0,
121 + null
122 + ],
123 + [
124 + "0baf489b918ca6a2e565",
125 + 0.0,
126 + null
127 + ],
128 + [
129 + "93537b10c9c3d43fdca3",
130 + 0.0,
131 + null
132 + ],
133 + [
134 + "b043bc1551282d61c4f1",
135 + 0.0,
136 + null
137 + ],
138 + [
139 + "e0a74132c0790e0f8e39",
140 + 0.0,
141 + null
142 + ],
143 + [
144 + "e81dd46e05627eaebb0c",
145 + 0.0,
146 + null
147 + ],
148 + [
149 + "2bc7b4eabca7ddeec540",
150 + 0.0,
151 + null
152 + ],
153 + [
154 + "768d20d20fe24ea2abb2",
155 + 0.0,
156 + null
157 + ],
158 + [
159 + "a8b9c386fa4bb91053a9",
160 + 0.0,
161 + null
162 + ],
163 + [
164 + "1f31eb05dda96af09770",
165 + 0.0,
166 + null
167 + ],
168 + [
169 + "1ea04bf0a611d3e6e8e1",
170 + 0.0,
171 + null
172 + ],
173 + [
174 + "36d0a293409cb18eb1b7",
175 + 0.0,
176 + null
177 + ],
178 + [
179 + "8c55318a560189f234a6",
180 + 0.0,
181 + null
182 + ],
183 + [
184 + "7232cc9db4b89c6d2ed5",
185 + 0.0,
186 + null
187 + ],
188 + [
189 + "a87788f9c89f1bb3d6c6",
190 + 0.0,
191 + null
192 + ],
193 + [
194 + "4d97915cd606d985efc5",
195 + 0.0,
196 + null
197 + ]
198 + ],
199 + "amount_raw_null_with_amount": 584,
200 + "amount_raw_notnull_without_amount": 0,
201 + "amount_le_0": 608,
202 + "mentions_with_run": [
203 + [
204 + "backfilled_june",
205 + 39930
206 + ],
207 + [
208 + "october",
209 + 268135
210 + ]
211 + ],
212 + "mentions_backfilled_block_vs_june_log": {
213 + "block_stored": 39930,
214 + "june_log": 39930
215 + },
216 + "fiscal_year_null_mentions_by_run": [
217 + [
218 + "backfilled_june",
219 + 39930,
220 + 39930
221 + ],
222 + [
223 + "october",
224 + 28897,
225 + 268135
226 + ]
227 + ]
228 +}
\ No newline at end of file
added results/ledger/collisions.json +30 −0
@@ -0,0 +1,30 @@
1 +{
2 + "sections_ledger_gt_stored": 304,
3 + "excess_mentions": 412,
4 + "sections_ledger_lt_stored": 0,
5 + "deficit_mentions": null,
6 + "sum_ledger_n_mentions": 308477,
7 + "stored_project_mentions": 308065,
8 + "mentions_whose_section_not_in_ledger": 0,
9 + "ledger_vs_stored_by_form": [
10 + [
11 + "10-K",
12 + 81622,
13 + 81495,
14 + 127
15 + ],
16 + [
17 + "10-Q",
18 + 152261,
19 + 152004,
20 + 257
21 + ],
22 + [
23 + "8-K",
24 + 74594,
25 + 74566,
26 + 28
27 + ]
28 + ],
29 + "duplicate_key_sections_in_stored": 0
30 +}
\ No newline at end of file
added results/ledger/flush_zero_groups_after_0622.csv +2 −0
@@ -0,0 +1,2 @@
1 +bin1,rows,zeros,zero_pct,calls,zero_pct_among_calls
2 +2026-10-28 06:40:00,5198,5173,99.52,2954,99.15
added results/ledger/flush_zero_groups_before_0110.csv +1 −0
@@ -0,0 +1 @@
1 +bin1,rows,zeros,zero_pct,calls,zero_pct_among_calls
added results/ledger/gate_pass_by_form.csv +4 −0
@@ -0,0 +1,4 @@
1 +form_type,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2 +10-K,143704,0,135310,135310,8394,0,30877,81622,81495,2464,21.49,22.82,0.568,0.6032
3 +10-Q,578793,1069,363985,363973,214808,0,75705,152261,152004,2494,13.08,20.8,0.2631,0.4183
4 +8-K,400601,1,204442,204442,196159,0,61648,74594,74566,2499,15.39,30.15,0.1862,0.3649
added results/ledger/gate_pass_by_form_item.csv +29 −0
@@ -0,0 +1,29 @@
1 +form_type,item,run,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2 +10-K,business,backfilled_june,5545,0,5542,5542,3,0,2185,5934,5934,460,39.4,39.43,1.0702,1.0707
3 +10-K,business,october,23740,0,23576,23576,164,0,8383,24558,24521,2095,35.31,35.56,1.0345,1.0417
4 +10-K,full_body,backfilled_june,1230,0,1229,1229,1,0,316,824,824,230,25.69,25.71,0.6699,0.6705
5 +10-K,md_and_a,backfilled_june,5198,0,5096,5096,102,0,2505,6703,6703,459,48.19,49.16,1.2895,1.3153
6 +10-K,md_and_a,october,23721,0,22954,22954,767,0,11779,32396,32318,2074,49.66,51.32,1.3657,1.4113
7 +10-K,properties,backfilled_june,5084,0,4897,4897,187,0,870,1721,1721,444,17.11,17.77,0.3385,0.3514
8 +10-K,properties,october,23152,0,21043,21043,2109,0,3013,6158,6148,2035,13.01,14.32,0.266,0.2926
9 +10-K,quantitative_disclosures,backfilled_june,4502,0,3992,3992,510,0,71,110,110,405,1.58,1.78,0.0244,0.0276
10 +10-K,quantitative_disclosures,october,22374,0,18040,18040,4334,0,284,421,420,2041,1.27,1.57,0.0188,0.0233
11 +10-K,risk_factors,backfilled_june,5605,0,5580,5580,25,0,257,495,495,469,4.59,4.61,0.0883,0.0887
12 +10-K,risk_factors,october,23553,0,23361,23361,192,0,1214,2302,2301,2079,5.15,5.2,0.0977,0.0985
13 +10-Q,controls_and_procedures,backfilled_june,3567,1,797,797,2770,0,280,338,338,82,7.85,35.13,0.0948,0.4241
14 +10-Q,controls_and_procedures,october,88826,0,20116,20116,68710,0,5323,6744,6741,2478,5.99,26.46,0.0759,0.3353
15 +10-Q,financial_statements,backfilled_june,3594,0,3507,3507,87,0,743,1183,1183,82,20.67,21.19,0.3292,0.3373
16 +10-Q,financial_statements,october,88288,0,86599,86599,1689,0,18266,28976,28897,2462,20.69,21.09,0.3282,0.3346
17 +10-Q,full_document,october,241,0,241,241,0,0,38,107,107,12,15.77,15.77,0.444,0.444
18 +10-Q,legal_proceedings,backfilled_june,3024,405,956,944,2068,0,18,20,20,82,0.6,1.91,0.0066,0.0212
19 +10-Q,legal_proceedings,october,63974,0,19087,19087,44887,0,623,825,825,2251,0.97,3.26,0.0129,0.0432
20 +10-Q,md_and_a,backfilled_june,3606,4,3599,3599,7,0,1563,3791,3791,83,43.34,43.43,1.0513,1.0533
21 +10-Q,md_and_a,october,87765,0,86670,86670,1095,0,38224,95514,95376,2465,43.55,44.1,1.0883,1.102
22 +10-Q,quantitative_disclosures,backfilled_june,3394,234,1662,1662,1732,0,37,51,51,81,1.09,2.23,0.015,0.0307
23 +10-Q,quantitative_disclosures,october,79744,0,45058,45058,34686,0,553,757,749,2394,0.69,1.23,0.0095,0.0168
24 +10-Q,risk_factors,backfilled_june,3344,425,1200,1200,2144,0,106,150,150,83,3.17,8.83,0.0449,0.125
25 +10-Q,risk_factors,october,69216,0,32626,32626,36590,0,2891,4270,4270,2408,4.18,8.86,0.0617,0.1309
26 +10-Q,unregistered_sales,backfilled_june,3344,0,2881,2881,463,0,222,284,284,83,6.64,7.71,0.0849,0.0986
27 +10-Q,unregistered_sales,october,76866,0,58986,58986,17880,0,6818,9251,9222,2444,8.87,11.56,0.1204,0.1568
28 +8-K,full_body,backfilled_june,103927,1,54360,54360,49567,0,15234,18326,18326,497,14.66,28.02,0.1763,0.3371
29 +8-K,full_body,october,296674,0,150082,150082,146592,0,46414,56268,56240,2499,15.64,30.93,0.1897,0.3749
added results/ledger/gate_pass_by_form_run.csv +7 −0
@@ -0,0 +1,7 @@
1 +form_type,run,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2 +10-K,backfilled_june,27164,0,26336,26336,828,0,6204,15787,15787,497,22.84,23.56,0.5812,0.5994
3 +10-K,october,116540,0,108974,108974,7566,0,24673,65835,65708,2101,21.17,22.64,0.5649,0.6041
4 +10-Q,backfilled_june,23873,1069,14602,14590,9271,0,2969,5817,5817,84,12.44,20.35,0.2437,0.3987
5 +10-Q,october,554920,0,349383,349383,205537,0,72736,146444,146187,2491,13.11,20.82,0.2639,0.4192
6 +8-K,backfilled_june,103927,1,54360,54360,49567,0,15234,18326,18326,497,14.66,28.02,0.1763,0.3371
7 +8-K,october,296674,0,150082,150082,146592,0,46414,56268,56240,2499,15.64,30.93,0.1897,0.3749
added results/ledger/gate_pass_by_run.csv +3 −0
@@ -0,0 +1,3 @@
1 +run,ledger_rows,lt30_words,gate_pass,calls,gate_rejected_zero,gate_fail_but_mentions,with_mentions,ledger_mentions,stored_mentions,issuers,productive_pct_ledger,productive_pct_calls,mentions_per_ledger_row,mentions_per_call
2 +backfilled_june,154964,1070,95298,95286,59666,0,24407,39930,39930,500,15.75,25.61,0.2577,0.4191
3 +october,968134,0,608439,608439,359695,0,143823,268547,268135,2499,14.86,23.64,0.2774,0.4414
added results/ledger/gate_pass_totals.json +27 −0
@@ -0,0 +1,27 @@
1 +{
2 + "ledger_rows": 1123098,
3 + "lt30_words": 1070,
4 + "gate_pass": 703737,
5 + "calls": 703725,
6 + "gate_rejected_zero": 419361,
7 + "gate_fail_but_mentions": 0,
8 + "with_mentions": 168230,
9 + "ledger_mentions": 308477,
10 + "stored_mentions": 308065,
11 + "issuers": 2499,
12 + "productive_pct_ledger": 14.98,
13 + "productive_pct_calls": 23.91,
14 + "mentions_per_ledger_row": 0.2747,
15 + "mentions_per_call": 0.4383,
16 + "usd_per_call_paper_total_775": 0.001274,
17 + "usd_per_october_row_paper_total_775": 0.000801,
18 + "usd_per_call_from_0_0008_x_ledger": 0.001277,
19 + "october_rows": 968134,
20 + "october_calls": 608439,
21 + "headline_before": {
22 + "sections_submitted": 1123098,
23 + "productive_pct": 15.0,
24 + "mentions_per_section": 0.27,
25 + "usd_per_section": 0.0008
26 + }
27 +}
\ No newline at end of file
added results/ledger/ledger_by_5min.csv +103 −0
@@ -0,0 +1,103 @@
1 +bin5,rows,with_mentions,mentions,zero_pct,calls,flushes
2 +2026-10-27 21:35:00,4350,609,1000,86.0,2908,4350
3 +2026-10-27 21:40:00,14971,2246,4123,85.0,9278,14971
4 +2026-10-27 21:45:00,16678,2317,4178,86.11,10174,16678
5 +2026-10-27 21:50:00,12278,1643,2951,86.62,7438,12278
6 +2026-10-27 21:55:00,8128,1156,2092,85.78,4731,8128
7 +2026-10-27 22:00:00,7568,1251,2436,83.47,4736,7568
8 +2026-10-27 22:05:00,8226,1321,2794,83.94,5159,8226
9 +2026-10-27 22:10:00,12652,1770,3371,86.01,7682,12652
10 +2026-10-27 22:15:00,6291,1012,1741,83.91,3959,6291
11 +2026-10-27 22:20:00,13126,1895,3228,85.56,8137,13126
12 +2026-10-27 22:25:00,15316,2276,4063,85.14,9166,15316
13 +2026-10-27 22:30:00,12444,1786,3301,85.65,7539,12444
14 +2026-10-27 22:35:00,16234,2561,4495,84.22,9837,16234
15 +2026-10-27 22:40:00,8706,1277,2272,85.33,5610,8706
16 +2026-10-27 22:45:00,8695,1095,1908,87.41,5337,8695
17 +2026-10-27 22:50:00,7557,1051,1827,86.09,4643,7557
18 +2026-10-27 22:55:00,9490,1209,2076,87.26,5474,9490
19 +2026-10-27 23:00:00,10337,1296,2126,87.46,5849,10337
20 +2026-10-27 23:05:00,8797,1166,2072,86.75,5161,8797
21 +2026-10-27 23:10:00,11704,1724,2917,85.27,6758,11704
22 +2026-10-27 23:15:00,9736,1309,2468,86.56,5873,9736
23 +2026-10-27 23:20:00,5649,889,1499,84.26,3657,5649
24 +2026-10-27 23:25:00,9272,1466,2778,84.19,5531,9272
25 +2026-10-27 23:30:00,6374,746,1503,88.3,3769,6374
26 +2026-10-27 23:35:00,9361,1115,2027,88.09,5877,9361
27 +2026-10-27 23:40:00,14467,2186,3883,84.89,9223,14467
28 +2026-10-27 23:45:00,9476,1277,2326,86.52,5957,9476
29 +2026-10-27 23:50:00,10573,1330,2314,87.42,6185,10573
30 +2026-10-27 23:55:00,10762,1395,2621,87.04,7042,10762
31 +2026-10-28 00:00:00,8266,1260,2608,84.76,5571,8266
32 +2026-10-28 00:05:00,10799,1580,3074,85.37,7176,10799
33 +2026-10-28 00:10:00,8422,1348,2507,83.99,5542,8422
34 +2026-10-28 00:15:00,10262,1461,2786,85.76,6392,10262
35 +2026-10-28 00:20:00,8922,1232,2647,86.19,5590,8922
36 +2026-10-28 00:25:00,5251,897,1754,82.92,3301,5251
37 +2026-10-28 00:30:00,5786,796,1500,86.24,3666,5786
38 +2026-10-28 00:35:00,7134,1000,1923,85.98,4397,7134
39 +2026-10-28 00:40:00,12516,1940,3607,84.5,8081,12516
40 +2026-10-28 00:45:00,7679,1248,2318,83.75,4851,7679
41 +2026-10-28 00:50:00,11588,1443,2612,87.55,7517,11588
42 +2026-10-28 00:55:00,11258,1579,2985,85.97,7198,11258
43 +2026-10-28 01:00:00,8677,1420,2841,83.63,5338,8677
44 +2026-10-28 01:05:00,7428,1732,3209,76.68,4781,7428
45 +2026-10-28 01:10:00,253,253,500,0.0,253,253
46 +2026-10-28 01:20:00,245,245,500,0.0,245,245
47 +2026-10-28 01:25:00,6,6,11,0.0,6,6
48 +2026-10-28 06:40:00,10458,488,1034,95.33,5998,10458
49 +2026-10-28 06:45:00,11183,1488,2933,86.69,6736,11183
50 +2026-10-28 06:50:00,4578,563,1248,87.7,2767,4578
51 +2026-10-28 06:55:00,8928,970,1877,89.14,5656,8928
52 +2026-10-28 07:00:00,13065,1846,3544,85.87,8199,13065
53 +2026-10-28 07:05:00,5495,737,1257,86.59,3531,5495
54 +2026-10-28 07:10:00,13148,1932,3884,85.31,8222,13148
55 +2026-10-28 07:15:00,14377,2207,4336,84.65,9039,14377
56 +2026-10-28 07:20:00,14323,2356,4280,83.55,9481,14323
57 +2026-10-28 07:25:00,15262,2242,4034,85.31,9867,15262
58 +2026-10-28 07:30:00,13643,1950,3661,85.71,8627,13643
59 +2026-10-28 07:35:00,11317,1918,3845,83.05,6959,11317
60 +2026-10-28 07:40:00,12049,2131,4081,82.31,7761,12049
61 +2026-10-28 07:45:00,14266,1947,3575,86.35,8570,14266
62 +2026-10-28 07:50:00,13271,1874,3360,85.88,8687,13271
63 +2026-10-28 07:55:00,8061,1150,2030,85.73,5247,8061
64 +2026-10-28 08:00:00,8671,1273,2519,85.32,5375,8671
65 +2026-10-28 08:05:00,6292,971,1807,84.57,4000,6292
66 +2026-10-28 08:10:00,13917,2218,4325,84.06,8464,13917
67 +2026-10-28 08:15:00,10201,1720,3341,83.14,6351,10201
68 +2026-10-28 08:20:00,12910,1818,3468,85.92,8165,12910
69 +2026-10-28 08:25:00,12030,1728,3307,85.64,7847,12030
70 +2026-10-28 08:30:00,11092,1597,3062,85.6,7157,11092
71 +2026-10-28 08:35:00,9716,1161,2001,88.05,6025,9716
72 +2026-10-28 08:40:00,8927,1327,2590,85.13,5581,8927
73 +2026-10-28 08:45:00,5340,774,1703,85.51,3383,5340
74 +2026-10-28 08:50:00,8143,1123,2185,86.21,5043,8143
75 +2026-10-28 08:55:00,7749,1124,2002,85.49,4697,7749
76 +2026-10-28 09:00:00,12882,1830,3218,85.79,8257,12882
77 +2026-10-28 09:05:00,11277,1510,2855,86.61,7136,11277
78 +2026-10-28 09:10:00,9890,1466,2742,85.18,6635,9890
79 +2026-10-28 09:15:00,8843,1304,2341,85.25,5743,8843
80 +2026-10-28 09:20:00,13311,2171,4295,83.69,8526,13311
81 +2026-10-28 09:25:00,10741,1668,3076,84.47,7093,10741
82 +2026-10-28 09:30:00,13081,1893,3691,85.53,8399,13081
83 +2026-10-28 09:35:00,8869,1402,2517,84.19,6010,8869
84 +2026-10-28 09:40:00,10958,1661,3082,84.84,7112,10958
85 +2026-10-28 09:45:00,7764,1113,2146,85.66,5017,7764
86 +2026-10-28 09:50:00,12713,1842,3387,85.51,8209,12713
87 +2026-10-28 09:55:00,10382,1610,3116,84.49,6761,10382
88 +2026-10-28 10:00:00,8939,1599,3090,82.11,5860,8939
89 +2026-10-28 10:05:00,9496,1498,2900,84.22,5904,9496
90 +2026-10-28 10:10:00,5551,942,2028,83.03,3609,5551
91 +2026-10-28 10:15:00,6155,1042,1939,83.07,3883,6155
92 +2026-10-28 10:20:00,10393,1756,3037,83.1,7032,10393
93 +2026-10-28 10:25:00,10744,1686,3266,84.31,6518,10744
94 +2026-10-28 10:30:00,5861,970,1825,83.45,3687,5861
95 +2026-10-28 10:35:00,10877,1896,3671,82.57,6864,10877
96 +2026-10-28 10:40:00,7599,1184,2043,84.42,4755,7599
97 +2026-10-28 10:45:00,7156,1365,2806,80.93,4518,7156
98 +2026-10-28 10:50:00,4957,918,1616,81.48,3274,4957
99 +2026-10-28 10:55:00,5525,905,1696,83.62,3602,5525
100 +2026-10-28 11:00:00,7816,1336,2302,82.91,5008,7816
101 +2026-10-28 11:05:00,5886,1129,1990,80.82,3669,5886
102 +2026-10-28 11:10:00,6677,1369,2298,79.5,4261,6677
103 +2026-10-28 11:15:00,1669,311,513,81.37,1067,1669
added results/ledger/ledger_by_minute.csv +443 −0
@@ -0,0 +1,443 @@
1 +bin1,rows,with_mentions,mentions,zero_pct,calls,zero_pct_among_calls
2 +2026-10-27 21:35:00,7,0,0,100.0,6,100.0
3 +2026-10-27 21:38:00,2144,296,500,86.19,1469,79.85
4 +2026-10-27 21:39:00,2199,313,500,85.77,1433,78.16
5 +2026-10-27 21:40:00,4187,467,855,88.85,2617,82.16
6 +2026-10-27 21:41:00,1657,303,548,81.71,1026,70.47
7 +2026-10-27 21:42:00,2874,536,1003,81.35,1800,70.22
8 +2026-10-27 21:43:00,3524,527,1001,85.05,2171,75.73
9 +2026-10-27 21:44:00,2729,413,716,84.87,1664,75.18
10 +2026-10-27 21:45:00,5114,616,1003,87.95,3133,80.34
11 +2026-10-27 21:46:00,2462,363,613,85.26,1514,76.02
12 +2026-10-27 21:47:00,3715,488,1001,86.86,2245,78.26
13 +2026-10-27 21:48:00,3411,540,1002,84.17,2115,74.47
14 +2026-10-27 21:49:00,1976,310,559,84.31,1167,73.44
15 +2026-10-27 21:50:00,2235,270,501,87.92,1368,80.26
16 +2026-10-27 21:51:00,1943,275,503,85.85,1227,77.59
17 +2026-10-27 21:52:00,4230,582,947,86.24,2727,78.66
18 +2026-10-27 21:53:00,2082,265,500,87.27,1150,76.96
19 +2026-10-27 21:54:00,1788,251,500,85.96,966,74.02
20 +2026-10-27 21:56:00,2270,284,501,87.49,1322,78.52
21 +2026-10-27 21:57:00,2641,286,502,89.17,1469,80.53
22 +2026-10-27 21:58:00,203,55,86,72.91,132,58.33
23 +2026-10-27 21:59:00,3014,531,1003,82.38,1808,70.63
24 +2026-10-27 22:00:00,1548,229,503,85.21,964,76.24
25 +2026-10-27 22:01:00,1763,268,500,84.8,1134,76.37
26 +2026-10-27 22:02:00,1455,271,500,81.37,895,69.72
27 +2026-10-27 22:03:00,1174,224,433,80.92,724,69.06
28 +2026-10-27 22:04:00,1628,259,500,84.09,1019,74.58
29 +2026-10-27 22:05:00,1166,231,501,80.19,733,68.49
30 +2026-10-27 22:06:00,1384,232,500,83.24,885,73.79
31 +2026-10-27 22:07:00,1567,222,500,85.83,985,77.46
32 +2026-10-27 22:08:00,2554,408,792,84.03,1623,74.86
33 +2026-10-27 22:09:00,1555,228,501,85.34,933,75.56
34 +2026-10-27 22:10:00,1546,240,500,84.48,919,73.88
35 +2026-10-27 22:11:00,1690,253,500,85.03,965,73.78
36 +2026-10-27 22:12:00,3797,459,870,87.91,2252,79.62
37 +2026-10-27 22:13:00,3832,551,1001,85.62,2435,77.37
38 +2026-10-27 22:14:00,1787,267,500,85.06,1111,75.97
39 +2026-10-27 22:15:00,2052,288,504,85.96,1281,77.52
40 +2026-10-27 22:17:00,2692,459,737,82.95,1732,73.5
41 +2026-10-27 22:19:00,1547,265,500,82.87,946,71.99
42 +2026-10-27 22:20:00,1512,256,500,83.07,957,73.25
43 +2026-10-27 22:21:00,2044,313,500,84.69,1329,76.45
44 +2026-10-27 22:22:00,2734,437,727,84.02,1672,73.86
45 +2026-10-27 22:23:00,4515,590,1001,86.93,2684,78.02
46 +2026-10-27 22:24:00,2321,299,500,87.12,1495,80.0
47 +2026-10-27 22:25:00,4612,643,1084,86.06,2762,76.72
48 +2026-10-27 22:26:00,1702,281,500,83.49,933,69.88
49 +2026-10-27 22:27:00,3514,539,1005,84.66,1998,73.02
50 +2026-10-27 22:28:00,1495,250,473,83.28,852,70.66
51 +2026-10-27 22:29:00,3993,563,1001,85.9,2621,78.52
52 +2026-10-27 22:30:00,2134,272,501,87.25,1362,80.03
53 +2026-10-27 22:31:00,1472,210,404,85.73,931,77.44
54 +2026-10-27 22:32:00,1929,285,501,85.23,1073,73.44
55 +2026-10-27 22:33:00,3466,509,956,85.31,2067,75.37
56 +2026-10-27 22:34:00,3443,510,939,85.19,2106,75.78
57 +2026-10-27 22:35:00,1994,340,624,82.95,1134,70.02
58 +2026-10-27 22:36:00,3233,591,1005,81.72,1979,70.14
59 +2026-10-27 22:37:00,3466,503,862,85.49,2116,76.23
60 +2026-10-27 22:38:00,3909,558,1003,85.73,2374,76.5
61 +2026-10-27 22:39:00,3632,569,1001,84.33,2234,74.53
62 +2026-10-27 22:40:00,692,137,268,80.2,431,68.21
63 +2026-10-27 22:41:00,3870,573,1001,85.19,2495,77.03
64 +2026-10-27 22:43:00,1884,273,503,85.51,1191,77.08
65 +2026-10-27 22:44:00,2260,294,500,86.99,1493,80.31
66 +2026-10-27 22:45:00,1691,249,406,85.27,1107,77.51
67 +2026-10-27 22:47:00,2688,270,500,89.96,1553,82.61
68 +2026-10-27 22:48:00,2309,298,501,87.09,1369,78.23
69 +2026-10-27 22:49:00,2007,278,501,86.15,1308,78.75
70 +2026-10-27 22:50:00,2564,334,500,86.97,1648,79.73
71 +2026-10-27 22:51:00,1233,200,327,83.78,812,75.37
72 +2026-10-27 22:52:00,1985,274,500,86.2,1151,76.19
73 +2026-10-27 22:53:00,1775,243,500,86.31,1032,76.45
74 +2026-10-27 22:55:00,1832,258,500,85.92,1039,75.17
75 +2026-10-27 22:56:00,2164,289,500,86.65,1290,77.6
76 +2026-10-27 22:57:00,2808,346,575,87.68,1690,79.53
77 +2026-10-27 22:59:00,2686,316,501,88.24,1455,78.28
78 +2026-10-27 23:01:00,2624,294,504,88.8,1375,78.62
79 +2026-10-27 23:02:00,5339,628,1002,88.24,3058,79.46
80 +2026-10-27 23:03:00,296,72,117,75.68,182,60.44
81 +2026-10-27 23:04:00,2078,302,503,85.47,1234,75.53
82 +2026-10-27 23:05:00,1824,266,500,85.42,1059,74.88
83 +2026-10-27 23:06:00,2241,279,500,87.55,1366,79.58
84 +2026-10-27 23:08:00,2531,345,571,86.37,1489,76.83
85 +2026-10-27 23:09:00,2201,276,501,87.46,1247,77.87
86 +2026-10-27 23:11:00,3501,573,1000,83.63,2069,72.31
87 +2026-10-27 23:12:00,1857,300,500,83.84,1082,72.27
88 +2026-10-27 23:13:00,1901,272,411,85.69,1092,75.09
89 +2026-10-27 23:14:00,4445,579,1006,86.97,2515,76.98
90 +2026-10-27 23:15:00,2407,296,500,87.7,1387,78.66
91 +2026-10-27 23:16:00,2101,271,501,87.1,1190,77.23
92 +2026-10-27 23:17:00,1691,262,462,84.51,983,73.35
93 +2026-10-27 23:18:00,1829,240,505,86.88,1176,79.59
94 +2026-10-27 23:19:00,1708,240,500,85.95,1137,78.89
95 +2026-10-27 23:20:00,1980,271,500,86.31,1276,78.76
96 +2026-10-27 23:22:00,1847,290,502,84.3,1198,75.79
97 +2026-10-27 23:23:00,1764,327,496,81.46,1153,71.64
98 +2026-10-27 23:24:00,58,1,1,98.28,30,96.67
99 +2026-10-27 23:25:00,3107,552,999,82.23,1831,69.85
100 +2026-10-27 23:26:00,1718,278,500,83.82,1011,72.5
101 +2026-10-27 23:27:00,1887,241,501,87.23,1151,79.06
102 +2026-10-27 23:28:00,1800,259,503,85.61,1064,75.66
103 +2026-10-27 23:29:00,760,136,275,82.11,474,71.31
104 +2026-10-27 23:30:00,2193,250,500,88.6,1279,80.45
105 +2026-10-27 23:32:00,1885,237,500,87.43,1088,78.22
106 +2026-10-27 23:34:00,2296,259,503,88.72,1402,81.53
107 +2026-10-27 23:35:00,2048,284,501,86.13,1312,78.35
108 +2026-10-27 23:37:00,1787,196,383,89.03,1135,82.73
109 +2026-10-27 23:38:00,2690,266,504,90.11,1631,83.69
110 +2026-10-27 23:39:00,2836,369,639,86.99,1799,79.49
111 +2026-10-27 23:40:00,4061,508,866,87.49,2615,80.57
112 +2026-10-27 23:41:00,3135,428,732,86.35,1925,77.77
113 +2026-10-27 23:42:00,2950,552,1003,81.29,1889,70.78
114 +2026-10-27 23:43:00,2594,411,768,84.16,1653,75.14
115 +2026-10-27 23:44:00,1727,287,514,83.38,1141,74.85
116 +2026-10-27 23:45:00,2554,293,502,88.53,1584,81.5
117 +2026-10-27 23:46:00,1996,263,500,86.82,1271,79.31
118 +2026-10-27 23:47:00,2021,282,500,86.05,1326,78.73
119 +2026-10-27 23:48:00,1754,265,500,84.89,1071,75.26
120 +2026-10-27 23:49:00,1151,174,324,84.88,705,75.32
121 +2026-10-27 23:50:00,2587,298,500,88.48,1424,79.07
122 +2026-10-27 23:51:00,2283,300,500,86.86,1312,77.13
123 +2026-10-27 23:53:00,3398,442,786,86.99,2092,78.87
124 +2026-10-27 23:54:00,2305,290,528,87.42,1357,78.63
125 +2026-10-27 23:55:00,2597,273,500,89.49,1674,83.69
126 +2026-10-27 23:56:00,1873,285,502,84.78,1301,78.09
127 +2026-10-27 23:57:00,2268,266,501,88.27,1505,82.33
128 +2026-10-27 23:58:00,2397,337,613,85.94,1520,77.83
129 +2026-10-27 23:59:00,1627,234,505,85.62,1042,77.54
130 +2026-10-28 00:00:00,1258,218,501,82.67,837,73.95
131 +2026-10-28 00:01:00,1674,239,500,85.72,1111,78.49
132 +2026-10-28 00:02:00,1426,207,500,85.48,945,78.1
133 +2026-10-28 00:03:00,1762,254,501,85.58,1222,79.21
134 +2026-10-28 00:04:00,2146,342,606,84.06,1456,76.51
135 +2026-10-28 00:05:00,1952,270,502,86.17,1337,79.81
136 +2026-10-28 00:06:00,3863,512,1001,86.75,2632,80.55
137 +2026-10-28 00:07:00,1556,244,501,84.32,1005,75.72
138 +2026-10-28 00:08:00,1772,315,570,82.22,1183,73.37
139 +2026-10-28 00:09:00,1656,239,500,85.57,1019,76.55
140 +2026-10-28 00:10:00,1637,258,500,84.24,1039,75.17
141 +2026-10-28 00:11:00,1596,247,500,84.52,1012,75.59
142 +2026-10-28 00:12:00,1449,275,501,81.02,984,72.05
143 +2026-10-28 00:13:00,2033,303,504,85.1,1401,78.37
144 +2026-10-28 00:14:00,1707,265,502,84.48,1106,76.04
145 +2026-10-28 00:15:00,662,99,190,85.05,450,78.0
146 +2026-10-28 00:16:00,2143,278,503,87.03,1287,78.4
147 +2026-10-28 00:17:00,1717,263,502,84.68,1080,75.65
148 +2026-10-28 00:18:00,3511,521,1005,85.16,2204,76.36
149 +2026-10-28 00:19:00,2229,300,586,86.54,1371,78.12
150 +2026-10-28 00:20:00,2036,328,645,83.89,1285,74.47
151 +2026-10-28 00:21:00,1534,225,501,85.33,957,76.49
152 +2026-10-28 00:22:00,1546,205,501,86.74,970,78.87
153 +2026-10-28 00:23:00,1807,231,500,87.22,1097,78.94
154 +2026-10-28 00:24:00,1999,243,500,87.84,1281,81.03
155 +2026-10-28 00:26:00,1717,274,502,84.04,1094,74.95
156 +2026-10-28 00:27:00,942,143,250,84.82,615,76.75
157 +2026-10-28 00:28:00,1325,247,501,81.36,807,69.39
158 +2026-10-28 00:29:00,1267,233,501,81.61,785,70.32
159 +2026-10-28 00:30:00,1502,244,500,83.75,941,74.07
160 +2026-10-28 00:32:00,1955,279,500,85.73,1239,77.48
161 +2026-10-28 00:34:00,2329,273,500,88.28,1486,81.63
162 +2026-10-28 00:35:00,1570,202,419,87.13,1012,80.04
163 +2026-10-28 00:36:00,1802,239,500,86.74,1038,76.97
164 +2026-10-28 00:37:00,1810,280,503,84.53,1109,74.75
165 +2026-10-28 00:39:00,1952,279,501,85.71,1238,77.46
166 +2026-10-28 00:40:00,3525,532,1000,84.91,2377,77.62
167 +2026-10-28 00:42:00,2133,356,605,83.31,1437,75.23
168 +2026-10-28 00:43:00,3359,523,1001,84.43,2021,74.12
169 +2026-10-28 00:44:00,3499,529,1001,84.88,2246,76.45
170 +2026-10-28 00:45:00,1822,283,500,84.47,1166,75.73
171 +2026-10-28 00:46:00,1084,172,317,84.13,692,75.14
172 +2026-10-28 00:47:00,1708,270,500,84.19,1054,74.38
173 +2026-10-28 00:48:00,1627,265,500,83.71,1005,73.63
174 +2026-10-28 00:49:00,1438,258,501,82.06,934,72.38
175 +2026-10-28 00:51:00,4632,538,1000,88.39,2975,81.92
176 +2026-10-28 00:52:00,387,50,111,87.08,249,79.92
177 +2026-10-28 00:53:00,2511,299,500,88.09,1569,80.94
178 +2026-10-28 00:54:00,4058,556,1001,86.3,2724,79.59
179 +2026-10-28 00:56:00,2179,300,500,86.23,1456,79.4
180 +2026-10-28 00:57:00,2489,307,484,87.67,1633,81.2
181 +2026-10-28 00:58:00,3000,497,1000,83.43,1825,72.77
182 +2026-10-28 00:59:00,3590,475,1001,86.77,2284,79.2
183 +2026-10-28 01:00:00,2360,373,711,84.19,1457,74.4
184 +2026-10-28 01:01:00,1747,263,500,84.95,1083,75.72
185 +2026-10-28 01:02:00,1106,239,501,78.39,676,64.64
186 +2026-10-28 01:03:00,3096,480,1002,84.5,1890,74.6
187 +2026-10-28 01:04:00,368,65,127,82.34,232,71.98
188 +2026-10-28 01:05:00,2294,346,767,84.92,1393,75.16
189 +2026-10-28 01:07:00,2155,287,500,86.68,1237,76.8
190 +2026-10-28 01:08:00,2452,572,1000,76.67,1624,64.78
191 +2026-10-28 01:09:00,527,527,942,0.0,527,0.0
192 +2026-10-28 01:13:00,240,240,465,0.0,240,0.0
193 +2026-10-28 01:14:00,13,13,35,0.0,13,0.0
194 +2026-10-28 01:20:00,80,80,173,0.0,80,0.0
195 +2026-10-28 01:21:00,165,165,327,0.0,165,0.0
196 +2026-10-28 01:25:00,6,6,11,0.0,6,0.0
197 +2026-10-28 06:40:00,5198,25,34,99.52,2954,99.15
198 +2026-10-28 06:43:00,5260,463,1000,91.2,3044,84.79
199 +2026-10-28 06:45:00,2630,258,503,90.19,1591,83.78
200 +2026-10-28 06:46:00,3503,484,928,86.18,2122,77.19
201 +2026-10-28 06:47:00,1631,257,501,84.24,977,73.69
202 +2026-10-28 06:48:00,1789,256,500,85.69,1079,76.27
203 +2026-10-28 06:49:00,1630,233,501,85.71,967,75.9
204 +2026-10-28 06:51:00,2863,346,747,87.91,1700,79.65
205 +2026-10-28 06:53:00,1715,217,501,87.35,1067,79.66
206 +2026-10-28 06:55:00,2316,242,500,89.55,1443,83.23
207 +2026-10-28 06:57:00,2681,265,500,90.12,1696,84.38
208 +2026-10-28 06:58:00,2014,209,376,89.62,1244,83.2
209 +2026-10-28 06:59:00,1917,254,501,86.75,1273,80.05
210 +2026-10-28 07:00:00,1615,225,502,86.07,1031,78.18
211 +2026-10-28 07:01:00,3110,456,1001,85.34,1990,77.09
212 +2026-10-28 07:02:00,3432,479,796,86.04,2208,78.31
213 +2026-10-28 07:03:00,619,146,243,76.41,431,66.13
214 +2026-10-28 07:04:00,4289,540,1002,87.41,2539,78.73
215 +2026-10-28 07:05:00,2091,277,503,86.75,1344,79.39
216 +2026-10-28 07:07:00,3404,460,754,86.49,2187,78.97
217 +2026-10-28 07:10:00,2690,272,500,89.89,1744,84.4
218 +2026-10-28 07:11:00,2403,301,502,87.47,1525,80.26
219 +2026-10-28 07:12:00,2931,442,1006,84.92,1828,75.82
220 +2026-10-28 07:13:00,2236,401,876,82.07,1428,71.92
221 +2026-10-28 07:14:00,2888,516,1000,82.13,1697,69.59
222 +2026-10-28 07:15:00,3392,512,1001,84.91,2002,74.43
223 +2026-10-28 07:16:00,2801,428,826,84.72,1594,73.15
224 +2026-10-28 07:17:00,4376,753,1504,82.79,2957,74.54
225 +2026-10-28 07:18:00,1822,258,502,85.84,1187,78.26
226 +2026-10-28 07:19:00,1986,256,503,87.11,1299,80.29
227 +2026-10-28 07:20:00,2581,372,650,85.59,1742,78.65
228 +2026-10-28 07:21:00,1746,279,502,84.02,1168,76.11
229 +2026-10-28 07:22:00,2813,506,1000,82.01,1923,73.69
230 +2026-10-28 07:23:00,3808,615,1128,83.85,2457,74.97
231 +2026-10-28 07:24:00,3375,584,1000,82.7,2191,73.35
232 +2026-10-28 07:25:00,3553,572,1000,83.9,2429,76.45
233 +2026-10-28 07:26:00,3975,572,1031,85.61,2614,78.12
234 +2026-10-28 07:27:00,1857,277,502,85.08,1164,76.2
235 +2026-10-28 07:28:00,4056,554,1001,86.34,2569,78.44
236 +2026-10-28 07:29:00,1821,267,500,85.34,1091,75.53
237 +2026-10-28 07:30:00,1308,199,384,84.79,790,74.81
238 +2026-10-28 07:31:00,2409,289,501,88.0,1573,81.63
239 +2026-10-28 07:32:00,3901,551,1000,85.88,2522,78.15
240 +2026-10-28 07:33:00,2737,420,769,84.65,1739,75.85
241 +2026-10-28 07:34:00,3288,491,1007,85.07,2003,75.49
242 +2026-10-28 07:35:00,3134,474,1001,84.88,1914,75.24
243 +2026-10-28 07:36:00,2250,439,842,80.49,1426,69.21
244 +2026-10-28 07:37:00,2909,505,1000,82.64,1775,71.55
245 +2026-10-28 07:38:00,1315,247,502,81.22,823,69.99
246 +2026-10-28 07:39:00,1709,253,500,85.2,1021,75.22
247 +2026-10-28 07:40:00,1416,285,501,79.87,911,68.72
248 +2026-10-28 07:41:00,1431,265,501,81.48,853,68.93
249 +2026-10-28 07:42:00,3341,572,1072,82.88,2159,73.51
250 +2026-10-28 07:43:00,3121,513,1001,83.56,2038,74.83
251 +2026-10-28 07:44:00,2740,496,1006,81.9,1800,72.44
252 +2026-10-28 07:45:00,1526,243,470,84.08,968,74.9
253 +2026-10-28 07:46:00,2450,308,500,87.43,1435,78.54
254 +2026-10-28 07:47:00,3348,531,1005,84.14,1996,73.4
255 +2026-10-28 07:48:00,4290,559,1098,86.97,2494,77.59
256 +2026-10-28 07:49:00,2652,306,502,88.46,1677,81.75
257 +2026-10-28 07:50:00,3861,575,1002,85.11,2574,77.66
258 +2026-10-28 07:51:00,2237,275,500,87.71,1405,80.43
259 +2026-10-28 07:52:00,3627,476,855,86.88,2354,79.78
260 +2026-10-28 07:53:00,1536,265,503,82.75,1005,73.63
261 +2026-10-28 07:54:00,2010,283,500,85.92,1349,79.02
262 +2026-10-28 07:55:00,1810,266,500,85.3,1181,77.48
263 +2026-10-28 07:56:00,1550,281,530,81.87,1025,72.59
264 +2026-10-28 07:58:00,2512,307,500,87.78,1625,81.11
265 +2026-10-28 07:59:00,2189,296,500,86.48,1416,79.1
266 +2026-10-28 08:00:00,1771,253,501,85.71,1140,77.81
267 +2026-10-28 08:01:00,1738,254,500,85.39,1071,76.28
268 +2026-10-28 08:02:00,3743,536,1018,85.68,2301,76.71
269 +2026-10-28 08:03:00,1419,230,500,83.79,863,73.35
270 +2026-10-28 08:05:00,1648,268,500,83.74,1022,73.78
271 +2026-10-28 08:06:00,1600,249,500,84.44,1042,76.1
272 +2026-10-28 08:07:00,1811,267,500,85.26,1150,76.78
273 +2026-10-28 08:08:00,1233,187,307,84.83,786,76.21
274 +2026-10-28 08:10:00,1685,265,500,84.27,976,72.85
275 +2026-10-28 08:11:00,1761,260,502,85.24,1053,75.31
276 +2026-10-28 08:12:00,4195,642,1314,84.7,2587,75.18
277 +2026-10-28 08:13:00,3205,542,1006,83.09,1951,72.22
278 +2026-10-28 08:14:00,3071,509,1003,83.43,1897,73.17
279 +2026-10-28 08:15:00,2920,517,1000,82.29,1790,71.12
280 +2026-10-28 08:16:00,2174,430,837,80.22,1377,68.77
281 +2026-10-28 08:17:00,1620,258,503,84.07,980,73.67
282 +2026-10-28 08:18:00,1573,239,500,84.81,996,76.0
283 +2026-10-28 08:19:00,1914,276,501,85.58,1208,77.15
284 +2026-10-28 08:20:00,3582,559,1000,84.39,2243,75.08
285 +2026-10-28 08:21:00,3016,347,671,88.49,1842,81.16
286 +2026-10-28 08:22:00,3039,431,794,85.82,1938,77.76
287 +2026-10-28 08:23:00,1630,239,500,85.34,1090,78.07
288 +2026-10-28 08:24:00,1643,242,503,85.27,1052,77.0
289 +2026-10-28 08:25:00,1370,236,491,82.77,919,74.32
290 +2026-10-28 08:26:00,1991,276,500,86.14,1278,78.4
291 +2026-10-28 08:27:00,2027,267,500,86.83,1233,78.35
292 +2026-10-28 08:28:00,3413,492,1003,85.58,2247,78.1
293 +2026-10-28 08:29:00,3229,457,813,85.85,2170,78.94
294 +2026-10-28 08:30:00,3190,479,1001,84.98,2160,77.82
295 +2026-10-28 08:31:00,3133,522,1059,83.34,2063,74.7
296 +2026-10-28 08:32:00,2288,287,450,87.46,1380,79.2
297 +2026-10-28 08:33:00,350,32,51,90.86,220,85.45
298 +2026-10-28 08:34:00,2131,277,501,87.0,1334,79.24
299 +2026-10-28 08:35:00,1193,129,230,89.19,784,83.55
300 +2026-10-28 08:36:00,1797,157,272,91.26,1099,85.71
301 +2026-10-28 08:37:00,2454,289,499,88.22,1497,80.69
302 +2026-10-28 08:38:00,2091,308,500,85.27,1317,76.61
303 +2026-10-28 08:39:00,2181,278,500,87.25,1328,79.07
304 +2026-10-28 08:40:00,3341,515,1002,84.59,2112,75.62
305 +2026-10-28 08:42:00,1966,306,585,84.44,1220,74.92
306 +2026-10-28 08:43:00,2357,287,503,87.82,1465,80.41
307 +2026-10-28 08:44:00,1263,219,500,82.66,784,72.07
308 +2026-10-28 08:45:00,1265,210,500,83.4,808,74.01
309 +2026-10-28 08:46:00,1671,243,500,85.46,1070,77.29
310 +2026-10-28 08:48:00,2404,321,703,86.65,1505,78.67
311 +2026-10-28 08:50:00,1928,255,501,86.77,1175,78.3
312 +2026-10-28 08:52:00,1552,245,502,84.21,949,74.18
313 +2026-10-28 08:53:00,2176,262,502,87.96,1341,80.46
314 +2026-10-28 08:54:00,2487,361,680,85.48,1578,77.12
315 +2026-10-28 08:55:00,1955,308,500,84.25,1187,74.05
316 +2026-10-28 08:56:00,1764,266,500,84.92,1069,75.12
317 +2026-10-28 08:58:00,2039,298,501,85.38,1268,76.5
318 +2026-10-28 08:59:00,1991,252,501,87.34,1173,78.52
319 +2026-10-28 09:00:00,768,106,243,86.2,489,78.32
320 +2026-10-28 09:01:00,1576,316,500,79.95,1040,69.62
321 +2026-10-28 09:02:00,4391,582,1000,86.75,2836,79.48
322 +2026-10-28 09:03:00,2376,274,474,88.47,1454,81.16
323 +2026-10-28 09:04:00,3771,552,1001,85.36,2438,77.36
324 +2026-10-28 09:05:00,2287,280,500,87.76,1439,80.54
325 +2026-10-28 09:06:00,2025,293,500,85.53,1248,76.52
326 +2026-10-28 09:07:00,1896,216,353,88.61,1144,81.12
327 +2026-10-28 09:08:00,1940,243,502,87.47,1243,80.45
328 +2026-10-28 09:09:00,3129,478,1000,84.72,2062,76.82
329 +2026-10-28 09:10:00,1711,274,554,83.99,1094,74.95
330 +2026-10-28 09:11:00,3366,523,1001,84.46,2238,76.63
331 +2026-10-28 09:12:00,2007,269,502,86.6,1418,81.03
332 +2026-10-28 09:13:00,2806,400,685,85.74,1885,78.78
333 +2026-10-28 09:15:00,1786,273,500,84.71,1094,75.05
334 +2026-10-28 09:16:00,1936,237,501,87.76,1249,81.02
335 +2026-10-28 09:17:00,2398,282,502,88.24,1534,81.62
336 +2026-10-28 09:18:00,2723,512,838,81.2,1866,72.56
337 +2026-10-28 09:20:00,1544,274,500,82.25,965,71.61
338 +2026-10-28 09:21:00,1895,385,738,79.68,1208,68.13
339 +2026-10-28 09:22:00,4158,642,1262,84.56,2599,75.3
340 +2026-10-28 09:23:00,2339,386,785,83.5,1519,74.59
341 +2026-10-28 09:24:00,3375,484,1010,85.66,2235,78.34
342 +2026-10-28 09:25:00,2853,458,816,83.95,1969,76.74
343 +2026-10-28 09:26:00,1640,259,500,84.21,1140,77.28
344 +2026-10-28 09:27:00,3412,514,1003,84.94,2280,77.46
345 +2026-10-28 09:28:00,701,145,257,79.32,470,69.15
346 +2026-10-28 09:29:00,2135,292,500,86.32,1234,76.34
347 +2026-10-28 09:30:00,4042,548,1002,86.44,2534,78.37
348 +2026-10-28 09:31:00,1107,164,246,85.19,758,78.36
349 +2026-10-28 09:32:00,3346,534,1000,84.04,2156,75.23
350 +2026-10-28 09:33:00,3328,464,1000,86.06,2126,78.17
351 +2026-10-28 09:34:00,1258,183,443,85.45,825,77.82
352 +2026-10-28 09:35:00,3230,547,1004,83.07,2154,74.61
353 +2026-10-28 09:36:00,2027,303,500,85.05,1342,77.42
354 +2026-10-28 09:37:00,1949,266,504,86.35,1323,79.89
355 +2026-10-28 09:38:00,1,1,9,0.0,1,0.0
356 +2026-10-28 09:39:00,1662,285,500,82.85,1190,76.05
357 +2026-10-28 09:40:00,1364,274,502,79.91,938,70.79
358 +2026-10-28 09:41:00,1501,270,500,82.01,1051,74.31
359 +2026-10-28 09:42:00,3387,563,1077,83.38,2194,74.34
360 +2026-10-28 09:44:00,4706,554,1003,88.23,2929,81.09
361 +2026-10-28 09:45:00,1667,288,501,82.72,1101,73.84
362 +2026-10-28 09:46:00,481,81,142,83.16,314,74.2
363 +2026-10-28 09:47:00,1859,261,501,85.96,1188,78.03
364 +2026-10-28 09:48:00,2045,242,500,88.17,1300,81.38
365 +2026-10-28 09:49:00,1712,241,502,85.92,1114,78.37
366 +2026-10-28 09:50:00,954,144,288,84.91,640,77.5
367 +2026-10-28 09:51:00,2343,315,500,86.56,1456,78.37
368 +2026-10-28 09:52:00,4092,551,1000,86.53,2524,78.17
369 +2026-10-28 09:53:00,2366,453,796,80.85,1578,71.29
370 +2026-10-28 09:54:00,2958,379,803,87.19,2011,81.15
371 +2026-10-28 09:55:00,2016,253,495,87.45,1325,80.91
372 +2026-10-28 09:56:00,1930,302,500,84.35,1259,76.01
373 +2026-10-28 09:57:00,3550,521,1001,85.32,2255,76.9
374 +2026-10-28 09:58:00,250,54,118,78.4,164,67.07
375 +2026-10-28 09:59:00,2636,480,1002,81.79,1758,72.7
376 +2026-10-28 10:00:00,2778,541,1132,80.53,1783,69.66
377 +2026-10-28 10:01:00,2672,434,757,83.76,1833,76.32
378 +2026-10-28 10:02:00,1443,261,500,81.91,915,71.48
379 +2026-10-28 10:03:00,2046,363,701,82.26,1329,72.69
380 +2026-10-28 10:05:00,2158,263,502,87.81,1300,79.77
381 +2026-10-28 10:06:00,1729,252,500,85.43,1037,75.7
382 +2026-10-28 10:07:00,1366,188,398,86.24,875,78.51
383 +2026-10-28 10:08:00,2960,535,1000,81.93,1859,71.22
384 +2026-10-28 10:09:00,1283,260,500,79.73,833,68.79
385 +2026-10-28 10:10:00,1234,257,524,79.17,821,68.7
386 +2026-10-28 10:12:00,1933,261,500,86.5,1288,79.74
387 +2026-10-28 10:13:00,1230,205,502,83.33,775,73.55
388 +2026-10-28 10:14:00,1154,219,502,81.02,725,69.79
389 +2026-10-28 10:15:00,1282,232,500,81.9,783,70.37
390 +2026-10-28 10:16:00,103,27,54,73.79,81,66.67
391 +2026-10-28 10:17:00,1641,188,330,88.54,1040,81.92
392 +2026-10-28 10:18:00,2170,390,676,82.03,1357,71.26
393 +2026-10-28 10:19:00,959,205,379,78.62,622,67.04
394 +2026-10-28 10:20:00,1900,318,501,83.26,1279,75.14
395 +2026-10-28 10:21:00,1569,274,500,82.54,1035,73.53
396 +2026-10-28 10:22:00,1836,297,508,83.82,1263,76.48
397 +2026-10-28 10:23:00,1781,287,500,83.89,1211,76.3
398 +2026-10-28 10:24:00,3307,580,1028,82.46,2244,74.15
399 +2026-10-28 10:25:00,3271,506,1002,84.53,1960,74.18
400 +2026-10-28 10:26:00,2706,442,823,83.67,1677,73.64
401 +2026-10-28 10:27:00,1873,268,502,85.69,1180,77.29
402 +2026-10-28 10:28:00,2894,470,939,83.76,1701,72.37
403 +2026-10-28 10:30:00,1424,258,500,81.88,944,72.67
404 +2026-10-28 10:31:00,1762,264,500,85.02,1067,75.26
405 +2026-10-28 10:32:00,956,182,324,80.96,597,69.51
406 +2026-10-28 10:33:00,1719,266,501,84.53,1079,75.35
407 +2026-10-28 10:35:00,2108,356,769,83.11,1272,72.01
408 +2026-10-28 10:36:00,1089,218,411,79.98,676,67.75
409 +2026-10-28 10:37:00,2163,314,504,85.48,1381,77.26
410 +2026-10-28 10:38:00,3278,622,1174,81.03,2139,70.92
411 +2026-10-28 10:39:00,2239,386,813,82.76,1396,72.35
412 +2026-10-28 10:40:00,1705,270,500,84.16,1139,76.29
413 +2026-10-28 10:41:00,2168,325,500,85.01,1463,77.79
414 +2026-10-28 10:42:00,108,27,41,75.0,93,70.97
415 +2026-10-28 10:43:00,1788,277,500,84.51,1033,73.18
416 +2026-10-28 10:44:00,1830,285,502,84.43,1027,72.25
417 +2026-10-28 10:45:00,1708,339,654,80.15,1074,68.44
418 +2026-10-28 10:46:00,2240,437,1002,80.49,1389,68.54
419 +2026-10-28 10:47:00,338,65,149,80.77,204,68.14
420 +2026-10-28 10:48:00,1456,277,500,80.98,988,71.96
421 +2026-10-28 10:49:00,1414,247,501,82.53,863,71.38
422 +2026-10-28 10:50:00,1144,253,427,77.88,743,65.95
423 +2026-10-28 10:52:00,1620,296,501,81.73,1080,72.59
424 +2026-10-28 10:53:00,1600,270,500,83.13,1048,74.24
425 +2026-10-28 10:54:00,593,99,188,83.31,403,75.43
426 +2026-10-28 10:56:00,1832,264,500,85.59,1199,77.98
427 +2026-10-28 10:57:00,1956,357,695,81.75,1223,70.81
428 +2026-10-28 10:59:00,1737,284,501,83.65,1180,75.93
429 +2026-10-28 11:00:00,3579,621,1066,82.65,2315,73.17
430 +2026-10-28 11:01:00,26,11,25,57.69,15,26.67
431 +2026-10-28 11:02:00,1567,248,429,84.17,985,74.82
432 +2026-10-28 11:03:00,1817,293,500,83.87,1155,74.63
433 +2026-10-28 11:04:00,827,163,282,80.29,538,69.7
434 +2026-10-28 11:05:00,1560,273,503,82.5,961,71.59
435 +2026-10-28 11:06:00,908,214,377,76.43,560,61.79
436 +2026-10-28 11:07:00,1585,292,501,81.58,990,70.51
437 +2026-10-28 11:08:00,1166,245,423,78.99,747,67.2
438 +2026-10-28 11:09:00,667,105,186,84.26,411,74.45
439 +2026-10-28 11:10:00,1764,463,752,73.75,1183,60.86
440 +2026-10-28 11:12:00,1925,331,550,82.81,1175,71.83
441 +2026-10-28 11:13:00,610,110,199,81.97,383,71.28
442 +2026-10-28 11:14:00,2378,465,797,80.45,1520,69.41
443 +2026-10-28 11:15:00,1669,311,513,81.37,1067,70.85
added results/ledger/ledger_collisions.csv +305 −0
Large diff (305 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/ledger_collisions_by_form.csv +4 −0
Large diff (4 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/log_failures_by_hour.csv +17 −0
Large diff (17 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/log_summary.json +1021 −0
Large diff (1,022 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/manifest.json +49 −0
Large diff (50 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/resume_batch1_vs_expected.csv +14 −0
Large diff (14 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/table3_ledger_vs_stored.csv +4 −0
Large diff (4 lines) is not loaded by default to keep the page fast.Load diff
added results/ledger/throughput.json +105 −0
Large diff (107 lines) is not loaded by default to keep the page fast.Load diff
Menu

Theme