AIR v0.1.0 — Accounting Intermediate Representation
The language of accounting: LLMs produce economic events (AIR), a deterministic compiler (AIC) produces balanced journal entries under versioned, source-cited ALSL policies. All six phases complete: - Phase 0: 9 cited research reports (GST/QST from official sources, LLVM architecture, ledger engines, ERP APIs, camt.053/MT940, ...) - Phase 1: AIR v0.1 schema, provenance graph (accounting SSA), double-entry invariant verified after every pass, 21 golden cases - Phase 2: incremental compilation (content-fingerprint diff, reversal + replacement entries) - Phase 3: QuickBooks backend (100% offline-tested mock transport), managed AIR home (hash-chained ledger, document archive) - Phase 4: LLM ingestion with confidence routing + human approval inbox - Phase 5: agent syscalls + hash-chained audit log, closed periods - Phase 6: fusion/netting/duplicate passes, bank reconciliation (camt.053, MT940 with integrity check, CSV) 99 offline tests + 2 opt-in live tests. No floats, no rules in code, append-only everything. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Showing 98 changed files with +11,076 and −0
added
.github/workflows/ci.yml
+49 −0
@@ -0,0 +1,49 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | |
| 3 | +# Contact : contact@spboucher.ai | |
| 4 | +# CI: offline test suite (no API keys) + author-header gate on every push/PR. | |
| 5 | + | |
| 6 | +name: CI | |
| 7 | + | |
| 8 | +on: | |
| 9 | + push: | |
| 10 | + branches: [main] | |
| 11 | + pull_request: | |
| 12 | + | |
| 13 | +jobs: | |
| 14 | + test: | |
| 15 | + runs-on: ubuntu-latest | |
| 16 | + strategy: | |
| 17 | + fail-fast: false | |
| 18 | + matrix: | |
| 19 | + python-version: ["3.11", "3.12", "3.13"] | |
| 20 | + steps: | |
| 21 | + - uses: actions/checkout@v4 | |
| 22 | + - uses: actions/setup-python@v5 | |
| 23 | + with: | |
| 24 | + python-version: ${{ matrix.python-version }} | |
| 25 | + - name: Install | |
| 26 | + run: pip install -e ".[dev]" | |
| 27 | + - name: Author-header gate | |
| 28 | + run: python scripts/check_headers.py | |
| 29 | + - name: Offline test suite (99 tests — no network, no API keys) | |
| 30 | + run: pytest tests/ -q --ignore=tests/test_llm_live.py | |
| 31 | + | |
| 32 | + golden-determinism: | |
| 33 | + # compile the demo document twice on a fresh interpreter each time: | |
| 34 | + # byte-identical CSV output or the build fails | |
| 35 | + runs-on: ubuntu-latest | |
| 36 | + steps: | |
| 37 | + - uses: actions/checkout@v4 | |
| 38 | + - uses: actions/setup-python@v5 | |
| 39 | + with: | |
| 40 | + python-version: "3.12" | |
| 41 | + - name: Install | |
| 42 | + run: pip install -e . | |
| 43 | + - name: Determinism check | |
| 44 | + run: | | |
| 45 | + python -m sdk.cli compile tests/fixtures/demo_document.yaml \ | |
| 46 | + --policies alsl/policies/ca-qc-2026.yaml --backend csv --out out_a | |
| 47 | + python -m sdk.cli compile tests/fixtures/demo_document.yaml \ | |
| 48 | + --policies alsl/policies/ca-qc-2026.yaml --backend csv --out out_b | |
| 49 | + diff out_a/*.csv out_b/*.csv && echo "byte-identical: OK" | |
added
.gitignore
+17 −0
@@ -0,0 +1,17 @@ | ||
| 1 | +__pycache__/ | |
| 2 | +*.pyc | |
| 3 | +.venv/ | |
| 4 | +.pytest_cache/ | |
| 5 | +.hypothesis/ | |
| 6 | +.mypy_cache/ | |
| 7 | +out/ | |
| 8 | +books/ | |
| 9 | +dist/ | |
| 10 | +build/ | |
| 11 | +*.egg-info/ | |
| 12 | +.DS_Store | |
| 13 | + | |
| 14 | +.env | |
| 15 | +books/ | |
| 16 | +out_a/ | |
| 17 | +out_b/ | |
added
CLAUDE.md
+320 −0
@@ -0,0 +1,320 @@ | ||
| 1 | +# CLAUDE.md — Projet AIR (Accounting Intermediate Representation) | |
| 2 | + | |
| 3 | +> **Auteur du projet : Simon-Pierre Boucher — contact@spboucher.ai** | |
| 4 | +> Ce fichier est la source de vérité pour Claude Code. Lis-le intégralement avant toute action. | |
| 5 | + | |
| 6 | +--- | |
| 7 | + | |
| 8 | +## 0. RÈGLE ABSOLUE — En-tête obligatoire de chaque fichier | |
| 9 | + | |
| 10 | +**CHAQUE fichier créé ou modifié dans ce projet DOIT commencer par un en-tête d'auteur.** | |
| 11 | +Aucune exception : code source, tests, docs, scripts, configs, schémas, notebooks. | |
| 12 | + | |
| 13 | +### Formats selon le type de fichier | |
| 14 | + | |
| 15 | +**Python (.py)** | |
| 16 | +```python | |
| 17 | +# ============================================================================= | |
| 18 | +# Projet : AIR — Accounting Intermediate Representation | |
| 19 | +# Auteur : Simon-Pierre Boucher | |
| 20 | +# Contact : contact@spboucher.ai | |
| 21 | +# Fichier : <nom_du_fichier> | |
| 22 | +# Description : <une ligne décrivant le rôle du fichier> | |
| 23 | +# ============================================================================= | |
| 24 | +``` | |
| 25 | + | |
| 26 | +**TypeScript / JavaScript / Rust / Go / C / Zig (.ts, .js, .rs, .go, .c, .zig)** | |
| 27 | +```ts | |
| 28 | +// ============================================================================= | |
| 29 | +// Projet : AIR — Accounting Intermediate Representation | |
| 30 | +// Auteur : Simon-Pierre Boucher | |
| 31 | +// Contact : contact@spboucher.ai | |
| 32 | +// Fichier : <nom_du_fichier> | |
| 33 | +// Description : <une ligne> | |
| 34 | +// ============================================================================= | |
| 35 | +``` | |
| 36 | + | |
| 37 | +**Markdown (.md)** | |
| 38 | +```markdown | |
| 39 | +<!-- | |
| 40 | +Projet : AIR — Accounting Intermediate Representation | |
| 41 | +Auteur : Simon-Pierre Boucher | |
| 42 | +Contact : contact@spboucher.ai | |
| 43 | +Fichier : <nom_du_fichier> | |
| 44 | +--> | |
| 45 | +``` | |
| 46 | + | |
| 47 | +**YAML / TOML / config (.yaml, .yml, .toml)** | |
| 48 | +```yaml | |
| 49 | +# Projet : AIR — Accounting Intermediate Representation | |
| 50 | +# Auteur : Simon-Pierre Boucher | |
| 51 | +# Contact : contact@spboucher.ai | |
| 52 | +``` | |
| 53 | + | |
| 54 | +**JSON** : le format JSON n'accepte pas les commentaires. Ajouter une clé au niveau racine : | |
| 55 | +```json | |
| 56 | +{ "_author": "Simon-Pierre Boucher <contact@spboucher.ai>", ... } | |
| 57 | +``` | |
| 58 | + | |
| 59 | +**SQL (.sql)** | |
| 60 | +```sql | |
| 61 | +-- ============================================================================= | |
| 62 | +-- Projet : AIR | Auteur : Simon-Pierre Boucher | contact@spboucher.ai | |
| 63 | +-- ============================================================================= | |
| 64 | +``` | |
| 65 | + | |
| 66 | +✅ Avant de terminer toute tâche, vérifie que tous les fichiers touchés portent l'en-tête. | |
| 67 | +✅ Ajoute un hook / script `scripts/check_headers.py` qui échoue en CI si un fichier n'a pas l'en-tête. | |
| 68 | + | |
| 69 | +--- | |
| 70 | + | |
| 71 | +## 1. RÈGLE ABSOLUE — Recherche web intensive AVANT de coder | |
| 72 | + | |
| 73 | +Ce projet touche des domaines où tes connaissances peuvent être incomplètes ou périmées : normes comptables, fiscalité, API des ERP, design de compilateurs. **Tu ne dois JAMAIS deviner.** | |
| 74 | + | |
| 75 | +### Protocole de recherche obligatoire | |
| 76 | + | |
| 77 | +Avant chaque nouveau module ou décision de design importante : | |
| 78 | + | |
| 79 | +1. **Fais au minimum 3 à 5 recherches web ciblées** sur le sujet (spécifications, docs officielles, articles récents, RFC, standards existants). | |
| 80 | +2. **Consulte les sources primaires** : documentation officielle des ERP (SAP, QuickBooks, Xero, Odoo, Oracle NetSuite, Sage, Dynamics), sites gouvernementaux (ARC/Revenu Québec pour TPS/TVQ, IRS pour sales tax US), sites des normalisateurs (IFRS Foundation, FASB pour US GAAP, CPA Canada pour ASPE). | |
| 81 | +3. **Documente tes trouvailles** dans `docs/research/<sujet>.md` (avec en-tête auteur) : sources, dates de consultation, résumé, décisions prises. | |
| 82 | +4. **Vérifie l'existant** : avant d'inventer un format, recherche les standards déjà en place (voir §1.1) pour t'en inspirer ou t'y aligner. | |
| 83 | +5. Si une info fiscale ou normative est incertaine ou pourrait avoir changé : **recherche web obligatoire**, jamais de mémoire seule. | |
| 84 | + | |
| 85 | +### 1.1 Sujets à rechercher intensivement (checklist de démarrage) | |
| 86 | + | |
| 87 | +- [ ] **Standards de données comptables existants** : XBRL / XBRL-GL, ISO 20022, UBL (Universal Business Language), Peppol, OFX, camt.053, EDIFACT, hledger/beancount/ledger-cli (plain text accounting), REA ontology (Resources-Events-Agents), ValueFlows, OpenCorporates schemas. | |
| 88 | +- [ ] **Architecture LLVM** : structure de l'IR, forme SSA, pass manager, backends, tablegen — pour transposer correctement les concepts. | |
| 89 | +- [ ] **API ERP** : SAP OData/BAPI/IDoc, QuickBooks Online API (JournalEntry, Invoice, Payment), Xero Accounting API, Odoo XML-RPC/ORM, NetSuite SuiteTalk, Sage Intacct. | |
| 90 | +- [ ] **Normes comptables** : IFRS (IFRS 15 revenus, IFRS 16 locations, IAS 21 devises), US GAAP (ASC 606, ASC 842), ASPE canadien (chapitres pertinents). | |
| 91 | +- [ ] **Fiscalité** : TPS/TVQ Canada (taux actuels, règles de lieu de fourniture), sales tax US (nexus, taux par État), TVA UE si pertinent. | |
| 92 | +- [ ] **Event sourcing & double-entry engines** : Martin Fowler (Accounting Patterns), TigerBeetle, Formance Ledger, Modern Treasury, Increase, Stripe Ledger — architectures de grands ledgers. | |
| 93 | +- [ ] **Rapprochement bancaire & formats bancaires** : ISO 20022 camt, MT940, Plaid/Flinks API. | |
| 94 | +- [ ] **Structured outputs LLM** : meilleures pratiques actuelles pour extraction structurée (JSON Schema, tool use, validation). | |
| 95 | + | |
| 96 | +⚠️ Les taux de taxes, seuils de capitalisation, et versions d'API **changent**. Toujours vérifier la date des sources et privilégier les documents officiels récents. | |
| 97 | + | |
| 98 | +--- | |
| 99 | + | |
| 100 | +## 2. Vision du projet | |
| 101 | + | |
| 102 | +AIR est à la comptabilité ce que LLVM est à la compilation. | |
| 103 | + | |
| 104 | +**Problème** : chaque ERP (SAP, Oracle, QuickBooks, Xero, Sage, Odoo, Dynamics) réinvente le même modèle comptable. Les LLM sont excellents pour comprendre ("cette facture = 3 ordinateurs payés par Visa") mais peu fiables pour appliquer des centaines de règles comptables/fiscales sans erreur. | |
| 105 | + | |
| 106 | +**Solution** : séparer la *compréhension* (LLM) de l'*application des règles* (compilateur déterministe). | |
| 107 | + | |
| 108 | +``` | |
| 109 | +Facture / Email / Banque / POS / API | |
| 110 | + ↓ | |
| 111 | + LLM (extraction) | |
| 112 | + ↓ | |
| 113 | + AIR (événement économique, PAS une écriture) | |
| 114 | + ↓ | |
| 115 | + Passes (validation, taxes, FX, fraude, approbation) | |
| 116 | + ↓ | |
| 117 | + Compilation déterministe (AIC) | |
| 118 | + ↓ | |
| 119 | + Backends : SAP | QuickBooks | Xero | Odoo | IFRS | US GAAP | ASPE | |
| 120 | +``` | |
| 121 | + | |
| 122 | +**Principe fondamental** : le LLM ne produit JAMAIS une écriture comptable finale. Il produit uniquement de l'AIR. Le compilateur applique les politiques, normes et taxes de façon déterministe, traçable et testable. | |
| 123 | + | |
| 124 | +--- | |
| 125 | + | |
| 126 | +## 3. Composants de l'écosystème | |
| 127 | + | |
| 128 | +| Composant | Analogie LLVM | Rôle | | |
| 129 | +|---|---|---| | |
| 130 | +| **AIR** | LLVM IR | Format universel décrivant les événements économiques | | |
| 131 | +| **AIC** | clang/llc | Compilateur AIR → écritures comptables | | |
| 132 | +| **ALSL** | TableGen | Langage déclaratif de règles (politiques, taxes, normes) | | |
| 133 | +| **Backends** | x86/ARM backends | Générateurs SAP, QBO, Xero, Odoo, etc. | | |
| 134 | +| **Passes** | Optimization passes | Validation, fusion, netting, doublons, conformité | | |
| 135 | +| **SDK Agents** | libclang | API standard pour agents IA (syscalls comptables) | | |
| 136 | +| **AIR Kernel** | microkernel | Services : Ledger, Tax, FX, Policy, Period, Audit, Approval, Reporting | | |
| 137 | + | |
| 138 | +### 3.1 AIR — le format | |
| 139 | + | |
| 140 | +Un événement économique (`EconomicEvent`), pas un journal. Exemple cible : | |
| 141 | + | |
| 142 | +```yaml | |
| 143 | +EconomicEvent: | |
| 144 | + id: evt_01H... # ULID | |
| 145 | + type: Sale | |
| 146 | + seller: company:acme | |
| 147 | + buyer: customer:cust_123 | |
| 148 | + items: | |
| 149 | + - sku: chair-std | |
| 150 | + qty: 3 | |
| 151 | + unit_price: {amount: 333.33, currency: CAD} | |
| 152 | + payment: | |
| 153 | + method: card.visa | |
| 154 | + gross: {amount: 1150.00, currency: CAD} | |
| 155 | + delivery: {status: pending, expected: 2026-09-01} | |
| 156 | + tax: | |
| 157 | + jurisdiction: CA-QC | |
| 158 | + codes: [GST, QST] | |
| 159 | + meta: | |
| 160 | + source: {kind: invoice_pdf, uri: "s3://...", ocr_score: 0.97} | |
| 161 | + llm: {model: "...", confidence: 0.93, reasoning_hash: "sha256:..."} | |
| 162 | + policy_version: "2026.08" | |
| 163 | + timestamps: {ingested: ..., approved: null} | |
| 164 | + approver: null | |
| 165 | +``` | |
| 166 | + | |
| 167 | +Exigences du format : | |
| 168 | +- **Schéma formel** (JSON Schema + types Pydantic/TypeScript générés). | |
| 169 | +- **Forme SSA comptable** : chaque montant a une origine unique et traçable (Invoice → Tax → Payment → FX → Settlement → Write-off). Implémenter un graphe de provenance immuable. | |
| 170 | +- **Versionné** : `air_version` dans chaque document ; migrations explicites. | |
| 171 | +- **Immutabilité + diff** : compilation incrémentale — si une facture change, on recompile uniquement le delta (comme Git), avec écritures de contrepassation générées automatiquement. | |
| 172 | + | |
| 173 | +### 3.2 AIC — le compilateur | |
| 174 | + | |
| 175 | +Pipeline de passes ordonnées : | |
| 176 | + | |
| 177 | +``` | |
| 178 | +OCR pass → Classification pass → Tax pass → FX pass → | |
| 179 | +Fraud pass → Approval pass → Optimization passes → Posting pass | |
| 180 | +``` | |
| 181 | + | |
| 182 | +Passes d'optimisation : | |
| 183 | +- **Fusion** : 50 paiements identiques → 1 batch. | |
| 184 | +- **Netting** : 100 remboursements → compensation. | |
| 185 | +- **Reclassement** : Expense → Asset selon politiques (ex. capitalisation > 5000 $). | |
| 186 | +- **Détection doublons** : hachage + similarité. | |
| 187 | +- **Invariant permanent** : `Assets = Liabilities + Equity` vérifié après CHAQUE passe. Toute violation = échec de compilation avec diagnostic précis (comme les erreurs clang : localisation, cause, suggestion). | |
| 188 | + | |
| 189 | +### 3.3 ALSL — le langage de règles | |
| 190 | + | |
| 191 | +Déclaratif, versionné, testable : | |
| 192 | + | |
| 193 | +``` | |
| 194 | +policy capitalization_ca: | |
| 195 | + when event.type == Purchase and event.amount > 5000 CAD | |
| 196 | + then classify as Asset(class: equipment) | |
| 197 | + | |
| 198 | +policy tax_quebec: | |
| 199 | + when event.jurisdiction == CA-QC | |
| 200 | + then apply GST(5%), QST(9.975%) | |
| 201 | +``` | |
| 202 | + | |
| 203 | +⚠️ Les taux ci-dessus sont des exemples — **vérifie les taux actuels par recherche web** avant de les coder, et ne les code JAMAIS en dur dans le moteur : ils vivent dans les policies ALSL versionnées. | |
| 204 | + | |
| 205 | +### 3.4 Backends | |
| 206 | + | |
| 207 | +Chaque backend implémente une interface commune `Backend`: | |
| 208 | +- `capabilities()` — ce que la cible supporte | |
| 209 | +- `compile(journal: CompiledJournal) -> TargetPayload` | |
| 210 | +- `post(payload) -> PostingReceipt` | |
| 211 | +- `reverse(receipt) -> ReversalReceipt` | |
| 212 | + | |
| 213 | +Ordre de développement : **1) Backend générique CSV/journal**, 2) QuickBooks Online (API la plus accessible), 3) Xero, 4) Odoo, 5) SAP (le plus complexe — recherche approfondie requise sur OData/BAPI). | |
| 214 | + | |
| 215 | +### 3.5 SDK Agents — les "syscalls" | |
| 216 | + | |
| 217 | +Les agents IA n'accèdent JAMAIS au grand livre directement. API exclusive : | |
| 218 | + | |
| 219 | +``` | |
| 220 | +CreateEconomicEvent() | Validate() | Compile() | Post() | Reverse() | |
| 221 | +Merge() | ClosePeriod() | Reconcile() | GenerateReport() | |
| 222 | +``` | |
| 223 | + | |
| 224 | +Chaque appel est journalisé (audit log append-only, hash-chaîné). | |
| 225 | + | |
| 226 | +--- | |
| 227 | + | |
| 228 | +## 4. Architecture technique | |
| 229 | + | |
| 230 | +### Stack recommandée (à valider par recherche) | |
| 231 | +- **Cœur (AIR + AIC + passes)** : Rust (fiabilité, typage fort) OU Python typé strict (vitesse de dev). Décision à documenter dans `docs/adr/0001-language.md` après recherche comparative. | |
| 232 | +- **Schémas** : JSON Schema comme source de vérité → génération de types Rust/Python/TS. | |
| 233 | +- **Montants** : JAMAIS de float. Décimal fixe (rust_decimal / Python Decimal), arrondi banker's rounding documenté par juridiction (à vérifier par recherche : règles d'arrondi TPS/TVQ). | |
| 234 | +- **Stockage** : event store append-only (PostgreSQL) + projections. | |
| 235 | +- **ALSL** : parser dédié (pest/lark) ; commencer par un sous-ensemble YAML avant le DSL complet. | |
| 236 | + | |
| 237 | +### Structure du dépôt | |
| 238 | + | |
| 239 | +``` | |
| 240 | +air/ | |
| 241 | +├── CLAUDE.md | |
| 242 | +├── README.md | |
| 243 | +├── docs/ | |
| 244 | +│ ├── adr/ # Architecture Decision Records | |
| 245 | +│ ├── research/ # Résultats de recherches web (OBLIGATOIRE) | |
| 246 | +│ └── spec/ # Spécification formelle AIR / ALSL | |
| 247 | +├── schemas/ # JSON Schemas versionnés | |
| 248 | +├── core/ # AIR types + graphe de provenance | |
| 249 | +├── aic/ # Compilateur + pass manager | |
| 250 | +│ └── passes/ | |
| 251 | +├── alsl/ # Parser + évaluateur de règles | |
| 252 | +├── backends/ | |
| 253 | +│ ├── generic_csv/ | |
| 254 | +│ ├── quickbooks/ | |
| 255 | +│ ├── xero/ | |
| 256 | +│ └── odoo/ | |
| 257 | +├── kernel/ # Services (ledger, tax, fx, policy, audit...) | |
| 258 | +├── sdk/ # SDK agents (syscalls) | |
| 259 | +├── ingestion/ # OCR + extraction LLM → AIR | |
| 260 | +├── tests/ | |
| 261 | +│ ├── golden/ # Cas dorés : AIR → écritures attendues | |
| 262 | +│ ├── property/ # Property-based (invariant bilan) | |
| 263 | +│ └── fixtures/ # Factures réelles anonymisées | |
| 264 | +└── scripts/ | |
| 265 | + └── check_headers.py # Vérifie les en-têtes auteur | |
| 266 | +``` | |
| 267 | + | |
| 268 | +--- | |
| 269 | + | |
| 270 | +## 5. Exigences de qualité non négociables | |
| 271 | + | |
| 272 | +1. **Déterminisme** : même AIR + mêmes policies = mêmes écritures, toujours. Aucun appel LLM dans le compilateur. | |
| 273 | +2. **Invariant comptable** : partie double vérifiée à chaque étape ; property-based testing (hypothesis/proptest) sur `Assets = Liabilities + Equity`. | |
| 274 | +3. **Traçabilité totale** : de chaque ligne d'écriture, on remonte à l'événement source, au document, au score OCR/LLM, à la version de policy, à l'approbateur. | |
| 275 | +4. **Golden tests** : chaque fonctionnalité comptable = cas dorés validés contre des exemples de la littérature comptable (trouvés par recherche web, sources citées). | |
| 276 | +5. **Diagnostics de qualité compilateur** : erreurs précises, localisées, avec suggestions. | |
| 277 | +6. **Aucun taux/seuil en dur** : tout paramètre fiscal ou de politique vit dans ALSL. | |
| 278 | +7. **En-têtes auteur** partout (voir §0), vérifiés en CI. | |
| 279 | + | |
| 280 | +--- | |
| 281 | + | |
| 282 | +## 6. Plan de développement par phases | |
| 283 | + | |
| 284 | +**Phase 0 — Recherche (docs/research/)** : compléter la checklist §1.1, produire un rapport par sujet, rédiger les ADR fondateurs (langage, alignement ou non avec XBRL-GL/UBL/REA, modèle de montants). | |
| 285 | + | |
| 286 | +**Phase 1 — Cœur** : schéma AIR v0.1, types, graphe de provenance, invariant bilan, backend CSV générique, 20 golden tests (vente simple, achat, taxes QC, FX, remboursement). | |
| 287 | + | |
| 288 | +**Phase 2 — Compilateur** : pass manager, passes Tax (CA-QC d'abord) / FX / Validation, ALSL v0.1 (YAML), compilation incrémentale (diff + reversal). | |
| 289 | + | |
| 290 | +**Phase 3 — Backend réel** : QuickBooks Online (sandbox), OAuth, idempotence, Post/Reverse. | |
| 291 | + | |
| 292 | +**Phase 4 — Ingestion** : PDF → extraction structurée LLM → AIR, avec scores de confiance et file d'approbation humaine sous seuil. | |
| 293 | + | |
| 294 | +**Phase 5 — Kernel + SDK** : services, syscalls, audit log hash-chaîné, agent de démonstration. | |
| 295 | + | |
| 296 | +**Phase 6 — Optimisations** : fusion, netting, doublons, rapprochement bancaire (camt.053/MT940). | |
| 297 | + | |
| 298 | +--- | |
| 299 | + | |
| 300 | +## 7. Workflow de travail attendu de Claude | |
| 301 | + | |
| 302 | +À chaque session : | |
| 303 | +1. Relire ce CLAUDE.md. | |
| 304 | +2. Identifier la tâche → **recherche web d'abord** si le sujet touche normes, taxes, API externes, ou standards. | |
| 305 | +3. Consigner la recherche dans `docs/research/`. | |
| 306 | +4. Écrire les tests avant/avec le code (golden + property). | |
| 307 | +5. Coder avec en-têtes auteur. | |
| 308 | +6. Lancer `scripts/check_headers.py` + suite de tests. | |
| 309 | +7. Mettre à jour la doc/ADR si une décision de design a été prise. | |
| 310 | + | |
| 311 | +### Interdits | |
| 312 | +- ❌ Coder une règle fiscale de mémoire sans source web vérifiée et citée. | |
| 313 | +- ❌ Utiliser des floats pour des montants. | |
| 314 | +- ❌ Laisser le LLM produire des écritures finales. | |
| 315 | +- ❌ Créer un fichier sans l'en-tête Simon-Pierre Boucher / contact@spboucher.ai. | |
| 316 | +- ❌ Casser l'invariant de la partie double, même temporairement. | |
| 317 | + | |
| 318 | +--- | |
| 319 | + | |
| 320 | +*Fin du CLAUDE.md — Projet AIR — Simon-Pierre Boucher — contact@spboucher.ai* | |
added
CONTRIBUTING.md
+57 −0
@@ -0,0 +1,57 @@ | ||
| 1 | +<!-- | |
| 2 | +Projet : AIR — Accounting Intermediate Representation | |
| 3 | +Auteur : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +Fichier : CONTRIBUTING.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Contributing to AIR | |
| 9 | + | |
| 10 | +Thank you for considering a contribution! AIR has a few **non-negotiable | |
| 11 | +rules** that CI enforces — read them before opening a PR. | |
| 12 | + | |
| 13 | +## The rules (enforced) | |
| 14 | + | |
| 15 | +1. **Author header on every file.** Every source/doc/config file starts with | |
| 16 | + the project header (see any file for the format). `python scripts/check_headers.py` | |
| 17 | + must pass — CI fails otherwise. | |
| 18 | +2. **No floats for money — ever.** Amounts, quantities, rates are `Decimal` | |
| 19 | + in code and strings in JSON/YAML. Floats are rejected at every boundary. | |
| 20 | +3. **No tax rate, threshold, or rounding mode in code.** All fiscal and policy | |
| 21 | + parameters live in versioned ALSL policy sets with a mandatory source | |
| 22 | + citation. The loader rejects uncited rates. | |
| 23 | +4. **Research before rules.** Anything touching tax law, accounting standards, | |
| 24 | + external APIs, or data formats needs a cited report in `docs/research/` | |
| 25 | + (official sources, consultation dates). Never from memory. | |
| 26 | +5. **The compiler stays deterministic.** No LLM, network, or clock inside AIC. | |
| 27 | + Same document + same policies = byte-identical journal. | |
| 28 | +6. **The invariant is sacred.** `Assets = Liabilities + Equity` is verified | |
| 29 | + after every compiler pass. New passes must keep entries balanced. | |
| 30 | +7. **Append-only history.** Corrections are reversal entries; nothing posted | |
| 31 | + is ever edited or deleted. | |
| 32 | + | |
| 33 | +## Workflow | |
| 34 | + | |
| 35 | +```bash | |
| 36 | +python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" | |
| 37 | +.venv/bin/python -m pytest tests/ -q --ignore=tests/test_llm_live.py # offline suite | |
| 38 | +python3 scripts/check_headers.py | |
| 39 | +``` | |
| 40 | + | |
| 41 | +- Write tests with the code: golden cases for accounting behavior | |
| 42 | + (`tests/golden/cases/*.yaml`), property tests for invariants, unit tests | |
| 43 | + for mechanics. | |
| 44 | +- New event types need: posting rules, golden cases, and a spec update. | |
| 45 | +- New backends implement `backends/base.py` (`capabilities/compile/post/reverse`) | |
| 46 | + and must be fully testable offline (mock transport pattern — | |
| 47 | + see `backends/quickbooks/`). | |
| 48 | +- Architecture decisions get an ADR in `docs/adr/`. | |
| 49 | + | |
| 50 | +## What makes a good first contribution | |
| 51 | + | |
| 52 | +- A new ALSL policy set for another jurisdiction (with cited research) | |
| 53 | +- The beancount/hledger export backend (near-free per the research) | |
| 54 | +- Reference-first matching (`EndToEndId`) in reconciliation | |
| 55 | +- A Xero or Odoo backend with an offline mock transport | |
| 56 | + | |
| 57 | +*Author: Simon-Pierre Boucher — contact@spboucher.ai* | |
added
LICENSE
+21 −0
@@ -0,0 +1,21 @@ | ||
| 1 | +MIT License | |
| 2 | + | |
| 3 | +Copyright (c) 2026 Simon-Pierre Boucher <contact@spboucher.ai> | |
| 4 | + | |
| 5 | +Permission is hereby granted, free of charge, to any person obtaining a copy | |
| 6 | +of this software and associated documentation files (the "Software"), to deal | |
| 7 | +in the Software without restriction, including without limitation the rights | |
| 8 | +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |
| 9 | +copies of the Software, and to permit persons to whom the Software is | |
| 10 | +furnished to do so, subject to the following conditions: | |
| 11 | + | |
| 12 | +The above copyright notice and this permission notice shall be included in all | |
| 13 | +copies or substantial portions of the Software. | |
| 14 | + | |
| 15 | +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 16 | +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 17 | +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |
| 18 | +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |
| 19 | +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |
| 20 | +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | |
| 21 | +SOFTWARE. | |
added
README.md
+233 −0
@@ -0,0 +1,233 @@ | ||
| 1 | +<!-- | |
| 2 | +Projet : AIR — Accounting Intermediate Representation | |
| 3 | +Auteur : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +Fichier : README.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +<div align="center"> | |
| 9 | + | |
| 10 | +# ⚙️ AIR | |
| 11 | + | |
| 12 | +### The Language of Accounting | |
| 13 | + | |
| 14 | +**LLVM for the ledger** — LLMs understand your documents, a deterministic compiler keeps your books. | |
| 15 | + | |
| 16 | +[](https://github.com/spboucher-ai/air/actions/workflows/ci.yml) | |
| 17 | +[](pyproject.toml) | |
| 18 | +[](LICENSE) | |
| 19 | +[](tests/) | |
| 20 | +[](tests/golden/cases/) | |
| 21 | +[](docs/adr/0003-monetary-amounts.md) | |
| 22 | +[](core/invariant.py) | |
| 23 | +[](kernel/ledger.py) | |
| 24 | +[](docs/cli-reference.md) | |
| 25 | + | |
| 26 | +*by [Simon-Pierre Boucher](mailto:contact@spboucher.ai)* | |
| 27 | + | |
| 28 | +</div> | |
| 29 | + | |
| 30 | +--- | |
| 31 | + | |
| 32 | +## The idea | |
| 33 | + | |
| 34 | +Every ERP reinvents the same accounting model. LLMs are brilliant at *understanding* | |
| 35 | +("this invoice = 3 chairs paid by Visa") and unreliable at *applying* hundreds of | |
| 36 | +tax and accounting rules. **AIR separates the two** — exactly like LLVM separated | |
| 37 | +language frontends from machine backends: | |
| 38 | + | |
| 39 | +```mermaid | |
| 40 | +flowchart LR | |
| 41 | + A[📄 Invoice / Email<br/>Bank / POS / API] --> B["🧠 LLM<br/><i>understanding</i>"] | |
| 42 | + B -->|"AIR events<br/>(never journal entries)"| C["⚙️ AIC Compiler<br/><i>deterministic rules</i>"] | |
| 43 | + P["📜 ALSL Policies<br/>taxes · thresholds · rounding<br/><i>versioned & cited</i>"] --> C | |
| 44 | + C -->|"balanced entries<br/>+ full provenance"| D["📗 Native Ledger<br/>hash-chained, standalone"] | |
| 45 | + C --> E["📤 CSV · QuickBooks<br/>Xero · Odoo · SAP*"] | |
| 46 | + D --> F["📊 Trial balance · Balance sheet<br/>Income statement · GL"] | |
| 47 | + style B fill:#f9e79f,stroke:#b7950b | |
| 48 | + style C fill:#aed6f1,stroke:#2471a3 | |
| 49 | + style D fill:#a9dfbf,stroke:#1e8449 | |
| 50 | + style P fill:#f5b7b1,stroke:#c0392b | |
| 51 | +``` | |
| 52 | + | |
| 53 | +> **The one rule that never bends:** an LLM (or any frontend) only ever produces | |
| 54 | +> **AIR** — a description of *what happened economically*. It never writes a journal | |
| 55 | +> entry, an account code, or a debit. The compiler does that, deterministically, | |
| 56 | +> with the double-entry invariant `Assets = Liabilities + Equity` verified after | |
| 57 | +> **every** pass and clang-style diagnostics when anything is wrong. | |
| 58 | + | |
| 59 | +## What you get | |
| 60 | + | |
| 61 | +| | | | |
| 62 | +|---|---| | |
| 63 | +| 🗣️ **A universal language** | `EconomicEvent` (REA-based): sales, purchases, refunds, payments, FX — perspective-neutral, schema-validated, no debits anywhere | | |
| 64 | +| ⚙️ **A real compiler** | Pass pipeline (validation → classification → tax → FX → posting), pass manager with verify-after-every-pass, diagnostics with location + cause + `help:` fix | | |
| 65 | +| 📜 **Rules as data (ALSL)** | GST 5%, QST 9.975%, HST, capitalization thresholds, rounding modes — all in versioned YAML policy sets with **mandatory source citations** (the loader rejects uncited rates) | | |
| 66 | +| 🏠 **Standalone books** | No ERP required: managed AIR home with an append-only, tamper-evident ledger, content-addressed document archive, and all financial statements in text/markdown/CSV/JSON | | |
| 67 | +| 🔁 **Git-style corrections** | Changed invoice? `air recompile` diffs by content fingerprint and posts reversal + replacement entries — history is never edited | | |
| 68 | +| 🤖 **An agent SDK** | AI agents get syscalls (`CreateEconomicEvent`, `Post`, `Reverse`, `ClosePeriod`, `Merge`, `Reconcile`…) — never the ledger. Every call, including refusals, lands in a hash-chained audit log | | |
| 69 | +| 🧾 **LLM ingestion** | Invoice text → Claude (structured outputs) → schema-validated AIR → confidence routing → human approval inbox. Fully offline mock for tests | | |
| 70 | +| 🚀 **Optimizations** | 50 identical payments → 1 batch entry (fusion), refund↔sale netting, duplicate detection — provenance preserved through every transformation | | |
| 71 | +| 🏦 **Bank reconciliation** | camt.053 + MT940 (with statement integrity check) + CSV → matched against the books, differences reported on both sides | | |
| 72 | +| 🔍 **Total traceability** | Every posted cent walks back through the provenance graph (accounting SSA) to its source event, document, OCR/LLM confidence, policy version, and rounding mode | | |
| 73 | + | |
| 74 | +## Quickstart | |
| 75 | + | |
| 76 | +```bash | |
| 77 | +git clone https://github.com/spboucher-ai/air && cd air | |
| 78 | +python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" | |
| 79 | +source .venv/bin/activate | |
| 80 | + | |
| 81 | +# 1) Your books live in a managed AIR home — one command | |
| 82 | +air init --home books --policies alsl/policies/ca-qc-2026.yaml | |
| 83 | + | |
| 84 | +# 2) Compile economic events into the books (document archived, ledger chained) | |
| 85 | +air compile tests/fixtures/demo_document.yaml --home books \ | |
| 86 | + --report trial-balance --report balance-sheet | |
| 87 | + | |
| 88 | +# 3) A source invoice was corrected? Post only the delta (reversal + replacement) | |
| 89 | +air recompile old.yaml corrected.yaml --home books | |
| 90 | + | |
| 91 | +# 4) Statements any time, any format | |
| 92 | +air report income-statement --home books --format markdown | |
| 93 | + | |
| 94 | +# 5) Ingest a real document (offline extractor; add --llm for Claude) | |
| 95 | +air ingest tests/fixtures/invoice_high_confidence.txt --home books | |
| 96 | +air inbox --home books | |
| 97 | + | |
| 98 | +# 6) Month end | |
| 99 | +air reconcile statement.mt940 --home books | |
| 100 | +air audit --home books | |
| 101 | +``` | |
| 102 | + | |
| 103 | +Full command reference: **[docs/cli-reference.md](docs/cli-reference.md)** | |
| 104 | + | |
| 105 | +## Sixty seconds of AIR | |
| 106 | + | |
| 107 | +An economic event — *what happened*, nothing else: | |
| 108 | + | |
| 109 | +```yaml | |
| 110 | +events: | |
| 111 | + - id: evt_01H8XGJWBW | |
| 112 | + type: Sale | |
| 113 | + date: 2026-07-20 | |
| 114 | + parties: {seller: "company:acme", buyer: "customer:cust_123"} | |
| 115 | + items: | |
| 116 | + - {sku: chair-std, qty: "3", unit_price: {amount: "333.33", currency: CAD}} | |
| 117 | + payment: {method: card.visa, immediate: true} | |
| 118 | + tax: {jurisdiction: CA-QC} # ← where. Rates live in policies, never here | |
| 119 | +``` | |
| 120 | + | |
| 121 | +The compiler applies the cited CA-QC policy set (GST 5%, QST 9.975% on the | |
| 122 | +pre-GST price, half-up per Excise Tax Act s.165.2(2)) and emits a balanced entry: | |
| 123 | + | |
| 124 | +``` | |
| 125 | +DR 1000 Cash 1149.74 | |
| 126 | + CR 4000 Sales revenue 999.99 | |
| 127 | + CR 2310 GST payable 50.00 tax:GST@0.05~half_up | |
| 128 | + CR 2320 QST payable 99.75 tax:QST@0.09975~half_up | |
| 129 | +``` | |
| 130 | + | |
| 131 | +Same input + same policies = **byte-identical output, always** (property-tested, | |
| 132 | +and CI compiles the demo twice and `diff`s the results). When something is wrong, | |
| 133 | +you get a compiler error, not a wrong number: | |
| 134 | + | |
| 135 | +``` | |
| 136 | +error[AIR-E400]: no tax policy in set 'ca-qc' matches jurisdiction 'CA-BC' | |
| 137 | + --> event evt_x, field tax.jurisdiction | |
| 138 | + pass: tax | |
| 139 | + help: add an ALSL tax policy for this jurisdiction or mark the event tax.exempt: true | |
| 140 | +``` | |
| 141 | + | |
| 142 | +## Agents keep books through syscalls — and only syscalls | |
| 143 | + | |
| 144 | +```python | |
| 145 | +from sdk.syscalls import AirKernel | |
| 146 | + | |
| 147 | +k = AirKernel("books", actor="agent:alice") | |
| 148 | +k.create_economic_event({...}) # schema-validated draft | |
| 149 | +k.post() # deterministic compile → books (idempotent) | |
| 150 | +k.reverse("je_evt_x") # corrections are contra entries, never edits | |
| 151 | +k.close_period("2026-01") # posting into it now fails with AIR-E700 | |
| 152 | +k.merge() # post with netting + payment fusion | |
| 153 | +k.reconcile("statement.mt940") # bank matching | |
| 154 | +``` | |
| 155 | + | |
| 156 | +``` | |
| 157 | +$ air audit --home books | |
| 158 | +#0000 OK agent:alice CreateEconomicEvent {"event_id": "evt_x", "type": "Sale"} | |
| 159 | +#0001 OK agent:alice Post {"drafts": 1} | |
| 160 | +#0008 ERR agent:alice Post {"drafts": 1} <- error[AIR-E700]: period 2026-01 is closed | |
| 161 | +10 syscalls, hash chain VALID | |
| 162 | +``` | |
| 163 | + | |
| 164 | +Refused actions are audited like successful ones. Editing any record breaks the chain. | |
| 165 | + | |
| 166 | +## The seven guarantees | |
| 167 | + | |
| 168 | +1. **Determinism** — no LLM, network, or clock inside the compiler. | |
| 169 | +2. **Double entry** — the invariant is verified after *every* pass; violations abort compilation. | |
| 170 | +3. **Traceability** — provenance graph from every posted line to its source (accounting SSA). | |
| 171 | +4. **No floats** — exact decimals end to end; floats rejected at every boundary. | |
| 172 | +5. **No rules in code** — every rate/threshold lives in cited, versioned ALSL policies. | |
| 173 | +6. **Append-only** — reversals, never edits; ledger and audit log are hash-chained. | |
| 174 | +7. **Human in the loop** — low-confidence or schema-invalid extractions always route to a person. | |
| 175 | + | |
| 176 | +## Repository map | |
| 177 | + | |
| 178 | +| Path | Role | LLVM analogy | | |
| 179 | +|---|---|---| | |
| 180 | +| [`schemas/`](schemas/) | AIR JSON Schema | the IR definition | | |
| 181 | +| [`core/`](core/) | events, Money, provenance, invariants | IR + verifier | | |
| 182 | +| [`aic/`](aic/) | pass manager, passes, diagnostics, incremental | opt/llc | | |
| 183 | +| [`alsl/`](alsl/) | rule language + cited policy sets | TableGen | | |
| 184 | +| [`backends/`](backends/) | native, CSV, QuickBooks (offline-tested) | targets | | |
| 185 | +| [`kernel/`](kernel/) | ledger, reporting, workspace, audit, reconciliation | runtime | | |
| 186 | +| [`sdk/`](sdk/) | CLI, agent syscalls, demo agent | libclang | | |
| 187 | +| [`ingestion/`](ingestion/) | extractors, confidence routing, approval queue | frontend | | |
| 188 | +| [`docs/spec/`](docs/spec/) | **the AIR/ALSL specification** | LangRef | | |
| 189 | +| [`docs/adr/`](docs/adr/) | architecture decision records | — | | |
| 190 | +| [`docs/research/`](docs/research/) | 9 cited research reports (tax law, formats, APIs) | — | | |
| 191 | +| [`tests/`](tests/) | golden cases, property tests, unit + live tests | lit | | |
| 192 | + | |
| 193 | +## Documentation | |
| 194 | + | |
| 195 | +- 📖 **[The AIR Specification](docs/spec/air-spec-v0.1.md)** — the language, the compiler, diagnostics, syscalls, reconciliation | |
| 196 | +- 🧭 **[CLI Reference](docs/cli-reference.md)** — every command with examples | |
| 197 | +- 🏛️ **[ADRs](docs/adr/)** — why Python, why REA events, why rounding is policy | |
| 198 | +- 🔬 **[Research](docs/research/)** — GST/QST from official sources, camt.053/MT940 specs, ERP API surveys, LLVM architecture lessons | |
| 199 | +- 🤝 **[Contributing](CONTRIBUTING.md)** — the non-negotiable rules CI enforces | |
| 200 | + | |
| 201 | +## Development | |
| 202 | + | |
| 203 | +```bash | |
| 204 | +.venv/bin/python -m pytest tests/ -q --ignore=tests/test_llm_live.py # 99 tests, fully offline | |
| 205 | +python3 scripts/check_headers.py # author-header gate | |
| 206 | + | |
| 207 | +# opt-in live LLM tests (cost a few cents) | |
| 208 | +ANTHROPIC_API_KEY=sk-ant-... .venv/bin/python -m pytest tests/test_llm_live.py -v | |
| 209 | +``` | |
| 210 | + | |
| 211 | +Everything — including the QuickBooks backend and the LLM ingestion — develops | |
| 212 | +and tests **fully offline**: mock transports simulate the real APIs' documented | |
| 213 | +behaviors (idempotency replay, duplicate errors), so no account or key is ever | |
| 214 | +required to work on AIR. | |
| 215 | + | |
| 216 | +## Roadmap | |
| 217 | + | |
| 218 | +- [x] Phases 0–6: research → core → compiler → backends → ingestion → agent SDK → optimizations & reconciliation *(complete)* | |
| 219 | +- [ ] beancount/hledger export backend | |
| 220 | +- [ ] Xero & Odoo backends (offline mock pattern) | |
| 221 | +- [ ] Reference-first bank matching (`EndToEndId`) | |
| 222 | +- [ ] More jurisdictions as cited ALSL policy sets (US sales tax, EU VAT) | |
| 223 | +- [ ] AIR v0.2: REA commitments (pending deliveries, IFRS 15 performance obligations) | |
| 224 | + | |
| 225 | +--- | |
| 226 | + | |
| 227 | +<div align="center"> | |
| 228 | + | |
| 229 | +**AIR** — *because your books deserve a compiler.* | |
| 230 | + | |
| 231 | +MIT © 2026 [Simon-Pierre Boucher](mailto:contact@spboucher.ai) | |
| 232 | + | |
| 233 | +</div> | |
added
aic/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
aic/compiler.py
+76 −0
@@ -0,0 +1,76 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : compiler.py | |
| 6 | +# Description : Top-level AIC entry point — AIR document + ALSL policies -> CompiledJournal. | |
| 7 | +# ============================================================================= | |
| 8 | +"""compile_document(): the deterministic core of AIR. | |
| 9 | + | |
| 10 | +Same AIR document + same policy set => byte-identical journal, always. | |
| 11 | +No LLM, no network, no clock inside this function. | |
| 12 | +""" | |
| 13 | +from __future__ import annotations | |
| 14 | + | |
| 15 | +from aic.diagnostics import Diagnostic | |
| 16 | +from aic.pass_manager import PassManager | |
| 17 | +from aic.passes.classification import ClassificationPass | |
| 18 | +from aic.passes.fx import FxPass | |
| 19 | +from aic.passes.posting import PostingPass | |
| 20 | +from aic.passes.tax import TaxPass | |
| 21 | +from aic.passes.validation import ValidationPass | |
| 22 | +from aic.unit import CompilationUnit | |
| 23 | +from alsl.model import PolicySet | |
| 24 | +from core.events import AirDocument | |
| 25 | +from core.journal import CompiledJournal | |
| 26 | + | |
| 27 | +DEFAULT_PIPELINE = [ | |
| 28 | + ValidationPass(), | |
| 29 | + ClassificationPass(), | |
| 30 | + TaxPass(), | |
| 31 | + FxPass(), | |
| 32 | + PostingPass(), | |
| 33 | +] | |
| 34 | + | |
| 35 | + | |
| 36 | +def _optimized_pipeline() -> list: | |
| 37 | + from aic.passes.optimize import ( | |
| 38 | + DuplicateDetectionPass, | |
| 39 | + FusionPass, | |
| 40 | + NettingPass, | |
| 41 | + ) | |
| 42 | + return [ | |
| 43 | + ValidationPass(), | |
| 44 | + DuplicateDetectionPass(), | |
| 45 | + ClassificationPass(), | |
| 46 | + TaxPass(), | |
| 47 | + FxPass(), | |
| 48 | + PostingPass(), | |
| 49 | + NettingPass(), | |
| 50 | + FusionPass(), | |
| 51 | + ] | |
| 52 | + | |
| 53 | + | |
| 54 | +def compile_document( | |
| 55 | + document: AirDocument, | |
| 56 | + policies: PolicySet, | |
| 57 | + pipeline: list | None = None, | |
| 58 | + optimize: bool = False, | |
| 59 | +) -> tuple[CompiledJournal, list[Diagnostic]]: | |
| 60 | + """Compile an AIR document into a journal. Raises CompilationError on errors. | |
| 61 | + | |
| 62 | + optimize=True adds the Phase 6 passes (duplicate detection, netting, | |
| 63 | + fusion); the invariant is still verified after every pass. | |
| 64 | + """ | |
| 65 | + if pipeline is None: | |
| 66 | + pipeline = _optimized_pipeline() if optimize else DEFAULT_PIPELINE | |
| 67 | + unit = CompilationUnit(document=document, policies=policies) | |
| 68 | + manager = PassManager(pipeline) | |
| 69 | + manager.run(unit) | |
| 70 | + journal = CompiledJournal( | |
| 71 | + entries=list(unit.entries), | |
| 72 | + provenance=unit.provenance, | |
| 73 | + policy_set=policies.name, | |
| 74 | + policy_version=policies.version, | |
| 75 | + ) | |
| 76 | + return journal, unit.diagnostics | |
added
aic/diagnostics.py
+53 −0
@@ -0,0 +1,53 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : diagnostics.py | |
| 6 | +# Description : Clang-style diagnostics — precise location, cause, suggestion. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Compiler-quality diagnostics for AIC. | |
| 9 | + | |
| 10 | +Modeled on clang: every diagnostic names WHERE (event id / field path), | |
| 11 | +WHAT (computed cause, e.g. the exact imbalance), and HOW TO FIX (suggestion). | |
| 12 | +Errors abort compilation; warnings and notes ride along in the result. | |
| 13 | +""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +import enum | |
| 17 | +from dataclasses import dataclass | |
| 18 | + | |
| 19 | + | |
| 20 | +class Severity(enum.Enum): | |
| 21 | + ERROR = "error" | |
| 22 | + WARNING = "warning" | |
| 23 | + NOTE = "note" | |
| 24 | + | |
| 25 | + | |
| 26 | +@dataclass(frozen=True, slots=True) | |
| 27 | +class Diagnostic: | |
| 28 | + code: str # e.g. "AIR-E101" | |
| 29 | + severity: Severity | |
| 30 | + message: str | |
| 31 | + location: str # e.g. "event evt_01H..., field items[0].unit_price" | |
| 32 | + suggestion: str | None = None | |
| 33 | + origin_pass: str | None = None | |
| 34 | + | |
| 35 | + def render(self) -> str: | |
| 36 | + head = f"{self.severity.value}[{self.code}]: {self.message}" | |
| 37 | + lines = [head, f" --> {self.location}"] | |
| 38 | + if self.origin_pass: | |
| 39 | + lines.append(f" pass: {self.origin_pass}") | |
| 40 | + if self.suggestion: | |
| 41 | + lines.append(f" help: {self.suggestion}") | |
| 42 | + return "\n".join(lines) | |
| 43 | + | |
| 44 | + | |
| 45 | +class CompilationError(Exception): | |
| 46 | + """Raised when any ERROR diagnostic is produced; carries all diagnostics.""" | |
| 47 | + | |
| 48 | + def __init__(self, diagnostics: list[Diagnostic]): | |
| 49 | + self.diagnostics = diagnostics | |
| 50 | + errors = [d for d in diagnostics if d.severity is Severity.ERROR] | |
| 51 | + super().__init__( | |
| 52 | + f"{len(errors)} error(s):\n" + "\n".join(d.render() for d in errors) | |
| 53 | + ) | |
added
aic/incremental.py
+154 −0
@@ -0,0 +1,154 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : incremental.py | |
| 6 | +# Description : Incremental compilation — diff two AIR documents, emit reversal + replacement entries. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Incremental compilation (Phase 2). | |
| 9 | + | |
| 10 | +When a source document changes (a corrected invoice, a removed duplicate, a | |
| 11 | +new event), AIR never mutates posted history. Instead, like Git, we diff the | |
| 12 | +old and new documents by event fingerprint and emit: | |
| 13 | + | |
| 14 | +- a REVERSAL entry (exact contra) for every changed or removed event; | |
| 15 | +- a replacement entry (with a revision-suffixed id) for every changed event; | |
| 16 | +- a normal entry for every added event; | |
| 17 | +- nothing for unchanged events. | |
| 18 | + | |
| 19 | +Determinism holds: same (old, new, policies) triple => identical delta. | |
| 20 | + | |
| 21 | +Provenance scoping: reversal lines keep the provenance ids of the prior | |
| 22 | +compilation (they describe the amounts being reversed); the `reverses` field | |
| 23 | +links each contra entry to the original. Replacement/added entries carry the | |
| 24 | +new compilation's provenance graph. | |
| 25 | +""" | |
| 26 | +from __future__ import annotations | |
| 27 | + | |
| 28 | +import hashlib | |
| 29 | +from dataclasses import dataclass, field, replace | |
| 30 | + | |
| 31 | +from aic.compiler import compile_document | |
| 32 | +from aic.diagnostics import CompilationError, Diagnostic, Severity | |
| 33 | +from alsl.model import PolicySet | |
| 34 | +from core.events import AirDocument, EconomicEvent | |
| 35 | +from core.invariant import verify_entries, verify_equation | |
| 36 | +from core.journal import CompiledJournal, JournalEntry, JournalLine, Side | |
| 37 | + | |
| 38 | + | |
| 39 | +def event_fingerprint(event: EconomicEvent) -> str: | |
| 40 | + """Content hash of an event (canonical JSON, field order fixed by the schema).""" | |
| 41 | + return hashlib.sha256(event.model_dump_json().encode("utf-8")).hexdigest() | |
| 42 | + | |
| 43 | + | |
| 44 | +@dataclass(frozen=True, slots=True) | |
| 45 | +class DocumentDiff: | |
| 46 | + added: tuple[str, ...] | |
| 47 | + removed: tuple[str, ...] | |
| 48 | + changed: tuple[str, ...] | |
| 49 | + unchanged: tuple[str, ...] | |
| 50 | + | |
| 51 | + def is_empty(self) -> bool: | |
| 52 | + return not (self.added or self.removed or self.changed) | |
| 53 | + | |
| 54 | + | |
| 55 | +def diff_documents(old: AirDocument, new: AirDocument) -> DocumentDiff: | |
| 56 | + old_fp = {e.id: event_fingerprint(e) for e in old.events} | |
| 57 | + new_fp = {e.id: event_fingerprint(e) for e in new.events} | |
| 58 | + added = tuple(i for i in new_fp if i not in old_fp) | |
| 59 | + removed = tuple(i for i in old_fp if i not in new_fp) | |
| 60 | + changed = tuple(i for i in new_fp if i in old_fp and new_fp[i] != old_fp[i]) | |
| 61 | + unchanged = tuple(i for i in new_fp if i in old_fp and new_fp[i] == old_fp[i]) | |
| 62 | + return DocumentDiff(added=added, removed=removed, changed=changed, | |
| 63 | + unchanged=unchanged) | |
| 64 | + | |
| 65 | + | |
| 66 | +def make_reversal(entry: JournalEntry) -> JournalEntry: | |
| 67 | + """Exact contra of a posted entry (sides swapped, amounts identical).""" | |
| 68 | + return JournalEntry( | |
| 69 | + id=f"rev_{entry.id}", | |
| 70 | + date=entry.date, | |
| 71 | + description=f"REVERSAL: {entry.description}", | |
| 72 | + lines=tuple( | |
| 73 | + JournalLine( | |
| 74 | + account=line.account, | |
| 75 | + side=Side.CREDIT if line.side is Side.DEBIT else Side.DEBIT, | |
| 76 | + amount=line.amount, | |
| 77 | + memo=f"reversal of {entry.id}: {line.memo}", | |
| 78 | + provenance_id=line.provenance_id, | |
| 79 | + ) | |
| 80 | + for line in entry.lines | |
| 81 | + ), | |
| 82 | + source_event_id=entry.source_event_id, | |
| 83 | + policy_set=entry.policy_set, | |
| 84 | + policy_version=entry.policy_version, | |
| 85 | + reverses=entry.id, | |
| 86 | + ) | |
| 87 | + | |
| 88 | + | |
| 89 | +@dataclass | |
| 90 | +class IncrementalResult: | |
| 91 | + """The delta between two compilations: what must be posted on top.""" | |
| 92 | + | |
| 93 | + diff: DocumentDiff | |
| 94 | + reversals: list[JournalEntry] = field(default_factory=list) | |
| 95 | + new_entries: list[JournalEntry] = field(default_factory=list) | |
| 96 | + journal: CompiledJournal = field(default_factory=CompiledJournal) # reversals + new | |
| 97 | + diagnostics: list[Diagnostic] = field(default_factory=list) | |
| 98 | + | |
| 99 | + | |
| 100 | +def recompile( | |
| 101 | + old: AirDocument, | |
| 102 | + new: AirDocument, | |
| 103 | + policies: PolicySet, | |
| 104 | +) -> IncrementalResult: | |
| 105 | + """Compile only the delta between two AIR documents. | |
| 106 | + | |
| 107 | + Both documents are compiled (compilation is cheap and deterministic; the | |
| 108 | + OLD compile reconstructs exactly what was posted, so no external state is | |
| 109 | + needed), but the returned journal contains ONLY the delta entries. | |
| 110 | + """ | |
| 111 | + old_journal, _ = compile_document(old, policies) | |
| 112 | + new_journal, new_diags = compile_document(new, policies) | |
| 113 | + diff = diff_documents(old, new) | |
| 114 | + | |
| 115 | + old_by_event = {e.source_event_id: e for e in old_journal.entries} | |
| 116 | + new_by_event = {e.source_event_id: e for e in new_journal.entries} | |
| 117 | + new_fp = {e.id: event_fingerprint(e) for e in new.events} | |
| 118 | + | |
| 119 | + result = IncrementalResult(diff=diff, diagnostics=list(new_diags)) | |
| 120 | + | |
| 121 | + for event_id in diff.removed + diff.changed: | |
| 122 | + original = old_by_event.get(event_id) | |
| 123 | + if original is not None: | |
| 124 | + result.reversals.append(make_reversal(original)) | |
| 125 | + | |
| 126 | + for event_id in diff.added: | |
| 127 | + entry = new_by_event.get(event_id) | |
| 128 | + if entry is not None: | |
| 129 | + result.new_entries.append(entry) | |
| 130 | + | |
| 131 | + for event_id in diff.changed: | |
| 132 | + entry = new_by_event.get(event_id) | |
| 133 | + if entry is not None: | |
| 134 | + # revision-suffixed id: deterministic, unique per content revision | |
| 135 | + revision = new_fp[event_id][:8] | |
| 136 | + result.new_entries.append(replace(entry, id=f"{entry.id}_r{revision}")) | |
| 137 | + | |
| 138 | + delta_entries = result.reversals + result.new_entries | |
| 139 | + result.journal = CompiledJournal( | |
| 140 | + entries=delta_entries, | |
| 141 | + provenance=new_journal.provenance, | |
| 142 | + policy_set=policies.name, | |
| 143 | + policy_version=policies.version, | |
| 144 | + ) | |
| 145 | + | |
| 146 | + # the permanent invariant applies to the delta as well | |
| 147 | + invariant = ( | |
| 148 | + verify_entries(delta_entries, "incremental") | |
| 149 | + + verify_equation(delta_entries, "incremental") | |
| 150 | + ) | |
| 151 | + result.diagnostics.extend(invariant) | |
| 152 | + if any(d.severity is Severity.ERROR for d in result.diagnostics): | |
| 153 | + raise CompilationError(result.diagnostics) | |
| 154 | + return result | |
added
aic/pass_manager.py
+35 −0
@@ -0,0 +1,35 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : pass_manager.py | |
| 6 | +# Description : Pass manager — ordered passes, invariant verified after every pass. | |
| 7 | +# ============================================================================= | |
| 8 | +"""AIC pass manager. | |
| 9 | + | |
| 10 | +Runs the pass pipeline in order. After EVERY pass, the double-entry invariant | |
| 11 | +(per-entry balance + accounting equation) is verified — the accounting | |
| 12 | +analogue of `opt -verify-each`. Any ERROR diagnostic aborts compilation with | |
| 13 | +a CompilationError carrying clang-style diagnostics. | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from aic.diagnostics import CompilationError, Severity | |
| 18 | +from aic.passes.base import Pass | |
| 19 | +from aic.unit import CompilationUnit | |
| 20 | +from core.invariant import verify_entries, verify_equation | |
| 21 | + | |
| 22 | + | |
| 23 | +class PassManager: | |
| 24 | + def __init__(self, passes: list[Pass]): | |
| 25 | + self.passes = passes | |
| 26 | + | |
| 27 | + def run(self, unit: CompilationUnit) -> CompilationUnit: | |
| 28 | + for p in self.passes: | |
| 29 | + unit.diagnostics.extend(p.run(unit)) | |
| 30 | + # the permanent invariant: checked after every pass, no exceptions | |
| 31 | + unit.diagnostics.extend(verify_entries(unit.entries, p.name)) | |
| 32 | + unit.diagnostics.extend(verify_equation(unit.entries, p.name)) | |
| 33 | + if any(d.severity is Severity.ERROR for d in unit.diagnostics): | |
| 34 | + raise CompilationError(unit.diagnostics) | |
| 35 | + return unit | |
added
aic/passes/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
aic/passes/base.py
+29 −0
@@ -0,0 +1,29 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : base.py | |
| 6 | +# Description : Pass interface — every AIC transformation implements this. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Base class for AIC passes (LLVM-pass analogue). | |
| 9 | + | |
| 10 | +A pass is a deterministic transformation over the CompilationUnit. It returns | |
| 11 | +diagnostics; it never calls an LLM, the network, or a clock. The pass manager | |
| 12 | +re-verifies the double-entry invariant after every pass. | |
| 13 | +""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +import abc | |
| 17 | + | |
| 18 | +from aic.diagnostics import Diagnostic | |
| 19 | +from aic.unit import CompilationUnit | |
| 20 | + | |
| 21 | + | |
| 22 | +class Pass(abc.ABC): | |
| 23 | + """One deterministic compilation step.""" | |
| 24 | + | |
| 25 | + name: str = "pass" | |
| 26 | + | |
| 27 | + @abc.abstractmethod | |
| 28 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 29 | + """Transform the unit in place; return any diagnostics produced.""" | |
added
aic/passes/classification.py
+55 −0
@@ -0,0 +1,55 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : classification.py | |
| 6 | +# Description : Classification pass — applies ALSL classification policies (e.g. capitalization). | |
| 7 | +# ============================================================================= | |
| 8 | +"""Classification pass. | |
| 9 | + | |
| 10 | +Decides which account role receives the debit side of a Purchase | |
| 11 | +(expense vs capitalized asset) by evaluating ALSL classification policies — | |
| 12 | +e.g. "capitalize equipment purchases at or above the policy threshold". | |
| 13 | +Thresholds live in ALSL, never here. | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from aic.diagnostics import Diagnostic, Severity | |
| 18 | +from aic.passes.base import Pass | |
| 19 | +from aic.unit import CompilationUnit | |
| 20 | +from alsl.evaluator import applicable_classifications | |
| 21 | +from core.events import EventType | |
| 22 | + | |
| 23 | + | |
| 24 | +class ClassificationPass(Pass): | |
| 25 | + name = "classification" | |
| 26 | + | |
| 27 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 28 | + diags: list[Diagnostic] = [] | |
| 29 | + for event in unit.document.events: | |
| 30 | + if event.type is not EventType.PURCHASE: | |
| 31 | + continue | |
| 32 | + state = unit.event_state(event.id) | |
| 33 | + if state.subtotal is None: | |
| 34 | + continue # validation already failed this event | |
| 35 | + | |
| 36 | + hits = applicable_classifications(unit.policies, event, state.subtotal) | |
| 37 | + if not hits: | |
| 38 | + continue | |
| 39 | + if len(hits) > 1: | |
| 40 | + diags.append(Diagnostic( | |
| 41 | + code="AIR-W300", severity=Severity.WARNING, | |
| 42 | + message=( | |
| 43 | + "multiple classification policies match: " | |
| 44 | + + ", ".join(p.name for p in hits) | |
| 45 | + + f"; applying '{hits[0].name}'" | |
| 46 | + ), | |
| 47 | + location=f"event {event.id}", | |
| 48 | + suggestion="tighten the policies' when-clauses so at most one matches", | |
| 49 | + origin_pass=self.name, | |
| 50 | + )) | |
| 51 | + policy = hits[0] | |
| 52 | + state.classify_as = policy.classify_as | |
| 53 | + state.classification_role = policy.account_role | |
| 54 | + state.classification_policy = policy.name | |
| 55 | + return diags | |
added
aic/passes/fx.py
+148 −0
@@ -0,0 +1,148 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : fx.py | |
| 6 | +# Description : FX pass — converts to the functional currency; computes settlement gains/losses. | |
| 7 | +# ============================================================================= | |
| 8 | +"""FX pass (IAS 21 semantics — see docs/research/accounting-standards.md and | |
| 9 | +docs/research/fx-handling.md). | |
| 10 | + | |
| 11 | +- Transactions in the functional currency pass through (quantized for posting). | |
| 12 | +- Foreign-currency transactions are converted at the event's observed rate | |
| 13 | + (a fact provided by ingestion, e.g. a Bank of Canada Valet daily rate — | |
| 14 | + never a constant in code). | |
| 15 | +- Foreign-currency settlements (payments) compare the settlement rate to the | |
| 16 | + booking rate of the related event and realize an FX gain or loss. | |
| 17 | +Every conversion is a provenance node: "fx:USD->CAD@1.3500". | |
| 18 | +""" | |
| 19 | +from __future__ import annotations | |
| 20 | + | |
| 21 | +from aic.diagnostics import Diagnostic, Severity | |
| 22 | +from aic.passes.base import Pass | |
| 23 | +from aic.unit import CompilationUnit, TaxLineState | |
| 24 | +from core.events import EconomicEvent, EventType | |
| 25 | +from core.money import Money | |
| 26 | + | |
| 27 | +SETTLEMENT_TYPES = (EventType.PAYMENT_RECEIVED, EventType.PAYMENT_SENT) | |
| 28 | + | |
| 29 | + | |
| 30 | +class FxPass(Pass): | |
| 31 | + name = "fx" | |
| 32 | + | |
| 33 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 34 | + diags: list[Diagnostic] = [] | |
| 35 | + functional = unit.policies.functional_currency | |
| 36 | + rounding = unit.policies.rounding.mode | |
| 37 | + | |
| 38 | + for event in unit.document.events: | |
| 39 | + state = unit.event_state(event.id) | |
| 40 | + if state.subtotal is None or state.subtotal_node is None: | |
| 41 | + continue | |
| 42 | + ccy = state.subtotal.currency | |
| 43 | + | |
| 44 | + if ccy == functional: | |
| 45 | + state.post_subtotal = state.subtotal.quantized(rounding) | |
| 46 | + state.post_subtotal_node = state.subtotal_node | |
| 47 | + state.post_taxes = list(state.taxes) | |
| 48 | + continue | |
| 49 | + | |
| 50 | + if event.fx is None: | |
| 51 | + diags.append(Diagnostic( | |
| 52 | + code="AIR-E500", severity=Severity.ERROR, | |
| 53 | + message=( | |
| 54 | + f"event is in {ccy} but functional currency is {functional} " | |
| 55 | + "and no fx rate is provided" | |
| 56 | + ), | |
| 57 | + location=f"event {event.id}, field fx", | |
| 58 | + suggestion="attach fx: {rate, source, rate_date} observed on the " | |
| 59 | + "transaction date (e.g. Bank of Canada Valet)", | |
| 60 | + origin_pass=self.name, | |
| 61 | + )) | |
| 62 | + continue | |
| 63 | + | |
| 64 | + rate = event.fx.rate | |
| 65 | + op = f"fx:{ccy}->{functional}@{rate}" | |
| 66 | + | |
| 67 | + converted = Money( | |
| 68 | + state.subtotal.amount * rate, functional | |
| 69 | + ).quantized(rounding) | |
| 70 | + node = unit.provenance.define( | |
| 71 | + kind="fx", operation=op, amount=converted, | |
| 72 | + inputs=(state.subtotal_node,), source_ref=event.fx.source, | |
| 73 | + ) | |
| 74 | + state.post_subtotal = converted | |
| 75 | + state.post_subtotal_node = node.id | |
| 76 | + | |
| 77 | + for tax in state.taxes: | |
| 78 | + tax_converted = Money( | |
| 79 | + tax.amount.amount * rate, functional | |
| 80 | + ).quantized(rounding) | |
| 81 | + tax_node = unit.provenance.define( | |
| 82 | + kind="fx", operation=op, amount=tax_converted, | |
| 83 | + inputs=(tax.node_id,), source_ref=event.fx.source, | |
| 84 | + ) | |
| 85 | + state.post_taxes.append(TaxLineState( | |
| 86 | + code=tax.code, amount=tax_converted, node_id=tax_node.id, | |
| 87 | + payable_role=tax.payable_role, | |
| 88 | + receivable_role=tax.receivable_role, | |
| 89 | + recoverable=tax.recoverable, policy=tax.policy, | |
| 90 | + )) | |
| 91 | + | |
| 92 | + if event.type in SETTLEMENT_TYPES: | |
| 93 | + diags.extend(self._settlement_diff(unit, event, rate, functional)) | |
| 94 | + return diags | |
| 95 | + | |
| 96 | + def _settlement_diff( | |
| 97 | + self, unit: CompilationUnit, event: EconomicEvent, | |
| 98 | + settle_rate: object, functional: str, | |
| 99 | + ) -> list[Diagnostic]: | |
| 100 | + """Realized FX gain/loss: settlement rate vs the related event's booking rate.""" | |
| 101 | + state = unit.event_state(event.id) | |
| 102 | + rounding = unit.policies.rounding.mode | |
| 103 | + | |
| 104 | + if event.related_event is None: | |
| 105 | + return [Diagnostic( | |
| 106 | + code="AIR-E501", severity=Severity.ERROR, | |
| 107 | + message="foreign-currency settlement without related_event: " | |
| 108 | + "the booking rate cannot be determined", | |
| 109 | + location=f"event {event.id}, field related_event", | |
| 110 | + suggestion="reference the Sale/Purchase this payment settles", | |
| 111 | + origin_pass=self.name, | |
| 112 | + )] | |
| 113 | + origin = unit.find_event(event.related_event) | |
| 114 | + if origin is None or origin.fx is None: | |
| 115 | + return [Diagnostic( | |
| 116 | + code="AIR-E502", severity=Severity.ERROR, | |
| 117 | + message=( | |
| 118 | + f"related event '{event.related_event}' not found in this " | |
| 119 | + "document or carries no fx rate" | |
| 120 | + ), | |
| 121 | + location=f"event {event.id}, field related_event", | |
| 122 | + suggestion="compile the settlement together with its origin event, " | |
| 123 | + "or attach the origin's booking rate", | |
| 124 | + origin_pass=self.name, | |
| 125 | + )] | |
| 126 | + | |
| 127 | + assert state.subtotal is not None and state.subtotal_node is not None | |
| 128 | + booked = Money( | |
| 129 | + state.subtotal.amount * origin.fx.rate, functional | |
| 130 | + ).quantized(rounding) | |
| 131 | + settled = state.post_subtotal | |
| 132 | + assert settled is not None and state.post_subtotal_node is not None | |
| 133 | + diff = settled - booked | |
| 134 | + state.booked_amount = booked | |
| 135 | + state.fx_diff = diff | |
| 136 | + if not diff.is_zero(): | |
| 137 | + node = unit.provenance.define( | |
| 138 | + kind="fx_realized", | |
| 139 | + operation=( | |
| 140 | + f"fx_realized:{state.subtotal.currency}->{functional}" | |
| 141 | + f"@{settle_rate}-vs-@{origin.fx.rate}" | |
| 142 | + ), | |
| 143 | + amount=diff, | |
| 144 | + inputs=(state.post_subtotal_node,), | |
| 145 | + source_ref=event.related_event, | |
| 146 | + ) | |
| 147 | + state.fx_diff_node = node.id | |
| 148 | + return [] | |
added
aic/passes/optimize.py
+222 −0
@@ -0,0 +1,222 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : optimize.py | |
| 6 | +# Description : Optimization passes — duplicate detection, netting, fusion (Phase 6). | |
| 7 | +# ============================================================================= | |
| 8 | +"""Optimization passes (the LLVM optimization-pass analogue). | |
| 9 | + | |
| 10 | +Opt-in (compile with optimize=True); the default pipeline is untouched. | |
| 11 | +All three preserve the double-entry invariant, which the pass manager | |
| 12 | +re-verifies after each of them, and full provenance: every optimized line's | |
| 13 | +provenance node points back to the lines it absorbed. | |
| 14 | + | |
| 15 | +- DuplicateDetectionPass — events with identical economic content but | |
| 16 | + different ids get a warning (AIR-W800); a human decides, nothing is dropped. | |
| 17 | +- NettingPass — a Refund referencing a Sale compiled in the same batch is | |
| 18 | + netted against it: one net entry (or none, when fully offset) replaces both. | |
| 19 | +- FusionPass — N same-day payments with the same posting shape become one | |
| 20 | + batch entry (50 identical payouts -> 1 entry, amounts summed per line). | |
| 21 | +""" | |
| 22 | +from __future__ import annotations | |
| 23 | + | |
| 24 | +import hashlib | |
| 25 | +import json | |
| 26 | +from collections import defaultdict | |
| 27 | +from decimal import Decimal | |
| 28 | + | |
| 29 | +from aic.diagnostics import Diagnostic, Severity | |
| 30 | +from aic.passes.base import Pass | |
| 31 | +from aic.unit import CompilationUnit | |
| 32 | +from core.events import EconomicEvent, EventType | |
| 33 | +from core.journal import JournalEntry, JournalLine, Side | |
| 34 | +from core.money import Money | |
| 35 | + | |
| 36 | + | |
| 37 | +def _event_fingerprint(event: EconomicEvent) -> str: | |
| 38 | + """Economic-content hash: identical business facts, ignoring id and meta.""" | |
| 39 | + payload = json.loads(event.model_dump_json(exclude_none=True)) | |
| 40 | + payload.pop("id", None) | |
| 41 | + payload.pop("meta", None) | |
| 42 | + return hashlib.sha256( | |
| 43 | + json.dumps(payload, sort_keys=True).encode("utf-8")).hexdigest() | |
| 44 | + | |
| 45 | + | |
| 46 | +class DuplicateDetectionPass(Pass): | |
| 47 | + name = "duplicate-detection" | |
| 48 | + | |
| 49 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 50 | + diags: list[Diagnostic] = [] | |
| 51 | + by_content: dict[str, list[str]] = defaultdict(list) | |
| 52 | + for event in unit.document.events: | |
| 53 | + by_content[_event_fingerprint(event)].append(event.id) | |
| 54 | + for ids in by_content.values(): | |
| 55 | + if len(ids) > 1: | |
| 56 | + diags.append(Diagnostic( | |
| 57 | + code="AIR-W800", severity=Severity.WARNING, | |
| 58 | + message=( | |
| 59 | + f"{len(ids)} events carry identical economic content: " | |
| 60 | + + ", ".join(ids) | |
| 61 | + ), | |
| 62 | + location=f"events {ids[0]} .. {ids[-1]}", | |
| 63 | + suggestion="if these are duplicates, remove all but one and " | |
| 64 | + "recompile; if legitimate repeats, add a " | |
| 65 | + "distinguishing description", | |
| 66 | + origin_pass=self.name, | |
| 67 | + )) | |
| 68 | + return diags | |
| 69 | + | |
| 70 | + | |
| 71 | +class NettingPass(Pass): | |
| 72 | + """Net Refund entries against the Sale they reverse (same compilation).""" | |
| 73 | + | |
| 74 | + name = "netting" | |
| 75 | + | |
| 76 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 77 | + diags: list[Diagnostic] = [] | |
| 78 | + by_event = {e.source_event_id: e for e in unit.entries} | |
| 79 | + | |
| 80 | + for event in unit.document.events: | |
| 81 | + if event.type is not EventType.REFUND or not event.related_event: | |
| 82 | + continue | |
| 83 | + refund_entry = by_event.get(event.id) | |
| 84 | + sale_entry = by_event.get(event.related_event) | |
| 85 | + if refund_entry is None or sale_entry is None: | |
| 86 | + continue | |
| 87 | + | |
| 88 | + # per-(account, sale side): refund lines mirror sale lines, so a | |
| 89 | + # refund debit offsets the sale's credit on the same account | |
| 90 | + sale_amts: dict[tuple[str, Side], Money] = { | |
| 91 | + (l.account.code, l.side): l.amount for l in sale_entry.lines} | |
| 92 | + refund_amts: dict[tuple[str, Side], Money] = {} | |
| 93 | + refund_prov: dict[tuple[str, Side], str | None] = {} | |
| 94 | + for l in refund_entry.lines: | |
| 95 | + mirrored = Side.CREDIT if l.side is Side.DEBIT else Side.DEBIT | |
| 96 | + refund_amts[(l.account.code, mirrored)] = l.amount | |
| 97 | + refund_prov[(l.account.code, mirrored)] = l.provenance_id | |
| 98 | + if set(refund_amts) - set(sale_amts): | |
| 99 | + continue # shapes differ; not safely nettable | |
| 100 | + if any((sale_amts[k] - refund_amts[k]).is_negative() | |
| 101 | + for k in refund_amts): | |
| 102 | + continue # refund exceeds sale; leave both entries alone | |
| 103 | + | |
| 104 | + net_lines: list[JournalLine] = [] | |
| 105 | + for sale_line in sale_entry.lines: | |
| 106 | + key = (sale_line.account.code, sale_line.side) | |
| 107 | + refunded = refund_amts.get(key) | |
| 108 | + net = (sale_line.amount - refunded) if refunded else sale_line.amount | |
| 109 | + if net.is_zero(): | |
| 110 | + continue | |
| 111 | + inputs = tuple(p for p in (sale_line.provenance_id, | |
| 112 | + refund_prov.get(key)) if p) | |
| 113 | + node = unit.provenance.define( | |
| 114 | + kind="netting", | |
| 115 | + operation=f"net:{event.related_event}-{event.id}", | |
| 116 | + amount=net, | |
| 117 | + inputs=inputs, | |
| 118 | + ) | |
| 119 | + net_lines.append(JournalLine( | |
| 120 | + account=sale_line.account, side=sale_line.side, amount=net, | |
| 121 | + memo=f"net of {sale_entry.id} minus {refund_entry.id}", | |
| 122 | + provenance_id=node.id, | |
| 123 | + )) | |
| 124 | + | |
| 125 | + unit.entries.remove(sale_entry) | |
| 126 | + unit.entries.remove(refund_entry) | |
| 127 | + if net_lines: | |
| 128 | + unit.entries.append(JournalEntry( | |
| 129 | + id=f"je_net_{event.related_event}", | |
| 130 | + date=refund_entry.date, | |
| 131 | + description=(f"Netting: {sale_entry.description} " | |
| 132 | + f"minus {refund_entry.description}"), | |
| 133 | + lines=tuple(net_lines), | |
| 134 | + source_event_id=f"net({event.related_event},{event.id})", | |
| 135 | + policy_set=sale_entry.policy_set, | |
| 136 | + policy_version=sale_entry.policy_version, | |
| 137 | + )) | |
| 138 | + detail = "netted into one entry" | |
| 139 | + else: | |
| 140 | + detail = "fully offset: no entry posted" | |
| 141 | + diags.append(Diagnostic( | |
| 142 | + code="AIR-N801", severity=Severity.NOTE, | |
| 143 | + message=(f"refund '{event.id}' netted against sale " | |
| 144 | + f"'{event.related_event}' ({detail})"), | |
| 145 | + location=f"events {event.related_event}, {event.id}", | |
| 146 | + origin_pass=self.name, | |
| 147 | + )) | |
| 148 | + return diags | |
| 149 | + | |
| 150 | + | |
| 151 | +class FusionPass(Pass): | |
| 152 | + """Fuse same-day payments with identical posting shape into one batch.""" | |
| 153 | + | |
| 154 | + name = "fusion" | |
| 155 | + FUSABLE = (EventType.PAYMENT_RECEIVED, EventType.PAYMENT_SENT) | |
| 156 | + | |
| 157 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 158 | + diags: list[Diagnostic] = [] | |
| 159 | + events_by_id = {e.id: e for e in unit.document.events} | |
| 160 | + | |
| 161 | + groups: dict[tuple, list[JournalEntry]] = defaultdict(list) | |
| 162 | + for entry in unit.entries: | |
| 163 | + event = events_by_id.get(entry.source_event_id) | |
| 164 | + if event is None or event.type not in self.FUSABLE: | |
| 165 | + continue | |
| 166 | + shape = tuple(sorted( | |
| 167 | + (l.account.code, l.side.value, l.amount.currency) | |
| 168 | + for l in entry.lines)) | |
| 169 | + groups[(event.type, entry.date, shape)].append(entry) | |
| 170 | + | |
| 171 | + for (etype, date, shape), entries in groups.items(): | |
| 172 | + if len(entries) < 2: | |
| 173 | + continue | |
| 174 | + summed: dict[tuple[str, str, str], Decimal] = defaultdict(Decimal) | |
| 175 | + accounts = {} | |
| 176 | + inputs: dict[tuple[str, str, str], list[str]] = defaultdict(list) | |
| 177 | + for entry in entries: | |
| 178 | + for line in entry.lines: | |
| 179 | + key = (line.account.code, line.side.value, | |
| 180 | + line.amount.currency) | |
| 181 | + summed[key] += line.amount.amount | |
| 182 | + accounts[line.account.code] = line.account | |
| 183 | + if line.provenance_id: | |
| 184 | + inputs[key].append(line.provenance_id) | |
| 185 | + | |
| 186 | + batch_hash = hashlib.sha256( | |
| 187 | + ",".join(sorted(e.id for e in entries)).encode("utf-8") | |
| 188 | + ).hexdigest()[:8] | |
| 189 | + lines = [] | |
| 190 | + for (code, side, ccy), amount in sorted(summed.items()): | |
| 191 | + money = Money(amount, ccy) | |
| 192 | + node = unit.provenance.define( | |
| 193 | + kind="fusion", | |
| 194 | + operation=f"fuse:{len(entries)}x{etype.value}", | |
| 195 | + amount=money, | |
| 196 | + inputs=tuple(inputs[(code, side, ccy)]), | |
| 197 | + ) | |
| 198 | + lines.append(JournalLine( | |
| 199 | + account=accounts[code], side=Side(side), amount=money, | |
| 200 | + memo=f"batch of {len(entries)} {etype.value} entries", | |
| 201 | + provenance_id=node.id, | |
| 202 | + )) | |
| 203 | + for entry in entries: | |
| 204 | + unit.entries.remove(entry) | |
| 205 | + unit.entries.append(JournalEntry( | |
| 206 | + id=f"je_batch_{batch_hash}", | |
| 207 | + date=date, | |
| 208 | + description=f"Batch: {len(entries)} x {etype.value}", | |
| 209 | + lines=tuple(lines), | |
| 210 | + source_event_id=( | |
| 211 | + "batch(" + ",".join(e.source_event_id for e in entries) + ")"), | |
| 212 | + policy_set=entries[0].policy_set, | |
| 213 | + policy_version=entries[0].policy_version, | |
| 214 | + )) | |
| 215 | + diags.append(Diagnostic( | |
| 216 | + code="AIR-N800", severity=Severity.NOTE, | |
| 217 | + message=(f"fused {len(entries)} {etype.value} entries dated " | |
| 218 | + f"{date} into one batch entry (je_batch_{batch_hash})"), | |
| 219 | + location=f"entries {', '.join(e.id for e in entries)}", | |
| 220 | + origin_pass=self.name, | |
| 221 | + )) | |
| 222 | + return diags | |
added
aic/passes/posting.py
+241 −0
@@ -0,0 +1,241 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : posting.py | |
| 6 | +# Description : Posting pass — lowers economic events into balanced journal entries. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Posting pass: the lowering stage (AIR -> journal entries). | |
| 9 | + | |
| 10 | +Maps each economic event to a balanced double-entry journal entry using the | |
| 11 | +account roles defined in the ALSL policy set. Every line carries a provenance | |
| 12 | +node so any posted figure traces back to its origin. | |
| 13 | + | |
| 14 | +Posting rules (v0.1): | |
| 15 | +- Sale DR cash/AR (total) CR revenue, CR tax payables | |
| 16 | +- Refund DR revenue, DR taxes CR cash/AR (mirror of Sale) | |
| 17 | +- Purchase DR expense/asset (+non-recoverable tax), DR recoverable taxes | |
| 18 | + CR cash/AP | |
| 19 | +- PaymentReceived DR cash (settled) CR AR (booked) [± realized FX] | |
| 20 | +- PaymentSent DR AP (booked) CR cash (settled) [± realized FX] | |
| 21 | +- OwnerContribution DR cash CR owner capital | |
| 22 | +- LoanReceived DR cash CR loan payable | |
| 23 | +""" | |
| 24 | +from __future__ import annotations | |
| 25 | + | |
| 26 | +from aic.diagnostics import Diagnostic, Severity | |
| 27 | +from aic.passes.base import Pass | |
| 28 | +from aic.unit import CompilationUnit, EventState | |
| 29 | +from core.events import EconomicEvent, EventType | |
| 30 | +from core.journal import Account, JournalEntry, JournalLine, Side | |
| 31 | +from core.money import Money | |
| 32 | + | |
| 33 | + | |
| 34 | +class PostingPass(Pass): | |
| 35 | + name = "posting" | |
| 36 | + | |
| 37 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 38 | + diags: list[Diagnostic] = [] | |
| 39 | + for event in unit.document.events: | |
| 40 | + state = unit.event_state(event.id) | |
| 41 | + if state.post_subtotal is None: | |
| 42 | + continue # a previous pass already rejected this event | |
| 43 | + try: | |
| 44 | + lines = self._lower(unit, event, state, diags) | |
| 45 | + except KeyError as exc: | |
| 46 | + diags.append(Diagnostic( | |
| 47 | + code="AIR-E600", severity=Severity.ERROR, | |
| 48 | + message=str(exc.args[0]), | |
| 49 | + location=f"event {event.id}", | |
| 50 | + suggestion="add the missing account role to the policy set's " | |
| 51 | + "accounts section", | |
| 52 | + origin_pass=self.name, | |
| 53 | + )) | |
| 54 | + continue | |
| 55 | + if not lines: | |
| 56 | + continue | |
| 57 | + unit.entries.append(JournalEntry( | |
| 58 | + id=f"je_{event.id}", | |
| 59 | + date=event.date, | |
| 60 | + description=event.description or event.type.value, | |
| 61 | + lines=tuple(lines), | |
| 62 | + source_event_id=event.id, | |
| 63 | + policy_set=unit.policies.name, | |
| 64 | + policy_version=unit.policies.version, | |
| 65 | + reverses=( | |
| 66 | + f"je_{event.related_event}" | |
| 67 | + if event.type is EventType.REFUND and event.related_event | |
| 68 | + else None | |
| 69 | + ), | |
| 70 | + )) | |
| 71 | + return diags | |
| 72 | + | |
| 73 | + # -- helpers --------------------------------------------------------------- | |
| 74 | + def _line( | |
| 75 | + self, unit: CompilationUnit, account: Account, side: Side, | |
| 76 | + amount: Money, memo: str, parent_node: str | None, | |
| 77 | + ) -> JournalLine: | |
| 78 | + node = unit.provenance.define( | |
| 79 | + kind="journal_line", | |
| 80 | + operation=f"post:{side.value}:{account.code}", | |
| 81 | + amount=amount, | |
| 82 | + inputs=(parent_node,) if parent_node else (), | |
| 83 | + ) | |
| 84 | + return JournalLine( | |
| 85 | + account=account, side=side, amount=amount, | |
| 86 | + memo=memo, provenance_id=node.id, | |
| 87 | + ) | |
| 88 | + | |
| 89 | + def _settlement_account(self, unit: CompilationUnit, event: EconomicEvent, | |
| 90 | + counterparty_role: str) -> Account: | |
| 91 | + immediate = bool(event.payment and event.payment.immediate) | |
| 92 | + return unit.policies.account("cash" if immediate else counterparty_role) | |
| 93 | + | |
| 94 | + def _check_gross(self, event: EconomicEvent, computed_total: Money, | |
| 95 | + diags: list[Diagnostic], pass_name: str) -> None: | |
| 96 | + """Warn when the document's stated gross disagrees with the computed total.""" | |
| 97 | + if event.payment is None or event.payment.gross is None: | |
| 98 | + return | |
| 99 | + gross = event.payment.gross.to_money() | |
| 100 | + if gross.currency != computed_total.currency: | |
| 101 | + return # gross stated in transaction currency, total in functional | |
| 102 | + if gross.amount != computed_total.amount: | |
| 103 | + diags.append(Diagnostic( | |
| 104 | + code="AIR-W600", severity=Severity.WARNING, | |
| 105 | + message=( | |
| 106 | + f"stated gross {gross} differs from computed total " | |
| 107 | + f"{computed_total}; the computed total is authoritative" | |
| 108 | + ), | |
| 109 | + location=f"event {event.id}, field payment.gross", | |
| 110 | + suggestion="check the source document; a small delta usually means " | |
| 111 | + "the issuer rounded differently", | |
| 112 | + origin_pass=pass_name, | |
| 113 | + )) | |
| 114 | + | |
| 115 | + # -- lowering per event type ------------------------------------------------- | |
| 116 | + def _lower( | |
| 117 | + self, unit: CompilationUnit, event: EconomicEvent, | |
| 118 | + state: EventState, diags: list[Diagnostic], | |
| 119 | + ) -> list[JournalLine]: | |
| 120 | + pol = unit.policies | |
| 121 | + assert state.post_subtotal is not None | |
| 122 | + subtotal = state.post_subtotal | |
| 123 | + sub_node = state.post_subtotal_node | |
| 124 | + taxes = state.post_taxes | |
| 125 | + total = subtotal | |
| 126 | + for t in taxes: | |
| 127 | + total = total + t.amount | |
| 128 | + | |
| 129 | + if event.type is EventType.SALE: | |
| 130 | + self._check_gross(event, total, diags, self.name) | |
| 131 | + counter = self._settlement_account(unit, event, "accounts_receivable") | |
| 132 | + lines = [self._line(unit, counter, Side.DEBIT, total, | |
| 133 | + "sale: consideration incl. taxes", sub_node)] | |
| 134 | + lines.append(self._line(unit, pol.account("revenue"), Side.CREDIT, | |
| 135 | + subtotal, "sale: revenue", sub_node)) | |
| 136 | + for t in taxes: | |
| 137 | + lines.append(self._line( | |
| 138 | + unit, pol.account(t.payable_role), Side.CREDIT, t.amount, | |
| 139 | + f"{t.code} collected ({t.policy})", t.node_id)) | |
| 140 | + return lines | |
| 141 | + | |
| 142 | + if event.type is EventType.REFUND: | |
| 143 | + counter = self._settlement_account(unit, event, "accounts_receivable") | |
| 144 | + lines = [self._line(unit, pol.account("revenue"), Side.DEBIT, | |
| 145 | + subtotal, "refund: revenue reversal", sub_node)] | |
| 146 | + for t in taxes: | |
| 147 | + lines.append(self._line( | |
| 148 | + unit, pol.account(t.payable_role), Side.DEBIT, t.amount, | |
| 149 | + f"{t.code} refunded ({t.policy})", t.node_id)) | |
| 150 | + lines.append(self._line(unit, counter, Side.CREDIT, total, | |
| 151 | + "refund: consideration incl. taxes", sub_node)) | |
| 152 | + return lines | |
| 153 | + | |
| 154 | + if event.type is EventType.PURCHASE: | |
| 155 | + debit_role = state.classification_role or "expense_default" | |
| 156 | + debit_base = subtotal | |
| 157 | + recoverable = [t for t in taxes if t.recoverable] | |
| 158 | + for t in taxes: | |
| 159 | + if not t.recoverable: | |
| 160 | + debit_base = debit_base + t.amount # non-recoverable tax is a cost | |
| 161 | + memo = ( | |
| 162 | + f"purchase: capitalized ({state.classification_policy})" | |
| 163 | + if state.classify_as == "asset" else "purchase: cost" | |
| 164 | + ) | |
| 165 | + lines = [self._line(unit, pol.account(debit_role), Side.DEBIT, | |
| 166 | + debit_base, memo, sub_node)] | |
| 167 | + for t in recoverable: | |
| 168 | + lines.append(self._line( | |
| 169 | + unit, pol.account(t.receivable_role), Side.DEBIT, t.amount, | |
| 170 | + f"{t.code} recoverable ({t.policy})", t.node_id)) | |
| 171 | + counter = self._settlement_account(unit, event, "accounts_payable") | |
| 172 | + lines.append(self._line(unit, counter, Side.CREDIT, total, | |
| 173 | + "purchase: consideration incl. taxes", sub_node)) | |
| 174 | + self._check_gross(event, total, diags, self.name) | |
| 175 | + return lines | |
| 176 | + | |
| 177 | + if event.type is EventType.PAYMENT_RECEIVED: | |
| 178 | + settled = subtotal | |
| 179 | + booked = state.booked_amount or settled | |
| 180 | + lines = [self._line(unit, pol.account("cash"), Side.DEBIT, settled, | |
| 181 | + "payment received", sub_node)] | |
| 182 | + lines.append(self._line( | |
| 183 | + unit, pol.account("accounts_receivable"), Side.CREDIT, booked, | |
| 184 | + f"settles {event.related_event or 'receivable'}", sub_node)) | |
| 185 | + diff = state.fx_diff | |
| 186 | + if diff is not None and not diff.is_zero(): | |
| 187 | + if diff.is_negative(): | |
| 188 | + lines.append(self._line(unit, pol.account("fx_loss"), | |
| 189 | + Side.DEBIT, -diff, | |
| 190 | + "realized FX loss", state.fx_diff_node)) | |
| 191 | + else: | |
| 192 | + lines.append(self._line(unit, pol.account("fx_gain"), | |
| 193 | + Side.CREDIT, diff, | |
| 194 | + "realized FX gain", state.fx_diff_node)) | |
| 195 | + return lines | |
| 196 | + | |
| 197 | + if event.type is EventType.PAYMENT_SENT: | |
| 198 | + settled = subtotal | |
| 199 | + booked = state.booked_amount or settled | |
| 200 | + lines = [self._line( | |
| 201 | + unit, pol.account("accounts_payable"), Side.DEBIT, booked, | |
| 202 | + f"settles {event.related_event or 'payable'}", sub_node)] | |
| 203 | + diff = state.fx_diff | |
| 204 | + if diff is not None and not diff.is_zero(): | |
| 205 | + if diff.is_negative(): | |
| 206 | + # paying fewer functional units than booked -> gain | |
| 207 | + lines.append(self._line(unit, pol.account("fx_gain"), | |
| 208 | + Side.CREDIT, -diff, | |
| 209 | + "realized FX gain", state.fx_diff_node)) | |
| 210 | + else: | |
| 211 | + lines.append(self._line(unit, pol.account("fx_loss"), | |
| 212 | + Side.DEBIT, diff, | |
| 213 | + "realized FX loss", state.fx_diff_node)) | |
| 214 | + lines.append(self._line(unit, pol.account("cash"), Side.CREDIT, settled, | |
| 215 | + "payment sent", sub_node)) | |
| 216 | + return lines | |
| 217 | + | |
| 218 | + if event.type is EventType.OWNER_CONTRIBUTION: | |
| 219 | + return [ | |
| 220 | + self._line(unit, pol.account("cash"), Side.DEBIT, subtotal, | |
| 221 | + "owner contribution", sub_node), | |
| 222 | + self._line(unit, pol.account("owner_capital"), Side.CREDIT, subtotal, | |
| 223 | + "owner contribution", sub_node), | |
| 224 | + ] | |
| 225 | + | |
| 226 | + if event.type is EventType.LOAN_RECEIVED: | |
| 227 | + return [ | |
| 228 | + self._line(unit, pol.account("cash"), Side.DEBIT, subtotal, | |
| 229 | + "loan proceeds", sub_node), | |
| 230 | + self._line(unit, pol.account("loan_payable"), Side.CREDIT, subtotal, | |
| 231 | + "loan principal", sub_node), | |
| 232 | + ] | |
| 233 | + | |
| 234 | + diags.append(Diagnostic( | |
| 235 | + code="AIR-E601", severity=Severity.ERROR, | |
| 236 | + message=f"no posting rule for event type '{event.type.value}'", | |
| 237 | + location=f"event {event.id}", | |
| 238 | + suggestion="extend the posting pass or use a supported event type", | |
| 239 | + origin_pass=self.name, | |
| 240 | + )) | |
| 241 | + return [] | |
added
aic/passes/tax.py
+105 −0
@@ -0,0 +1,105 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : tax.py | |
| 6 | +# Description : Tax pass — applies ALSL tax policies deterministically (no rates in code). | |
| 7 | +# ============================================================================= | |
| 8 | +"""Tax pass. | |
| 9 | + | |
| 10 | +For each taxable event, evaluates the ALSL tax policies matching the event's | |
| 11 | +jurisdiction and computes each tax component on its base with the policy | |
| 12 | +set's rounding mode. Every computed tax amount becomes a provenance node | |
| 13 | +derived from the event subtotal. | |
| 14 | + | |
| 15 | +NO tax rate exists in this file — rates live in versioned ALSL policy sets | |
| 16 | +with mandatory source citations (see docs/research/canada-gst-qst.md). | |
| 17 | +""" | |
| 18 | +from __future__ import annotations | |
| 19 | + | |
| 20 | +from aic.diagnostics import Diagnostic, Severity | |
| 21 | +from aic.passes.base import Pass | |
| 22 | +from aic.unit import CompilationUnit, TaxLineState | |
| 23 | +from alsl.evaluator import applicable_tax_policies | |
| 24 | +from core.events import EventType | |
| 25 | + | |
| 26 | +TAXABLE_TYPES = (EventType.SALE, EventType.PURCHASE, EventType.REFUND) | |
| 27 | + | |
| 28 | + | |
| 29 | +class TaxPass(Pass): | |
| 30 | + name = "tax" | |
| 31 | + | |
| 32 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 33 | + diags: list[Diagnostic] = [] | |
| 34 | + rounding = unit.policies.rounding.mode | |
| 35 | + | |
| 36 | + for event in unit.document.events: | |
| 37 | + if event.type not in TAXABLE_TYPES: | |
| 38 | + continue | |
| 39 | + if event.tax is None or event.tax.exempt: | |
| 40 | + continue | |
| 41 | + state = unit.event_state(event.id) | |
| 42 | + if state.subtotal is None or state.subtotal_node is None: | |
| 43 | + continue # validation already failed this event | |
| 44 | + | |
| 45 | + policies = applicable_tax_policies(unit.policies, event, state.subtotal) | |
| 46 | + if not policies: | |
| 47 | + diags.append(Diagnostic( | |
| 48 | + code="AIR-E400", severity=Severity.ERROR, | |
| 49 | + message=( | |
| 50 | + f"no tax policy in set '{unit.policies.name}' matches " | |
| 51 | + f"jurisdiction '{event.tax.jurisdiction}'" | |
| 52 | + ), | |
| 53 | + location=f"event {event.id}, field tax.jurisdiction", | |
| 54 | + suggestion="add an ALSL tax policy for this jurisdiction " | |
| 55 | + "or mark the event tax.exempt: true", | |
| 56 | + origin_pass=self.name, | |
| 57 | + )) | |
| 58 | + continue | |
| 59 | + | |
| 60 | + wanted = set(event.tax.codes) # optional explicit filter | |
| 61 | + produced: set[str] = set() | |
| 62 | + for policy in policies: | |
| 63 | + for comp in policy.components: | |
| 64 | + if wanted and comp.code not in wanted: | |
| 65 | + continue | |
| 66 | + if comp.base != "subtotal": | |
| 67 | + diags.append(Diagnostic( | |
| 68 | + code="AIR-E401", severity=Severity.ERROR, | |
| 69 | + message=f"unsupported tax base '{comp.base}' in policy " | |
| 70 | + f"'{policy.name}' (ALSL v0.1 supports 'subtotal')", | |
| 71 | + location=f"policy {policy.name}, component {comp.code}", | |
| 72 | + suggestion="use base: subtotal", | |
| 73 | + origin_pass=self.name, | |
| 74 | + )) | |
| 75 | + continue | |
| 76 | + raw = state.subtotal.multiply(comp.rate) | |
| 77 | + amount = raw.quantized(rounding) | |
| 78 | + node = unit.provenance.define( | |
| 79 | + kind="tax", | |
| 80 | + operation=f"tax:{comp.code}@{comp.rate}~{rounding.value}", | |
| 81 | + amount=amount, | |
| 82 | + inputs=(state.subtotal_node,), | |
| 83 | + source_ref=policy.source, | |
| 84 | + ) | |
| 85 | + state.taxes.append(TaxLineState( | |
| 86 | + code=comp.code, | |
| 87 | + amount=amount, | |
| 88 | + node_id=node.id, | |
| 89 | + payable_role=comp.payable_role, | |
| 90 | + receivable_role=comp.receivable_role, | |
| 91 | + recoverable=comp.recoverable_on_purchase, | |
| 92 | + policy=policy.name, | |
| 93 | + )) | |
| 94 | + produced.add(comp.code) | |
| 95 | + | |
| 96 | + for missing in wanted - produced: | |
| 97 | + diags.append(Diagnostic( | |
| 98 | + code="AIR-W400", severity=Severity.WARNING, | |
| 99 | + message=f"event requested tax code '{missing}' but no policy " | |
| 100 | + "produced it", | |
| 101 | + location=f"event {event.id}, field tax.codes", | |
| 102 | + suggestion="check the policy set covers this code for the jurisdiction", | |
| 103 | + origin_pass=self.name, | |
| 104 | + )) | |
| 105 | + return diags | |
added
aic/passes/validation.py
+109 −0
@@ -0,0 +1,109 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : validation.py | |
| 6 | +# Description : Validation pass — structural checks + subtotal provenance definitions. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Validation pass: the front door of the compiler. | |
| 9 | + | |
| 10 | +Checks structural well-formedness of the AIR document and defines each | |
| 11 | +event's subtotal in the provenance graph (the SSA origin every later | |
| 12 | +derivation points back to). | |
| 13 | +""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +from aic.diagnostics import Diagnostic, Severity | |
| 17 | +from aic.passes.base import Pass | |
| 18 | +from aic.unit import CompilationUnit | |
| 19 | +from core.events import EventType | |
| 20 | + | |
| 21 | +TAXABLE_TYPES = (EventType.SALE, EventType.PURCHASE, EventType.REFUND) | |
| 22 | + | |
| 23 | + | |
| 24 | +class ValidationPass(Pass): | |
| 25 | + name = "validation" | |
| 26 | + | |
| 27 | + def run(self, unit: CompilationUnit) -> list[Diagnostic]: | |
| 28 | + diags: list[Diagnostic] = [] | |
| 29 | + seen: set[str] = set() | |
| 30 | + | |
| 31 | + for event in unit.document.events: | |
| 32 | + loc = f"event {event.id}" | |
| 33 | + | |
| 34 | + if event.id in seen: | |
| 35 | + diags.append(Diagnostic( | |
| 36 | + code="AIR-E200", severity=Severity.ERROR, | |
| 37 | + message=f"duplicate event id '{event.id}'", | |
| 38 | + location=loc, | |
| 39 | + suggestion="event ids must be unique within a document (use ULIDs)", | |
| 40 | + origin_pass=self.name, | |
| 41 | + )) | |
| 42 | + continue | |
| 43 | + seen.add(event.id) | |
| 44 | + | |
| 45 | + # single transaction currency per event | |
| 46 | + currencies = {i.unit_price.currency for i in event.items} | |
| 47 | + if event.amount is not None: | |
| 48 | + currencies.add(event.amount.currency) | |
| 49 | + if len(currencies) > 1: | |
| 50 | + diags.append(Diagnostic( | |
| 51 | + code="AIR-E201", severity=Severity.ERROR, | |
| 52 | + message=f"mixed currencies in one event: {sorted(currencies)}", | |
| 53 | + location=f"{loc}, field items[].unit_price", | |
| 54 | + suggestion="split into one event per currency", | |
| 55 | + origin_pass=self.name, | |
| 56 | + )) | |
| 57 | + continue | |
| 58 | + | |
| 59 | + subtotal = event.subtotal() | |
| 60 | + if subtotal is None: | |
| 61 | + diags.append(Diagnostic( | |
| 62 | + code="AIR-E202", severity=Severity.ERROR, | |
| 63 | + message="event has no amount: neither items nor a flat amount", | |
| 64 | + location=f"{loc}, fields items / amount", | |
| 65 | + suggestion="provide items[] with unit prices, or a flat amount", | |
| 66 | + origin_pass=self.name, | |
| 67 | + )) | |
| 68 | + continue | |
| 69 | + if subtotal.is_negative(): | |
| 70 | + diags.append(Diagnostic( | |
| 71 | + code="AIR-E203", severity=Severity.ERROR, | |
| 72 | + message=f"negative subtotal {subtotal}", | |
| 73 | + location=f"{loc}, field items/amount", | |
| 74 | + suggestion="amounts are positive; direction comes from the event type " | |
| 75 | + "(use Refund instead of a negative Sale)", | |
| 76 | + origin_pass=self.name, | |
| 77 | + )) | |
| 78 | + continue | |
| 79 | + | |
| 80 | + if event.type in TAXABLE_TYPES and event.tax is None: | |
| 81 | + diags.append(Diagnostic( | |
| 82 | + code="AIR-W200", severity=Severity.WARNING, | |
| 83 | + message=f"{event.type.value} event has no tax context; " | |
| 84 | + "it will compile untaxed", | |
| 85 | + location=f"{loc}, field tax", | |
| 86 | + suggestion="set tax.jurisdiction (e.g. 'CA-QC') or tax.exempt: true", | |
| 87 | + origin_pass=self.name, | |
| 88 | + )) | |
| 89 | + | |
| 90 | + if event.type is EventType.REFUND and event.related_event is None: | |
| 91 | + diags.append(Diagnostic( | |
| 92 | + code="AIR-W201", severity=Severity.WARNING, | |
| 93 | + message="Refund does not reference the Sale it reverses", | |
| 94 | + location=f"{loc}, field related_event", | |
| 95 | + suggestion="set related_event to the original Sale id for traceability", | |
| 96 | + origin_pass=self.name, | |
| 97 | + )) | |
| 98 | + | |
| 99 | + state = unit.event_state(event.id) | |
| 100 | + state.subtotal = subtotal | |
| 101 | + node = unit.provenance.define( | |
| 102 | + kind="event_subtotal", | |
| 103 | + operation=f"subtotal:{event.type.value}", | |
| 104 | + amount=subtotal, | |
| 105 | + source_ref=event.id, | |
| 106 | + ) | |
| 107 | + state.subtotal_node = node.id | |
| 108 | + | |
| 109 | + return diags | |
added
aic/unit.py
+81 −0
@@ -0,0 +1,81 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : unit.py | |
| 6 | +# Description : CompilationUnit — the mutable state threaded through AIC passes. | |
| 7 | +# ============================================================================= | |
| 8 | +"""The compilation unit: AIR document in, journal entries out. | |
| 9 | + | |
| 10 | +Passes read the immutable AIR events and accumulate derived state here | |
| 11 | +(subtotals, tax lines, FX conversions, classifications), all anchored in the | |
| 12 | +provenance graph. The pass manager verifies the double-entry invariant on | |
| 13 | +`entries` after every pass. | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from dataclasses import dataclass, field | |
| 18 | + | |
| 19 | +from aic.diagnostics import Diagnostic | |
| 20 | +from alsl.model import PolicySet | |
| 21 | +from core.events import AirDocument, EconomicEvent | |
| 22 | +from core.journal import JournalEntry | |
| 23 | +from core.money import Money | |
| 24 | +from core.provenance import ProvenanceGraph | |
| 25 | + | |
| 26 | + | |
| 27 | +@dataclass | |
| 28 | +class TaxLineState: | |
| 29 | + """A computed tax amount for one event (transaction currency).""" | |
| 30 | + | |
| 31 | + code: str | |
| 32 | + amount: Money | |
| 33 | + node_id: str | |
| 34 | + payable_role: str | |
| 35 | + receivable_role: str | |
| 36 | + recoverable: bool | |
| 37 | + policy: str # name of the ALSL policy that produced it | |
| 38 | + | |
| 39 | + | |
| 40 | +@dataclass | |
| 41 | +class EventState: | |
| 42 | + """Everything the passes derive for a single event.""" | |
| 43 | + | |
| 44 | + # transaction-currency amounts | |
| 45 | + subtotal: Money | None = None | |
| 46 | + subtotal_node: str | None = None | |
| 47 | + taxes: list[TaxLineState] = field(default_factory=list) | |
| 48 | + | |
| 49 | + # classification (purchases): account role for the debit side | |
| 50 | + classify_as: str | None = None # "asset" | "expense" | |
| 51 | + classification_role: str | None = None | |
| 52 | + classification_policy: str | None = None | |
| 53 | + | |
| 54 | + # functional-currency amounts, ready for posting (set by the FX pass) | |
| 55 | + post_subtotal: Money | None = None | |
| 56 | + post_subtotal_node: str | None = None | |
| 57 | + post_taxes: list[TaxLineState] = field(default_factory=list) | |
| 58 | + | |
| 59 | + # settlement FX difference (payments): + = more functional units than booked | |
| 60 | + fx_diff: Money | None = None | |
| 61 | + fx_diff_node: str | None = None | |
| 62 | + booked_amount: Money | None = None # functional value at booking rate | |
| 63 | + | |
| 64 | + | |
| 65 | +@dataclass | |
| 66 | +class CompilationUnit: | |
| 67 | + document: AirDocument | |
| 68 | + policies: PolicySet | |
| 69 | + provenance: ProvenanceGraph = field(default_factory=ProvenanceGraph) | |
| 70 | + state: dict[str, EventState] = field(default_factory=dict) | |
| 71 | + entries: list[JournalEntry] = field(default_factory=list) | |
| 72 | + diagnostics: list[Diagnostic] = field(default_factory=list) | |
| 73 | + | |
| 74 | + def event_state(self, event_id: str) -> EventState: | |
| 75 | + return self.state.setdefault(event_id, EventState()) | |
| 76 | + | |
| 77 | + def find_event(self, event_id: str) -> EconomicEvent | None: | |
| 78 | + for event in self.document.events: | |
| 79 | + if event.id == event_id: | |
| 80 | + return event | |
| 81 | + return None | |
added
alsl/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
alsl/evaluator.py
+46 −0
@@ -0,0 +1,46 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : evaluator.py | |
| 6 | +# Description : ALSL v0.1 evaluator — matches policy `when` clauses against economic events. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Deterministic evaluation of ALSL `when` clauses against AIR events.""" | |
| 9 | +from __future__ import annotations | |
| 10 | + | |
| 11 | +from alsl.model import ClassificationPolicy, PolicySet, TaxPolicy, WhenClause | |
| 12 | +from core.events import EconomicEvent | |
| 13 | +from core.money import Money | |
| 14 | + | |
| 15 | + | |
| 16 | +def matches(when: WhenClause, event: EconomicEvent, subtotal: Money | None) -> bool: | |
| 17 | + if when.event_type is not None and event.type.value != when.event_type: | |
| 18 | + return False | |
| 19 | + jurisdiction = event.tax.jurisdiction if event.tax else None | |
| 20 | + if when.jurisdiction is not None and jurisdiction != when.jurisdiction: | |
| 21 | + return False | |
| 22 | + if when.jurisdiction_in is not None and jurisdiction not in when.jurisdiction_in: | |
| 23 | + return False | |
| 24 | + if when.min_amount is not None: | |
| 25 | + if subtotal is None: | |
| 26 | + return False | |
| 27 | + if when.currency is not None and subtotal.currency != when.currency: | |
| 28 | + return False | |
| 29 | + if subtotal.amount < when.min_amount: | |
| 30 | + return False | |
| 31 | + return True | |
| 32 | + | |
| 33 | + | |
| 34 | +def applicable_tax_policies( | |
| 35 | + policies: PolicySet, event: EconomicEvent, subtotal: Money | None | |
| 36 | +) -> list[TaxPolicy]: | |
| 37 | + return [p for p in policies.tax_policies if matches(p.when, event, subtotal)] | |
| 38 | + | |
| 39 | + | |
| 40 | +def applicable_classifications( | |
| 41 | + policies: PolicySet, event: EconomicEvent, subtotal: Money | None | |
| 42 | +) -> list[ClassificationPolicy]: | |
| 43 | + return [ | |
| 44 | + p for p in policies.classification_policies | |
| 45 | + if matches(p.when, event, subtotal) | |
| 46 | + ] | |
added
alsl/loader.py
+144 −0
@@ -0,0 +1,144 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : loader.py | |
| 6 | +# Description : ALSL v0.1 YAML loader — strict parsing, floats and uncited rates rejected. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Load ALSL v0.1 policy sets from YAML. | |
| 9 | + | |
| 10 | +Strictness rules: | |
| 11 | +- Rates, thresholds and any decimal value MUST be YAML strings ("0.05"), | |
| 12 | + never bare numbers — bare YAML floats lose precision and are rejected. | |
| 13 | +- Every tax policy MUST cite a source (research doc or official URL). | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from decimal import Decimal | |
| 18 | +from pathlib import Path | |
| 19 | +from typing import Any | |
| 20 | + | |
| 21 | +import yaml | |
| 22 | + | |
| 23 | +from alsl.model import ( | |
| 24 | + ALSL_VERSION, | |
| 25 | + ClassificationPolicy, | |
| 26 | + PolicySet, | |
| 27 | + RoundingPolicy, | |
| 28 | + TaxComponent, | |
| 29 | + TaxPolicy, | |
| 30 | + WhenClause, | |
| 31 | + parse_account_type, | |
| 32 | +) | |
| 33 | +from core.journal import Account | |
| 34 | +from core.money import RoundingMode | |
| 35 | + | |
| 36 | + | |
| 37 | +class AlslLoadError(ValueError): | |
| 38 | + """Raised when a policy file violates ALSL v0.1 rules.""" | |
| 39 | + | |
| 40 | + | |
| 41 | +def _decimal(value: Any, context: str) -> Decimal: | |
| 42 | + if isinstance(value, float): | |
| 43 | + raise AlslLoadError( | |
| 44 | + f"{context}: decimal values must be YAML strings (e.g. \"0.05\"), " | |
| 45 | + f"got float {value!r} — floats are forbidden in ALSL" | |
| 46 | + ) | |
| 47 | + if isinstance(value, (int, str)): | |
| 48 | + return Decimal(str(value)) | |
| 49 | + raise AlslLoadError(f"{context}: cannot parse decimal from {value!r}") | |
| 50 | + | |
| 51 | + | |
| 52 | +def _when(raw: dict[str, Any] | None, context: str) -> WhenClause: | |
| 53 | + raw = raw or {} | |
| 54 | + jurisdiction_in = raw.get("jurisdiction_in") | |
| 55 | + return WhenClause( | |
| 56 | + jurisdiction=raw.get("jurisdiction"), | |
| 57 | + jurisdiction_in=tuple(jurisdiction_in) if jurisdiction_in else None, | |
| 58 | + event_type=raw.get("event_type"), | |
| 59 | + min_amount=( | |
| 60 | + _decimal(raw["min_amount"], context) if "min_amount" in raw else None | |
| 61 | + ), | |
| 62 | + currency=raw.get("currency"), | |
| 63 | + ) | |
| 64 | + | |
| 65 | + | |
| 66 | +def load_policy_set(path: str | Path) -> PolicySet: | |
| 67 | + path = Path(path) | |
| 68 | + data = yaml.safe_load(path.read_text(encoding="utf-8")) | |
| 69 | + if not isinstance(data, dict): | |
| 70 | + raise AlslLoadError(f"{path}: not a mapping") | |
| 71 | + | |
| 72 | + version = str(data.get("alsl_version", "")) | |
| 73 | + if version != ALSL_VERSION: | |
| 74 | + raise AlslLoadError( | |
| 75 | + f"{path}: alsl_version {version!r} unsupported (expected {ALSL_VERSION!r})" | |
| 76 | + ) | |
| 77 | + | |
| 78 | + accounts: dict[str, Account] = {} | |
| 79 | + for role, spec in (data.get("accounts") or {}).items(): | |
| 80 | + accounts[role] = Account( | |
| 81 | + code=str(spec["code"]), | |
| 82 | + name=str(spec["name"]), | |
| 83 | + type=parse_account_type(str(spec["type"])), | |
| 84 | + ) | |
| 85 | + | |
| 86 | + rounding_raw = data.get("rounding") or {} | |
| 87 | + rounding = RoundingPolicy( | |
| 88 | + mode=RoundingMode(rounding_raw.get("mode", "half_up")), | |
| 89 | + source=str(rounding_raw.get("source", "")), | |
| 90 | + ) | |
| 91 | + | |
| 92 | + tax_policies: list[TaxPolicy] = [] | |
| 93 | + classification_policies: list[ClassificationPolicy] = [] | |
| 94 | + for raw in data.get("policies") or []: | |
| 95 | + name = str(raw.get("name", "<unnamed>")) | |
| 96 | + kind = raw.get("kind") | |
| 97 | + ctx = f"{path}: policy '{name}'" | |
| 98 | + if kind == "tax": | |
| 99 | + source = str(raw.get("source", "")).strip() | |
| 100 | + if not source: | |
| 101 | + raise AlslLoadError( | |
| 102 | + f"{ctx}: tax policies must cite a source (research doc or " | |
| 103 | + "official URL); uncited rates are forbidden" | |
| 104 | + ) | |
| 105 | + components = tuple( | |
| 106 | + TaxComponent( | |
| 107 | + code=str(c["code"]), | |
| 108 | + rate=_decimal(c["rate"], f"{ctx} component {c.get('code')}"), | |
| 109 | + base=str(c.get("base", "subtotal")), | |
| 110 | + payable_role=str(c.get("payable_role", "")), | |
| 111 | + receivable_role=str(c.get("receivable_role", "")), | |
| 112 | + recoverable_on_purchase=bool(c.get("recoverable_on_purchase", True)), | |
| 113 | + ) | |
| 114 | + for c in raw.get("apply") or [] | |
| 115 | + ) | |
| 116 | + if not components: | |
| 117 | + raise AlslLoadError(f"{ctx}: tax policy applies no components") | |
| 118 | + tax_policies.append(TaxPolicy( | |
| 119 | + name=name, when=_when(raw.get("when"), ctx), | |
| 120 | + components=components, source=source, | |
| 121 | + )) | |
| 122 | + elif kind == "classification": | |
| 123 | + then = raw.get("then") or {} | |
| 124 | + classification_policies.append(ClassificationPolicy( | |
| 125 | + name=name, | |
| 126 | + when=_when(raw.get("when"), ctx), | |
| 127 | + classify_as=str(then.get("classify", "expense")), | |
| 128 | + account_role=str(then.get("account_role", "")), | |
| 129 | + source=str(raw.get("source", "")), | |
| 130 | + )) | |
| 131 | + else: | |
| 132 | + raise AlslLoadError(f"{ctx}: unknown policy kind {kind!r}") | |
| 133 | + | |
| 134 | + return PolicySet( | |
| 135 | + alsl_version=version, | |
| 136 | + name=str(data.get("policy_set", path.stem)), | |
| 137 | + version=str(data.get("version", "")), | |
| 138 | + description=str(data.get("description", "")), | |
| 139 | + functional_currency=str(data.get("functional_currency", "CAD")), | |
| 140 | + rounding=rounding, | |
| 141 | + accounts=accounts, | |
| 142 | + tax_policies=tuple(tax_policies), | |
| 143 | + classification_policies=tuple(classification_policies), | |
| 144 | + ) | |
added
alsl/model.py
+101 −0
@@ -0,0 +1,101 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : model.py | |
| 6 | +# Description : ALSL v0.1 data model — policy sets, tax/classification policies, account roles. | |
| 7 | +# ============================================================================= | |
| 8 | +"""ALSL (Accounting Language for Standards and Legislation) v0.1 model. | |
| 9 | + | |
| 10 | +ALSL is AIR's TableGen: ALL fiscal and policy parameters (tax rates, rounding | |
| 11 | +modes, capitalization thresholds, chart-of-account mappings) live in versioned | |
| 12 | +policy sets — never in compiler code. v0.1 is a YAML subset; the full DSL | |
| 13 | +comes later. | |
| 14 | + | |
| 15 | +Every tax policy must cite its source (a docs/research/ file or an official | |
| 16 | +URL) — uncited rates are rejected by the loader. | |
| 17 | +""" | |
| 18 | +from __future__ import annotations | |
| 19 | + | |
| 20 | +from dataclasses import dataclass, field | |
| 21 | +from decimal import Decimal | |
| 22 | + | |
| 23 | +from core.journal import Account, AccountType | |
| 24 | +from core.money import RoundingMode | |
| 25 | + | |
| 26 | +ALSL_VERSION = "0.1" | |
| 27 | + | |
| 28 | + | |
| 29 | +@dataclass(frozen=True, slots=True) | |
| 30 | +class WhenClause: | |
| 31 | + """Conditions under which a policy fires. All present fields must match.""" | |
| 32 | + | |
| 33 | + jurisdiction: str | None = None # exact match, e.g. "CA-QC" | |
| 34 | + jurisdiction_in: tuple[str, ...] | None = None | |
| 35 | + event_type: str | None = None # e.g. "Purchase" | |
| 36 | + min_amount: Decimal | None = None # inclusive threshold on the subtotal | |
| 37 | + currency: str | None = None # currency the threshold is expressed in | |
| 38 | + | |
| 39 | + | |
| 40 | +@dataclass(frozen=True, slots=True) | |
| 41 | +class TaxComponent: | |
| 42 | + """One tax to apply (e.g. GST at 0.05 on the subtotal).""" | |
| 43 | + | |
| 44 | + code: str # "GST", "QST", "HST" | |
| 45 | + rate: Decimal | |
| 46 | + base: str = "subtotal" # v0.1: only "subtotal" (price before other taxes) | |
| 47 | + payable_role: str = "" # account role credited on sales | |
| 48 | + receivable_role: str = "" # account role debited on purchases (ITC/ITR) | |
| 49 | + recoverable_on_purchase: bool = True | |
| 50 | + | |
| 51 | + | |
| 52 | +@dataclass(frozen=True, slots=True) | |
| 53 | +class TaxPolicy: | |
| 54 | + name: str | |
| 55 | + when: WhenClause | |
| 56 | + components: tuple[TaxComponent, ...] | |
| 57 | + source: str # citation — mandatory | |
| 58 | + | |
| 59 | + | |
| 60 | +@dataclass(frozen=True, slots=True) | |
| 61 | +class ClassificationPolicy: | |
| 62 | + """e.g. capitalize purchases >= threshold as assets.""" | |
| 63 | + | |
| 64 | + name: str | |
| 65 | + when: WhenClause | |
| 66 | + classify_as: str # "asset" | "expense" | |
| 67 | + account_role: str # role receiving the debit (e.g. "equipment") | |
| 68 | + source: str = "" | |
| 69 | + | |
| 70 | + | |
| 71 | +@dataclass(frozen=True, slots=True) | |
| 72 | +class RoundingPolicy: | |
| 73 | + mode: RoundingMode = RoundingMode.HALF_UP | |
| 74 | + source: str = "" | |
| 75 | + | |
| 76 | + | |
| 77 | +@dataclass(frozen=True, slots=True) | |
| 78 | +class PolicySet: | |
| 79 | + """A versioned, self-contained rule set for one compilation.""" | |
| 80 | + | |
| 81 | + alsl_version: str | |
| 82 | + name: str # e.g. "ca-qc" | |
| 83 | + version: str # e.g. "2026.08" | |
| 84 | + description: str = "" | |
| 85 | + functional_currency: str = "CAD" | |
| 86 | + rounding: RoundingPolicy = field(default=RoundingPolicy()) | |
| 87 | + accounts: dict[str, Account] = field(default_factory=dict) # role -> Account | |
| 88 | + tax_policies: tuple[TaxPolicy, ...] = () | |
| 89 | + classification_policies: tuple[ClassificationPolicy, ...] = () | |
| 90 | + | |
| 91 | + def account(self, role: str) -> Account: | |
| 92 | + try: | |
| 93 | + return self.accounts[role] | |
| 94 | + except KeyError: | |
| 95 | + raise KeyError( | |
| 96 | + f"policy set '{self.name}' defines no account for role '{role}'" | |
| 97 | + ) from None | |
| 98 | + | |
| 99 | + | |
| 100 | +def parse_account_type(value: str) -> AccountType: | |
| 101 | + return AccountType(value) | |
added
alsl/policies/ca-qc-2026.yaml
+100 −0
@@ -0,0 +1,100 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | |
| 3 | +# Contact : contact@spboucher.ai | |
| 4 | +# | |
| 5 | +# ALSL v0.1 policy set — Canada / Quebec, verified 2026-08-05. | |
| 6 | +# Every rate below is cited; see docs/research/canada-gst-qst.md for sources | |
| 7 | +# (Revenu Québec rate tables, CRA canada.ca, Excise Tax Act s. 165.2). | |
| 8 | +# Rates are YAML strings on purpose: bare floats are rejected by the loader. | |
| 9 | + | |
| 10 | +alsl_version: "0.1" | |
| 11 | +policy_set: ca-qc | |
| 12 | +version: "2026.08" | |
| 13 | +description: > | |
| 14 | + Canadian sales taxes (Quebec focus) + baseline classification policies. | |
| 15 | + GST 5% (since 2008-01-01); QST 9.975% on the pre-GST price (since | |
| 16 | + 2013-01-01); HST for participating provinces; GST-only for AB/NT/NU/YT. | |
| 17 | +functional_currency: CAD | |
| 18 | + | |
| 19 | +# Excise Tax Act s. 165.2(2): fractions < $0.005 disregarded, >= $0.005 deemed | |
| 20 | +# one cent => round-half-up to the cent. Same rule for QST (RQ IN-203-V). | |
| 21 | +# NOT banker's rounding — this is a legal requirement, hence a policy value. | |
| 22 | +rounding: | |
| 23 | + mode: half_up | |
| 24 | + source: docs/research/canada-gst-qst.md | |
| 25 | + | |
| 26 | +accounts: | |
| 27 | + cash: {code: "1000", name: "Cash", type: asset} | |
| 28 | + accounts_receivable: {code: "1100", name: "Accounts receivable", type: asset} | |
| 29 | + gst_receivable: {code: "1210", name: "GST recoverable (ITC)", type: asset} | |
| 30 | + qst_receivable: {code: "1220", name: "QST recoverable (ITR)", type: asset} | |
| 31 | + hst_receivable: {code: "1230", name: "HST recoverable (ITC)", type: asset} | |
| 32 | + equipment: {code: "1500", name: "Equipment", type: asset} | |
| 33 | + accounts_payable: {code: "2000", name: "Accounts payable", type: liability} | |
| 34 | + gst_payable: {code: "2310", name: "GST payable", type: liability} | |
| 35 | + qst_payable: {code: "2320", name: "QST payable", type: liability} | |
| 36 | + hst_payable: {code: "2330", name: "HST payable", type: liability} | |
| 37 | + loan_payable: {code: "2500", name: "Loans payable", type: liability} | |
| 38 | + owner_capital: {code: "3000", name: "Owner capital", type: equity} | |
| 39 | + revenue: {code: "4000", name: "Sales revenue", type: revenue} | |
| 40 | + fx_gain: {code: "4500", name: "Realized FX gain", type: revenue} | |
| 41 | + expense_default: {code: "5000", name: "Operating expenses", type: expense} | |
| 42 | + fx_loss: {code: "5500", name: "Realized FX loss", type: expense} | |
| 43 | + | |
| 44 | +policies: | |
| 45 | + # GST 5% — applies across Canada outside HST provinces (here: QC + AB). | |
| 46 | + # Source: CRA GST/HST rate table; Revenu Québec "Basic Rules"; verified 2026-08-05. | |
| 47 | + - name: gst_canada | |
| 48 | + kind: tax | |
| 49 | + when: | |
| 50 | + jurisdiction_in: [CA-QC, CA-AB] | |
| 51 | + apply: | |
| 52 | + - code: GST | |
| 53 | + rate: "0.05" | |
| 54 | + base: subtotal | |
| 55 | + payable_role: gst_payable | |
| 56 | + receivable_role: gst_receivable | |
| 57 | + recoverable_on_purchase: true | |
| 58 | + source: docs/research/canada-gst-qst.md | |
| 59 | + | |
| 60 | + # QST 9.975% — calculated on the selling price NOT including GST (since 2013). | |
| 61 | + # Combined effective rate with GST: 14.975%. Verified 2026-08-05. | |
| 62 | + - name: qst_quebec | |
| 63 | + kind: tax | |
| 64 | + when: | |
| 65 | + jurisdiction: CA-QC | |
| 66 | + apply: | |
| 67 | + - code: QST | |
| 68 | + rate: "0.09975" | |
| 69 | + base: subtotal | |
| 70 | + payable_role: qst_payable | |
| 71 | + receivable_role: qst_receivable | |
| 72 | + recoverable_on_purchase: true | |
| 73 | + source: docs/research/canada-gst-qst.md | |
| 74 | + | |
| 75 | + # HST Ontario 13% — single harmonized tax. Verified 2026-08-05. | |
| 76 | + - name: hst_ontario | |
| 77 | + kind: tax | |
| 78 | + when: | |
| 79 | + jurisdiction: CA-ON | |
| 80 | + apply: | |
| 81 | + - code: HST | |
| 82 | + rate: "0.13" | |
| 83 | + base: subtotal | |
| 84 | + payable_role: hst_payable | |
| 85 | + receivable_role: hst_receivable | |
| 86 | + recoverable_on_purchase: true | |
| 87 | + source: docs/research/canada-gst-qst.md | |
| 88 | + | |
| 89 | + # Illustrative capitalization threshold: purchases >= 5000 CAD are capitalized | |
| 90 | + # as equipment. This is an entity accounting POLICY (not law) — tune per entity. | |
| 91 | + - name: capitalization_equipment | |
| 92 | + kind: classification | |
| 93 | + when: | |
| 94 | + event_type: Purchase | |
| 95 | + min_amount: "5000" | |
| 96 | + currency: CAD | |
| 97 | + then: | |
| 98 | + classify: asset | |
| 99 | + account_role: equipment | |
| 100 | + source: "entity accounting policy (example); materiality thresholds are entity-specific" | |
added
backends/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
backends/base.py
+75 −0
@@ -0,0 +1,75 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : base.py | |
| 6 | +# Description : Common Backend interface — capabilities / compile / post / reverse. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Backend interface (the LLVM target abstraction). | |
| 9 | + | |
| 10 | +A backend lowers a CompiledJournal into a target payload (CSV rows, QBO | |
| 11 | +JournalEntry JSON, Xero ManualJournal, the native ledger...) and posts it. | |
| 12 | +Design informed by docs/research/erp-apis.md: | |
| 13 | +- amounts are unsigned Decimals + a debit/credit side (the canonical form | |
| 14 | + every surveyed API can be derived from); | |
| 15 | +- post() takes an idempotency key — retries must never double-post; | |
| 16 | +- reversal is a first-class operation (strategies vary by target: native, | |
| 17 | + void, or contra entry) — posted entries are NEVER deleted. | |
| 18 | +""" | |
| 19 | +from __future__ import annotations | |
| 20 | + | |
| 21 | +import abc | |
| 22 | +from dataclasses import dataclass, field | |
| 23 | +from typing import Any | |
| 24 | + | |
| 25 | +from core.journal import CompiledJournal | |
| 26 | + | |
| 27 | + | |
| 28 | +@dataclass(frozen=True, slots=True) | |
| 29 | +class BackendCapabilities: | |
| 30 | + name: str | |
| 31 | + posts_remotely: bool # False: file/local targets | |
| 32 | + native_reversal: bool # target supports reversal natively | |
| 33 | + multi_currency: bool | |
| 34 | + idempotency: str # "native" | "external-id" | "client-side" | |
| 35 | + notes: str = "" | |
| 36 | + | |
| 37 | + | |
| 38 | +@dataclass(frozen=True, slots=True) | |
| 39 | +class TargetPayload: | |
| 40 | + """What compile() produces: target-shaped data, not yet posted.""" | |
| 41 | + | |
| 42 | + backend: str | |
| 43 | + format: str # e.g. "csv", "json", "qbo.journalentry" | |
| 44 | + body: Any # backend-specific representation | |
| 45 | + | |
| 46 | + | |
| 47 | +@dataclass(frozen=True, slots=True) | |
| 48 | +class PostingReceipt: | |
| 49 | + backend: str | |
| 50 | + reference: str # file path, remote id, ledger sequence... | |
| 51 | + idempotency_key: str | |
| 52 | + details: dict[str, str] = field(default_factory=dict) | |
| 53 | + | |
| 54 | + | |
| 55 | +@dataclass(frozen=True, slots=True) | |
| 56 | +class ReversalReceipt: | |
| 57 | + backend: str | |
| 58 | + reference: str | |
| 59 | + reversed_reference: str | |
| 60 | + | |
| 61 | + | |
| 62 | +class Backend(abc.ABC): | |
| 63 | + """Every AIR target implements this interface.""" | |
| 64 | + | |
| 65 | + @abc.abstractmethod | |
| 66 | + def capabilities(self) -> BackendCapabilities: ... | |
| 67 | + | |
| 68 | + @abc.abstractmethod | |
| 69 | + def compile(self, journal: CompiledJournal) -> TargetPayload: ... | |
| 70 | + | |
| 71 | + @abc.abstractmethod | |
| 72 | + def post(self, payload: TargetPayload, idempotency_key: str) -> PostingReceipt: ... | |
| 73 | + | |
| 74 | + @abc.abstractmethod | |
| 75 | + def reverse(self, receipt: PostingReceipt) -> ReversalReceipt: ... | |
added
backends/generic_csv/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
backends/generic_csv/backend.py
+121 −0
@@ -0,0 +1,121 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : backend.py | |
| 6 | +# Description : Generic CSV backend — the first, universal journal export target. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Generic CSV backend. | |
| 9 | + | |
| 10 | +Lowers a CompiledJournal to flat CSV rows (one row per journal line) that any | |
| 11 | +accounting system or spreadsheet can import. Reversal emits contra rows | |
| 12 | +(sides swapped) — history is never deleted. | |
| 13 | +""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +import csv | |
| 17 | +import io | |
| 18 | +from pathlib import Path | |
| 19 | + | |
| 20 | +from backends.base import ( | |
| 21 | + Backend, | |
| 22 | + BackendCapabilities, | |
| 23 | + PostingReceipt, | |
| 24 | + ReversalReceipt, | |
| 25 | + TargetPayload, | |
| 26 | +) | |
| 27 | +from core.journal import CompiledJournal, Side | |
| 28 | + | |
| 29 | +COLUMNS = [ | |
| 30 | + "entry_id", "date", "description", "account_code", "account_name", | |
| 31 | + "account_type", "side", "amount", "currency", "memo", | |
| 32 | + "source_event_id", "provenance_id", "policy_set", "policy_version", | |
| 33 | + "reverses", | |
| 34 | +] | |
| 35 | + | |
| 36 | + | |
| 37 | +class GenericCsvBackend(Backend): | |
| 38 | + """Writes journals to CSV files under an output directory.""" | |
| 39 | + | |
| 40 | + def __init__(self, output_dir: str | Path = "out"): | |
| 41 | + self.output_dir = Path(output_dir) | |
| 42 | + | |
| 43 | + def capabilities(self) -> BackendCapabilities: | |
| 44 | + return BackendCapabilities( | |
| 45 | + name="generic_csv", | |
| 46 | + posts_remotely=False, | |
| 47 | + native_reversal=False, # reversal = contra rows | |
| 48 | + multi_currency=True, | |
| 49 | + idempotency="client-side", | |
| 50 | + notes="universal flat-file journal export", | |
| 51 | + ) | |
| 52 | + | |
| 53 | + def _rows(self, journal: CompiledJournal, *, reverse: bool = False) -> list[list[str]]: | |
| 54 | + rows: list[list[str]] = [] | |
| 55 | + for entry in journal.entries: | |
| 56 | + for line in entry.lines: | |
| 57 | + side = line.side | |
| 58 | + if reverse: | |
| 59 | + side = Side.CREDIT if side is Side.DEBIT else Side.DEBIT | |
| 60 | + rows.append([ | |
| 61 | + ("rev_" if reverse else "") + entry.id, | |
| 62 | + entry.date.isoformat(), | |
| 63 | + ("REVERSAL: " if reverse else "") + entry.description, | |
| 64 | + line.account.code, | |
| 65 | + line.account.name, | |
| 66 | + line.account.type.value, | |
| 67 | + side.value, | |
| 68 | + str(line.amount.amount), | |
| 69 | + line.amount.currency, | |
| 70 | + line.memo, | |
| 71 | + entry.source_event_id, | |
| 72 | + line.provenance_id or "", | |
| 73 | + entry.policy_set, | |
| 74 | + entry.policy_version, | |
| 75 | + (entry.id if reverse else entry.reverses or ""), | |
| 76 | + ]) | |
| 77 | + return rows | |
| 78 | + | |
| 79 | + def compile(self, journal: CompiledJournal) -> TargetPayload: | |
| 80 | + buf = io.StringIO() | |
| 81 | + writer = csv.writer(buf) | |
| 82 | + writer.writerow(COLUMNS) | |
| 83 | + writer.writerows(self._rows(journal)) | |
| 84 | + return TargetPayload(backend="generic_csv", format="csv", body=buf.getvalue()) | |
| 85 | + | |
| 86 | + def post(self, payload: TargetPayload, idempotency_key: str) -> PostingReceipt: | |
| 87 | + self.output_dir.mkdir(parents=True, exist_ok=True) | |
| 88 | + path = self.output_dir / f"journal_{idempotency_key}.csv" | |
| 89 | + if not path.exists(): # idempotent: same key never rewrites | |
| 90 | + path.write_text(str(payload.body), encoding="utf-8") | |
| 91 | + return PostingReceipt( | |
| 92 | + backend="generic_csv", | |
| 93 | + reference=str(path), | |
| 94 | + idempotency_key=idempotency_key, | |
| 95 | + ) | |
| 96 | + | |
| 97 | + def reverse(self, receipt: PostingReceipt) -> ReversalReceipt: | |
| 98 | + original = Path(receipt.reference) | |
| 99 | + reader = csv.reader(io.StringIO(original.read_text(encoding="utf-8"))) | |
| 100 | + rows = list(reader) | |
| 101 | + header, body = rows[0], rows[1:] | |
| 102 | + side_i, entry_i, desc_i, rev_i = ( | |
| 103 | + header.index("side"), header.index("entry_id"), | |
| 104 | + header.index("description"), header.index("reverses"), | |
| 105 | + ) | |
| 106 | + for row in body: | |
| 107 | + row[rev_i] = row[entry_i] | |
| 108 | + row[entry_i] = "rev_" + row[entry_i] | |
| 109 | + row[desc_i] = "REVERSAL: " + row[desc_i] | |
| 110 | + row[side_i] = "credit" if row[side_i] == "debit" else "debit" | |
| 111 | + buf = io.StringIO() | |
| 112 | + writer = csv.writer(buf) | |
| 113 | + writer.writerow(header) | |
| 114 | + writer.writerows(body) | |
| 115 | + path = original.with_name(original.stem + "_reversal.csv") | |
| 116 | + path.write_text(buf.getvalue(), encoding="utf-8") | |
| 117 | + return ReversalReceipt( | |
| 118 | + backend="generic_csv", | |
| 119 | + reference=str(path), | |
| 120 | + reversed_reference=receipt.reference, | |
| 121 | + ) | |
added
backends/native/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
backends/native/backend.py
+73 −0
@@ -0,0 +1,73 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : backend.py | |
| 6 | +# Description : Native backend — posts into AIR's own ledger (standalone mode, no ERP needed). | |
| 7 | +# ============================================================================= | |
| 8 | +"""Native backend: AIR as a self-sufficient accounting system. | |
| 9 | + | |
| 10 | +When no third-party system (QuickBooks, Xero, ...) is used, compiled journals | |
| 11 | +post into AIR's own append-only, hash-chained ledger, from which all | |
| 12 | +statements (general ledger, trial balance, income statement, balance sheet) | |
| 13 | +are generated in multiple formats. | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from pathlib import Path | |
| 18 | + | |
| 19 | +from backends.base import ( | |
| 20 | + Backend, | |
| 21 | + BackendCapabilities, | |
| 22 | + PostingReceipt, | |
| 23 | + ReversalReceipt, | |
| 24 | + TargetPayload, | |
| 25 | +) | |
| 26 | +from core.journal import CompiledJournal | |
| 27 | +from kernel.ledger import Ledger | |
| 28 | + | |
| 29 | + | |
| 30 | +class NativeLedgerBackend(Backend): | |
| 31 | + def __init__(self, ledger_path: str | Path | None = None): | |
| 32 | + self.ledger = Ledger(path=Path(ledger_path) if ledger_path else None) | |
| 33 | + | |
| 34 | + def capabilities(self) -> BackendCapabilities: | |
| 35 | + return BackendCapabilities( | |
| 36 | + name="native", | |
| 37 | + posts_remotely=False, | |
| 38 | + native_reversal=True, | |
| 39 | + multi_currency=True, | |
| 40 | + idempotency="native", | |
| 41 | + notes="AIR's own append-only hash-chained ledger; full reporting built in", | |
| 42 | + ) | |
| 43 | + | |
| 44 | + def compile(self, journal: CompiledJournal) -> TargetPayload: | |
| 45 | + # The native target consumes the journal as-is: no lowering needed. | |
| 46 | + return TargetPayload(backend="native", format="journal", body=journal) | |
| 47 | + | |
| 48 | + def post(self, payload: TargetPayload, idempotency_key: str) -> PostingReceipt: | |
| 49 | + journal = payload.body | |
| 50 | + assert isinstance(journal, CompiledJournal) | |
| 51 | + appended = self.ledger.post_journal(journal, idempotency_key) | |
| 52 | + return PostingReceipt( | |
| 53 | + backend="native", | |
| 54 | + reference=str(self.ledger.path or "<memory>"), | |
| 55 | + idempotency_key=idempotency_key, | |
| 56 | + details={"entries_appended": str(appended)}, | |
| 57 | + ) | |
| 58 | + | |
| 59 | + def reverse(self, receipt: PostingReceipt) -> ReversalReceipt: | |
| 60 | + # Reverse every entry posted under the receipt's idempotency key. | |
| 61 | + reversed_ids = [] | |
| 62 | + for entry in list(self.ledger.entries): | |
| 63 | + if entry.id.startswith("rev_"): | |
| 64 | + continue | |
| 65 | + contra = self.ledger.reverse_entry( | |
| 66 | + entry.id, f"rev:{receipt.idempotency_key}:{entry.id}" | |
| 67 | + ) | |
| 68 | + reversed_ids.append(contra.id) | |
| 69 | + return ReversalReceipt( | |
| 70 | + backend="native", | |
| 71 | + reference=",".join(reversed_ids), | |
| 72 | + reversed_reference=receipt.idempotency_key, | |
| 73 | + ) | |
added
backends/quickbooks/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
backends/quickbooks/backend.py
+94 −0
@@ -0,0 +1,94 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : backend.py | |
| 6 | +# Description : QuickBooks Online backend — optional target, fully testable offline. | |
| 7 | +# ============================================================================= | |
| 8 | +"""QuickBooks Online backend. | |
| 9 | + | |
| 10 | +QuickBooks is an OPTIONAL export target: AIR's books of record live in the | |
| 11 | +native ledger regardless. This backend lowers compiled journals to QBO | |
| 12 | +JournalEntry payloads and posts them through a pluggable transport — the | |
| 13 | +offline mock by default (no QuickBooks account needed anywhere in the | |
| 14 | +project), the HTTP transport only when the user supplies credentials. | |
| 15 | + | |
| 16 | +Idempotency: every create carries requestid = f"{key}:{DocNumber}" so retries | |
| 17 | +never double-post. Reversal: QBO has no native journal reversal, so reverse() | |
| 18 | +posts exact contra entries (never deletes). | |
| 19 | +""" | |
| 20 | +from __future__ import annotations | |
| 21 | + | |
| 22 | +from typing import Any | |
| 23 | + | |
| 24 | +from backends.base import ( | |
| 25 | + Backend, | |
| 26 | + BackendCapabilities, | |
| 27 | + PostingReceipt, | |
| 28 | + ReversalReceipt, | |
| 29 | + TargetPayload, | |
| 30 | +) | |
| 31 | +from backends.quickbooks.mapper import contra_body, map_journal | |
| 32 | +from backends.quickbooks.transport import MockQboTransport, QboTransport | |
| 33 | +from core.journal import CompiledJournal | |
| 34 | + | |
| 35 | + | |
| 36 | +class QuickBooksBackend(Backend): | |
| 37 | + def __init__( | |
| 38 | + self, | |
| 39 | + transport: QboTransport | None = None, | |
| 40 | + account_map: dict[str, dict[str, str]] | None = None, | |
| 41 | + ): | |
| 42 | + self.transport = transport if transport is not None else MockQboTransport() | |
| 43 | + self.account_map = account_map | |
| 44 | + self._posted: dict[str, dict[str, Any]] = {} # DocNumber -> body posted | |
| 45 | + | |
| 46 | + def capabilities(self) -> BackendCapabilities: | |
| 47 | + return BackendCapabilities( | |
| 48 | + name="quickbooks", | |
| 49 | + posts_remotely=True, | |
| 50 | + native_reversal=False, # contra-entry strategy | |
| 51 | + multi_currency=True, | |
| 52 | + idempotency="requestid", | |
| 53 | + notes="QBO JournalEntry API, minorversion=75; optional target — " | |
| 54 | + "native ledger remains the book of record", | |
| 55 | + ) | |
| 56 | + | |
| 57 | + def compile(self, journal: CompiledJournal) -> TargetPayload: | |
| 58 | + return TargetPayload( | |
| 59 | + backend="quickbooks", | |
| 60 | + format="qbo.journalentry", | |
| 61 | + body=map_journal(journal, self.account_map), | |
| 62 | + ) | |
| 63 | + | |
| 64 | + def post(self, payload: TargetPayload, idempotency_key: str) -> PostingReceipt: | |
| 65 | + bodies = payload.body | |
| 66 | + assert isinstance(bodies, list) | |
| 67 | + details: dict[str, str] = {} | |
| 68 | + for body in bodies: | |
| 69 | + request_id = f"{idempotency_key}:{body['DocNumber']}" | |
| 70 | + entity = self.transport.create_journal_entry(body, request_id) | |
| 71 | + details[body["DocNumber"]] = entity["Id"] | |
| 72 | + self._posted[body["DocNumber"]] = body | |
| 73 | + return PostingReceipt( | |
| 74 | + backend="quickbooks", | |
| 75 | + reference=",".join(details.values()), | |
| 76 | + idempotency_key=idempotency_key, | |
| 77 | + details=details, | |
| 78 | + ) | |
| 79 | + | |
| 80 | + def reverse(self, receipt: PostingReceipt) -> ReversalReceipt: | |
| 81 | + reversed_ids: list[str] = [] | |
| 82 | + for doc_number in receipt.details: | |
| 83 | + body = self._posted.get(doc_number) | |
| 84 | + if body is None: | |
| 85 | + continue | |
| 86 | + contra = contra_body(body) | |
| 87 | + request_id = f"rev:{receipt.idempotency_key}:{contra['DocNumber']}" | |
| 88 | + entity = self.transport.create_journal_entry(contra, request_id) | |
| 89 | + reversed_ids.append(entity["Id"]) | |
| 90 | + return ReversalReceipt( | |
| 91 | + backend="quickbooks", | |
| 92 | + reference=",".join(reversed_ids), | |
| 93 | + reversed_reference=receipt.reference, | |
| 94 | + ) | |
added
backends/quickbooks/mapper.py
+114 −0
@@ -0,0 +1,114 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : mapper.py | |
| 6 | +# Description : Lower a CompiledJournal to QuickBooks Online JournalEntry payloads (pure, offline). | |
| 7 | +# ============================================================================= | |
| 8 | +"""CompiledJournal -> QBO JournalEntry payloads. | |
| 9 | + | |
| 10 | +Pure lowering, no network: this module runs identically with or without a | |
| 11 | +QuickBooks account, so the whole backend is testable offline. | |
| 12 | + | |
| 13 | +Shape per docs/research/erp-apis.md (QBO API, minorversion=75 baseline): | |
| 14 | +- Line[] with unsigned Amount + PostingType ("Debit"/"Credit") + AccountRef | |
| 15 | +- DocNumber max 21 chars (longer ids are truncated with a stable hash suffix) | |
| 16 | +- amounts stay Decimal end-to-end here; serialization to exact JSON numbers | |
| 17 | + is the transport's job (never through float). | |
| 18 | +""" | |
| 19 | +from __future__ import annotations | |
| 20 | + | |
| 21 | +import hashlib | |
| 22 | +from typing import Any | |
| 23 | + | |
| 24 | +from core.journal import CompiledJournal, JournalEntry, Side | |
| 25 | + | |
| 26 | +DOCNUMBER_MAX = 21 | |
| 27 | + | |
| 28 | + | |
| 29 | +class AccountMappingError(KeyError): | |
| 30 | + """An account code has no QuickBooks AccountRef mapping.""" | |
| 31 | + | |
| 32 | + | |
| 33 | +def doc_number(entry_id: str) -> str: | |
| 34 | + """QBO DocNumber (<= 21 chars), stable for a given entry id.""" | |
| 35 | + if len(entry_id) <= DOCNUMBER_MAX: | |
| 36 | + return entry_id | |
| 37 | + digest = hashlib.sha256(entry_id.encode("utf-8")).hexdigest()[:6] | |
| 38 | + return f"{entry_id[:DOCNUMBER_MAX - 7]}-{digest}" | |
| 39 | + | |
| 40 | + | |
| 41 | +def map_entry( | |
| 42 | + entry: JournalEntry, | |
| 43 | + account_map: dict[str, dict[str, str]] | None = None, | |
| 44 | +) -> dict[str, Any]: | |
| 45 | + """One JournalEntry -> one QBO JournalEntry create body. | |
| 46 | + | |
| 47 | + account_map: our account code -> {"value": <qbo account id>, "name": ...}. | |
| 48 | + When None, the code itself is used as the AccountRef value (fine for the | |
| 49 | + mock transport and for offline export; a real realm needs the map). | |
| 50 | + """ | |
| 51 | + lines: list[dict[str, Any]] = [] | |
| 52 | + for line in entry.lines: | |
| 53 | + if account_map is not None: | |
| 54 | + try: | |
| 55 | + ref = account_map[line.account.code] | |
| 56 | + except KeyError: | |
| 57 | + raise AccountMappingError( | |
| 58 | + f"account '{line.account.code} {line.account.name}' has no " | |
| 59 | + "QuickBooks AccountRef in the account map; add it or post " | |
| 60 | + "to the native backend instead" | |
| 61 | + ) from None | |
| 62 | + else: | |
| 63 | + ref = {"value": line.account.code, "name": line.account.name} | |
| 64 | + lines.append({ | |
| 65 | + "Description": line.memo, | |
| 66 | + "Amount": line.amount.amount, # Decimal, exact | |
| 67 | + "DetailType": "JournalEntryLineDetail", | |
| 68 | + "JournalEntryLineDetail": { | |
| 69 | + "PostingType": "Debit" if line.side is Side.DEBIT else "Credit", | |
| 70 | + "AccountRef": dict(ref), | |
| 71 | + }, | |
| 72 | + }) | |
| 73 | + return { | |
| 74 | + "DocNumber": doc_number(entry.id), | |
| 75 | + "TxnDate": entry.date.isoformat(), | |
| 76 | + "PrivateNote": ( | |
| 77 | + f"{entry.description} | AIR event {entry.source_event_id} | " | |
| 78 | + f"policies {entry.policy_set}@{entry.policy_version}" | |
| 79 | + ), | |
| 80 | + "Line": lines, | |
| 81 | + "CurrencyRef": {"value": entry.lines[0].amount.currency}, | |
| 82 | + } | |
| 83 | + | |
| 84 | + | |
| 85 | +def map_journal( | |
| 86 | + journal: CompiledJournal, | |
| 87 | + account_map: dict[str, dict[str, str]] | None = None, | |
| 88 | +) -> list[dict[str, Any]]: | |
| 89 | + return [map_entry(entry, account_map) for entry in journal.entries] | |
| 90 | + | |
| 91 | + | |
| 92 | +def contra_body(body: dict[str, Any]) -> dict[str, Any]: | |
| 93 | + """Reversal payload: QBO has no native journal-entry reversal, so we | |
| 94 | + synthesize an exact contra entry (sides swapped, amounts identical).""" | |
| 95 | + contra = { | |
| 96 | + "DocNumber": doc_number("R" + body["DocNumber"]), | |
| 97 | + "TxnDate": body["TxnDate"], | |
| 98 | + "PrivateNote": "REVERSAL of " + body["DocNumber"] + " | " + body["PrivateNote"], | |
| 99 | + "Line": [], | |
| 100 | + "CurrencyRef": dict(body["CurrencyRef"]), | |
| 101 | + } | |
| 102 | + for line in body["Line"]: | |
| 103 | + detail = line["JournalEntryLineDetail"] | |
| 104 | + contra["Line"].append({ | |
| 105 | + "Description": "reversal: " + line["Description"], | |
| 106 | + "Amount": line["Amount"], | |
| 107 | + "DetailType": "JournalEntryLineDetail", | |
| 108 | + "JournalEntryLineDetail": { | |
| 109 | + "PostingType": ("Credit" if detail["PostingType"] == "Debit" | |
| 110 | + else "Debit"), | |
| 111 | + "AccountRef": dict(detail["AccountRef"]), | |
| 112 | + }, | |
| 113 | + }) | |
| 114 | + return contra | |
added
backends/quickbooks/transport.py
+147 −0
@@ -0,0 +1,147 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : transport.py | |
| 6 | +# Description : QBO transports — offline mock (default, no QuickBooks needed) and real HTTP. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Transports for the QuickBooks backend. | |
| 9 | + | |
| 10 | +AIR must build and test WITHOUT any QuickBooks access, so the transport is an | |
| 11 | +interface with two implementations: | |
| 12 | + | |
| 13 | +- MockQboTransport (default): a faithful in-memory simulation of the QBO | |
| 14 | + behaviors that matter (per docs/research/erp-apis.md): `requestid` | |
| 15 | + idempotency replay, duplicate-DocNumber error 6140, server-assigned Ids. | |
| 16 | + All tests run against this — no network, no account, ever. | |
| 17 | +- HttpQboTransport: the real thing (stdlib urllib, OAuth2 bearer token, | |
| 18 | + minorversion=75, exact-decimal JSON serialization). It is only constructed | |
| 19 | + when the user explicitly provides credentials; nothing in the test suite | |
| 20 | + or default CLI paths touches it. | |
| 21 | +""" | |
| 22 | +from __future__ import annotations | |
| 23 | + | |
| 24 | +import abc | |
| 25 | +import json | |
| 26 | +import re | |
| 27 | +import urllib.request | |
| 28 | +from decimal import Decimal | |
| 29 | +from typing import Any | |
| 30 | + | |
| 31 | +QBO_MINOR_VERSION = "75" # minorversion baseline since 2025-08-01 (research) | |
| 32 | + | |
| 33 | + | |
| 34 | +class QboError(RuntimeError): | |
| 35 | + def __init__(self, code: str, message: str): | |
| 36 | + self.code = code | |
| 37 | + super().__init__(f"QBO error {code}: {message}") | |
| 38 | + | |
| 39 | + | |
| 40 | +class QboTransport(abc.ABC): | |
| 41 | + """Minimal surface the backend needs: create a JournalEntry.""" | |
| 42 | + | |
| 43 | + @abc.abstractmethod | |
| 44 | + def create_journal_entry(self, body: dict[str, Any], request_id: str) -> dict[str, Any]: | |
| 45 | + """POST a JournalEntry; MUST be idempotent on request_id. Returns the | |
| 46 | + created (or replayed) entity, including its server 'Id'.""" | |
| 47 | + | |
| 48 | + | |
| 49 | +class MockQboTransport(QboTransport): | |
| 50 | + """Offline QBO simulator — the default transport. | |
| 51 | + | |
| 52 | + Simulated behaviors: | |
| 53 | + - requestid replay: same request_id returns the original entity, creates | |
| 54 | + nothing (QBO guarantees this for supported entities); | |
| 55 | + - duplicate DocNumber with a NEW request_id raises error 6140; | |
| 56 | + - server-assigned incremental Ids and SyncToken 0. | |
| 57 | + """ | |
| 58 | + | |
| 59 | + def __init__(self) -> None: | |
| 60 | + self.store: dict[str, dict[str, Any]] = {} # Id -> entity | |
| 61 | + self._by_request: dict[str, str] = {} # request_id -> Id | |
| 62 | + self._by_docnumber: dict[str, str] = {} # DocNumber -> Id | |
| 63 | + self._next_id = 1 | |
| 64 | + | |
| 65 | + def create_journal_entry(self, body: dict[str, Any], request_id: str) -> dict[str, Any]: | |
| 66 | + if request_id in self._by_request: # idempotent replay | |
| 67 | + return self.store[self._by_request[request_id]] | |
| 68 | + doc = str(body.get("DocNumber", "")) | |
| 69 | + if doc and doc in self._by_docnumber: | |
| 70 | + raise QboError( | |
| 71 | + "6140", | |
| 72 | + f"Duplicate Document Number Error: DocNumber '{doc}' already exists", | |
| 73 | + ) | |
| 74 | + debits = sum( | |
| 75 | + Decimal(str(l["Amount"])) for l in body["Line"] | |
| 76 | + if l["JournalEntryLineDetail"]["PostingType"] == "Debit" | |
| 77 | + ) | |
| 78 | + credits = sum( | |
| 79 | + Decimal(str(l["Amount"])) for l in body["Line"] | |
| 80 | + if l["JournalEntryLineDetail"]["PostingType"] == "Credit" | |
| 81 | + ) | |
| 82 | + if debits != credits: | |
| 83 | + raise QboError("6000", f"Journal entry must balance: {debits} != {credits}") | |
| 84 | + | |
| 85 | + entity = dict(body) | |
| 86 | + entity["Id"] = str(self._next_id) | |
| 87 | + entity["SyncToken"] = "0" | |
| 88 | + self._next_id += 1 | |
| 89 | + self.store[entity["Id"]] = entity | |
| 90 | + self._by_request[request_id] = entity["Id"] | |
| 91 | + if doc: | |
| 92 | + self._by_docnumber[doc] = entity["Id"] | |
| 93 | + return entity | |
| 94 | + | |
| 95 | + | |
| 96 | +_DEC_SENTINEL = re.compile(r'"__DEC__(-?\d+(?:\.\d+)?)__"') | |
| 97 | + | |
| 98 | + | |
| 99 | +def _dumps_exact(obj: Any) -> str: | |
| 100 | + """JSON with Decimals emitted as exact numeric literals (never float). | |
| 101 | + | |
| 102 | + Decimals are encoded as strict sentinel strings, then unquoted. The | |
| 103 | + sentinel pattern only matches the exact canonical form generated here, | |
| 104 | + so ordinary string values can never be corrupted. | |
| 105 | + """ | |
| 106 | + def encode(o: Any) -> Any: | |
| 107 | + if isinstance(o, Decimal): | |
| 108 | + return f"__DEC__{o}__" | |
| 109 | + raise TypeError(type(o).__name__) | |
| 110 | + | |
| 111 | + return _DEC_SENTINEL.sub(r"\1", json.dumps(obj, default=encode)) | |
| 112 | + | |
| 113 | + | |
| 114 | +class HttpQboTransport(QboTransport): | |
| 115 | + """Real QuickBooks Online transport. OPTIONAL — requires explicit | |
| 116 | + credentials; never used by tests or default flows. | |
| 117 | + | |
| 118 | + Note: token refresh is the caller's concern for now (Phase 3 scope); | |
| 119 | + pass a valid OAuth2 access token. | |
| 120 | + """ | |
| 121 | + | |
| 122 | + SANDBOX_BASE = "https://sandbox-quickbooks.api.intuit.com" | |
| 123 | + PRODUCTION_BASE = "https://quickbooks.api.intuit.com" | |
| 124 | + | |
| 125 | + def __init__(self, realm_id: str, access_token: str, *, sandbox: bool = True): | |
| 126 | + self.realm_id = realm_id | |
| 127 | + self.access_token = access_token | |
| 128 | + self.base = self.SANDBOX_BASE if sandbox else self.PRODUCTION_BASE | |
| 129 | + | |
| 130 | + def create_journal_entry(self, body: dict[str, Any], request_id: str) -> dict[str, Any]: | |
| 131 | + url = ( | |
| 132 | + f"{self.base}/v3/company/{self.realm_id}/journalentry" | |
| 133 | + f"?minorversion={QBO_MINOR_VERSION}&requestid={request_id}" | |
| 134 | + ) | |
| 135 | + request = urllib.request.Request( | |
| 136 | + url, | |
| 137 | + data=_dumps_exact(body).encode("utf-8"), | |
| 138 | + headers={ | |
| 139 | + "Authorization": f"Bearer {self.access_token}", | |
| 140 | + "Content-Type": "application/json", | |
| 141 | + "Accept": "application/json", | |
| 142 | + }, | |
| 143 | + method="POST", | |
| 144 | + ) | |
| 145 | + with urllib.request.urlopen(request) as response: # pragma: no cover | |
| 146 | + payload = json.loads(response.read().decode("utf-8")) | |
| 147 | + return payload.get("JournalEntry", payload) # pragma: no cover | |
added
core/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
core/document_io.py
+43 −0
@@ -0,0 +1,43 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : document_io.py | |
| 6 | +# Description : Load/serialize AIR documents (YAML/JSON) with strict float rejection. | |
| 7 | +# ============================================================================= | |
| 8 | +"""AIR document I/O. | |
| 9 | + | |
| 10 | +AIR documents are YAML or JSON. All amounts, quantities, and rates MUST be | |
| 11 | +strings ("19.99") — bare numbers would arrive as binary floats and are | |
| 12 | +rejected by the schema (see core/events.py). This keeps every figure exact. | |
| 13 | +""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +import json | |
| 17 | +from pathlib import Path | |
| 18 | +from typing import Any | |
| 19 | + | |
| 20 | +import yaml | |
| 21 | + | |
| 22 | +from core.events import AirDocument | |
| 23 | + | |
| 24 | + | |
| 25 | +def load_air_document(path: str | Path) -> AirDocument: | |
| 26 | + path = Path(path) | |
| 27 | + text = path.read_text(encoding="utf-8") | |
| 28 | + data: Any | |
| 29 | + if path.suffix == ".json": | |
| 30 | + data = json.loads(text) | |
| 31 | + data.pop("_author", None) # header key, not part of the schema | |
| 32 | + else: | |
| 33 | + data = yaml.safe_load(text) | |
| 34 | + if not isinstance(data, dict): | |
| 35 | + raise ValueError(f"{path}: expected a mapping at top level") | |
| 36 | + return AirDocument.model_validate(data) | |
| 37 | + | |
| 38 | + | |
| 39 | +def dump_air_document(document: AirDocument) -> str: | |
| 40 | + """Canonical JSON serialization (amounts as strings).""" | |
| 41 | + payload = {"_author": "Simon-Pierre Boucher <contact@spboucher.ai>"} | |
| 42 | + payload.update(json.loads(document.model_dump_json(exclude_none=True))) | |
| 43 | + return json.dumps(payload, indent=2, default=str) | |
added
core/events.py
+178 −0
@@ -0,0 +1,178 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : events.py | |
| 6 | +# Description : AIR v0.1 — the EconomicEvent model (the language of accounting, not journal entries). | |
| 7 | +# ============================================================================= | |
| 8 | +"""AIR economic events. | |
| 9 | + | |
| 10 | +An AIR document describes WHAT HAPPENED economically ("3 chairs sold to | |
| 11 | +customer X, paid by Visa, delivery pending") — never the journal entries. | |
| 12 | +Journal entries are produced later, deterministically, by the AIC compiler. | |
| 13 | +This is the core separation: LLMs understand and emit AIR; the compiler | |
| 14 | +applies the rules. | |
| 15 | + | |
| 16 | +Floats are rejected everywhere an amount, quantity, or rate appears. | |
| 17 | +""" | |
| 18 | +from __future__ import annotations | |
| 19 | + | |
| 20 | +import enum | |
| 21 | +from datetime import date, datetime | |
| 22 | +from decimal import Decimal | |
| 23 | +from typing import Annotated, Any | |
| 24 | + | |
| 25 | +from pydantic import BaseModel, BeforeValidator, ConfigDict, Field | |
| 26 | + | |
| 27 | +from core.money import Money | |
| 28 | + | |
| 29 | +AIR_VERSION = "0.1" | |
| 30 | + | |
| 31 | + | |
| 32 | +def _reject_float(v: Any) -> Any: | |
| 33 | + if isinstance(v, float): | |
| 34 | + raise ValueError( | |
| 35 | + "floats are forbidden for amounts/quantities/rates in AIR; " | |
| 36 | + "use a JSON string or integer (e.g. \"19.99\")" | |
| 37 | + ) | |
| 38 | + return v | |
| 39 | + | |
| 40 | + | |
| 41 | +# Exact decimal that refuses to be built from a float. | |
| 42 | +ExactDecimal = Annotated[Decimal, BeforeValidator(_reject_float)] | |
| 43 | + | |
| 44 | + | |
| 45 | +class _AirModel(BaseModel): | |
| 46 | + model_config = ConfigDict(frozen=True, extra="forbid") | |
| 47 | + | |
| 48 | + | |
| 49 | +class EventType(str, enum.Enum): | |
| 50 | + SALE = "Sale" | |
| 51 | + PURCHASE = "Purchase" | |
| 52 | + REFUND = "Refund" # refund issued to a customer (reverses a Sale) | |
| 53 | + PAYMENT_RECEIVED = "PaymentReceived" # settles a receivable | |
| 54 | + PAYMENT_SENT = "PaymentSent" # settles a payable | |
| 55 | + OWNER_CONTRIBUTION = "OwnerContribution" | |
| 56 | + LOAN_RECEIVED = "LoanReceived" | |
| 57 | + | |
| 58 | + | |
| 59 | +class AmountSpec(_AirModel): | |
| 60 | + """An amount as it appears in AIR documents (exact decimal + currency).""" | |
| 61 | + | |
| 62 | + amount: ExactDecimal | |
| 63 | + currency: str | |
| 64 | + | |
| 65 | + def to_money(self) -> Money: | |
| 66 | + return Money(self.amount, self.currency) | |
| 67 | + | |
| 68 | + | |
| 69 | +class LineItem(_AirModel): | |
| 70 | + sku: str | None = None | |
| 71 | + description: str | None = None | |
| 72 | + qty: ExactDecimal = Decimal(1) | |
| 73 | + unit_price: AmountSpec | |
| 74 | + | |
| 75 | + | |
| 76 | +class PaymentInfo(_AirModel): | |
| 77 | + method: str | None = None # e.g. "card.visa", "bank_transfer", "cash" | |
| 78 | + gross: AmountSpec | None = None | |
| 79 | + immediate: bool = False # True: cash settles at event date (no AR/AP) | |
| 80 | + | |
| 81 | + | |
| 82 | +class DeliveryInfo(_AirModel): | |
| 83 | + status: str = "pending" # pending | delivered | partial | |
| 84 | + expected: date | None = None | |
| 85 | + | |
| 86 | + | |
| 87 | +class TaxContext(_AirModel): | |
| 88 | + """Where the supply takes place. Rates NEVER live here — they live in | |
| 89 | + versioned ALSL policies; the compiler resolves jurisdiction → policy.""" | |
| 90 | + | |
| 91 | + jurisdiction: str # e.g. "CA-QC", "CA-ON", "CA-AB" | |
| 92 | + codes: tuple[str, ...] = () # optional explicit tax codes, e.g. ("GST", "QST") | |
| 93 | + exempt: bool = False # zero-rated / exempt supply | |
| 94 | + | |
| 95 | + | |
| 96 | +class FxInfo(_AirModel): | |
| 97 | + """Observed FX data attached to the event (input data, not a rule). | |
| 98 | + | |
| 99 | + The rate is a fact about the world (e.g. Bank of Canada daily rate on the | |
| 100 | + transaction date); it is provided by ingestion, never hardcoded in AIR/AIC. | |
| 101 | + """ | |
| 102 | + | |
| 103 | + rate: ExactDecimal # units of functional currency per 1 unit of event currency | |
| 104 | + source: str # e.g. "bankofcanada.valet:FXUSDCAD" | |
| 105 | + rate_date: date | |
| 106 | + | |
| 107 | + | |
| 108 | +class SourceInfo(_AirModel): | |
| 109 | + kind: str | None = None # invoice_pdf | email | bank_feed | pos | api | manual | |
| 110 | + uri: str | None = None | |
| 111 | + ocr_score: ExactDecimal | None = None | |
| 112 | + | |
| 113 | + | |
| 114 | +class LlmInfo(_AirModel): | |
| 115 | + model: str | None = None | |
| 116 | + confidence: ExactDecimal | None = None | |
| 117 | + reasoning_hash: str | None = None | |
| 118 | + | |
| 119 | + | |
| 120 | +class Timestamps(_AirModel): | |
| 121 | + ingested: datetime | None = None | |
| 122 | + approved: datetime | None = None | |
| 123 | + | |
| 124 | + | |
| 125 | +class Meta(_AirModel): | |
| 126 | + source: SourceInfo | None = None | |
| 127 | + llm: LlmInfo | None = None | |
| 128 | + policy_version: str | None = None | |
| 129 | + timestamps: Timestamps | None = None | |
| 130 | + approver: str | None = None | |
| 131 | + | |
| 132 | + | |
| 133 | +class EconomicEvent(_AirModel): | |
| 134 | + """A single economic event — the atom of the AIR language.""" | |
| 135 | + | |
| 136 | + air_version: str = AIR_VERSION | |
| 137 | + id: str # ULID/unique id, e.g. "evt_01H..." | |
| 138 | + type: EventType | |
| 139 | + date: date | |
| 140 | + description: str | None = None | |
| 141 | + parties: dict[str, str] = Field(default_factory=dict) # role -> entity id | |
| 142 | + items: tuple[LineItem, ...] = () | |
| 143 | + amount: AmountSpec | None = None # for item-less events (payments, loans, ...) | |
| 144 | + payment: PaymentInfo | None = None | |
| 145 | + delivery: DeliveryInfo | None = None | |
| 146 | + tax: TaxContext | None = None | |
| 147 | + fx: FxInfo | None = None | |
| 148 | + related_event: str | None = None # e.g. the Sale a Refund reverses | |
| 149 | + meta: Meta | None = None | |
| 150 | + | |
| 151 | + def currency(self) -> str | None: | |
| 152 | + """The event's transaction currency, inferred from its amounts.""" | |
| 153 | + if self.items: | |
| 154 | + return self.items[0].unit_price.currency | |
| 155 | + if self.amount is not None: | |
| 156 | + return self.amount.currency | |
| 157 | + if self.payment is not None and self.payment.gross is not None: | |
| 158 | + return self.payment.gross.currency | |
| 159 | + return None | |
| 160 | + | |
| 161 | + def subtotal(self) -> Money | None: | |
| 162 | + """Sum of item lines (unrounded), or the flat amount if item-less.""" | |
| 163 | + if self.items: | |
| 164 | + total: Money | None = None | |
| 165 | + for item in self.items: | |
| 166 | + line = item.unit_price.to_money().multiply(item.qty) | |
| 167 | + total = line if total is None else total + line | |
| 168 | + return total | |
| 169 | + if self.amount is not None: | |
| 170 | + return self.amount.to_money() | |
| 171 | + return None | |
| 172 | + | |
| 173 | + | |
| 174 | +class AirDocument(_AirModel): | |
| 175 | + """A batch of economic events (the compilation unit's input).""" | |
| 176 | + | |
| 177 | + air_version: str = AIR_VERSION | |
| 178 | + events: tuple[EconomicEvent, ...] = () | |
added
core/invariant.py
+116 −0
@@ -0,0 +1,116 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : invariant.py | |
| 6 | +# Description : The accounting verifier — double-entry balance checked after every pass. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Accounting invariants (AIR's equivalent of the LLVM IR verifier). | |
| 9 | + | |
| 10 | +Two levels, both hard errors when violated: | |
| 11 | + | |
| 12 | +1. Entry level — every journal entry balances per currency: | |
| 13 | + sum(debits) == sum(credits). This is double-entry itself. | |
| 14 | +2. Ledger level — the accounting equation: | |
| 15 | + Assets = Liabilities + Equity + (Revenue - Expenses) | |
| 16 | + computed from typed account balances across all entries. | |
| 17 | + | |
| 18 | +The pass manager runs these after EVERY pass that touches journal entries; | |
| 19 | +a violation aborts compilation with a clang-style diagnostic. | |
| 20 | +""" | |
| 21 | +from __future__ import annotations | |
| 22 | + | |
| 23 | +from collections import defaultdict | |
| 24 | +from decimal import Decimal | |
| 25 | + | |
| 26 | +from aic.diagnostics import Diagnostic, Severity | |
| 27 | +from core.journal import AccountType, JournalEntry, Side | |
| 28 | + | |
| 29 | + | |
| 30 | +def entry_imbalances(entry: JournalEntry) -> dict[str, Decimal]: | |
| 31 | + """Per-currency imbalance (debits - credits). Empty dict == balanced.""" | |
| 32 | + balance: dict[str, Decimal] = defaultdict(Decimal) | |
| 33 | + for line in entry.lines: | |
| 34 | + balance[line.amount.currency] += line.signed() | |
| 35 | + return {ccy: diff for ccy, diff in balance.items() if diff != 0} | |
| 36 | + | |
| 37 | + | |
| 38 | +def verify_entries(entries: list[JournalEntry], pass_name: str) -> list[Diagnostic]: | |
| 39 | + """Entry-level double-entry check. Returns error diagnostics (empty == OK).""" | |
| 40 | + diags: list[Diagnostic] = [] | |
| 41 | + for entry in entries: | |
| 42 | + if not entry.lines: | |
| 43 | + diags.append(Diagnostic( | |
| 44 | + code="AIR-E100", | |
| 45 | + severity=Severity.ERROR, | |
| 46 | + message=f"journal entry '{entry.id}' has no lines", | |
| 47 | + location=f"entry {entry.id} (event {entry.source_event_id})", | |
| 48 | + suggestion="the posting pass must emit at least two lines per entry", | |
| 49 | + origin_pass=pass_name, | |
| 50 | + )) | |
| 51 | + continue | |
| 52 | + for ccy, diff in entry_imbalances(entry).items(): | |
| 53 | + diags.append(Diagnostic( | |
| 54 | + code="AIR-E101", | |
| 55 | + severity=Severity.ERROR, | |
| 56 | + message=( | |
| 57 | + f"journal entry '{entry.id}' is unbalanced in {ccy}: " | |
| 58 | + f"debits - credits = {diff}" | |
| 59 | + ), | |
| 60 | + location=f"entry {entry.id} (event {entry.source_event_id})", | |
| 61 | + suggestion="every entry must satisfy sum(debits) == sum(credits) per currency", | |
| 62 | + origin_pass=pass_name, | |
| 63 | + )) | |
| 64 | + return diags | |
| 65 | + | |
| 66 | + | |
| 67 | +def balances_by_type(entries: list[JournalEntry]) -> dict[AccountType, dict[str, Decimal]]: | |
| 68 | + """Normal-side balances per account type per currency. | |
| 69 | + | |
| 70 | + Asset/expense balances are debit-positive; liability/equity/revenue | |
| 71 | + balances are credit-positive. | |
| 72 | + """ | |
| 73 | + out: dict[AccountType, dict[str, Decimal]] = { | |
| 74 | + t: defaultdict(Decimal) for t in AccountType | |
| 75 | + } | |
| 76 | + for entry in entries: | |
| 77 | + for line in entry.lines: | |
| 78 | + sign = 1 if line.side is line.account.type.normal_side else -1 | |
| 79 | + out[line.account.type][line.amount.currency] += sign * line.amount.amount | |
| 80 | + return out | |
| 81 | + | |
| 82 | + | |
| 83 | +def accounting_equation_residual(entries: list[JournalEntry]) -> dict[str, Decimal]: | |
| 84 | + """Assets - (Liabilities + Equity + Revenue - Expenses), per currency. | |
| 85 | + | |
| 86 | + Zero everywhere iff the ledger satisfies the accounting equation. | |
| 87 | + """ | |
| 88 | + b = balances_by_type(entries) | |
| 89 | + currencies = {ccy for per_ccy in b.values() for ccy in per_ccy} | |
| 90 | + residual: dict[str, Decimal] = {} | |
| 91 | + for ccy in currencies: | |
| 92 | + assets = b[AccountType.ASSET][ccy] | |
| 93 | + liabilities = b[AccountType.LIABILITY][ccy] | |
| 94 | + equity = b[AccountType.EQUITY][ccy] | |
| 95 | + revenue = b[AccountType.REVENUE][ccy] | |
| 96 | + expenses = b[AccountType.EXPENSE][ccy] | |
| 97 | + residual[ccy] = assets - (liabilities + equity + revenue - expenses) | |
| 98 | + return {ccy: r for ccy, r in residual.items() if r != 0} | |
| 99 | + | |
| 100 | + | |
| 101 | +def verify_equation(entries: list[JournalEntry], pass_name: str) -> list[Diagnostic]: | |
| 102 | + """Ledger-level accounting-equation check.""" | |
| 103 | + diags: list[Diagnostic] = [] | |
| 104 | + for ccy, res in accounting_equation_residual(entries).items(): | |
| 105 | + diags.append(Diagnostic( | |
| 106 | + code="AIR-E102", | |
| 107 | + severity=Severity.ERROR, | |
| 108 | + message=( | |
| 109 | + f"accounting equation violated in {ccy}: " | |
| 110 | + f"Assets - (Liabilities + Equity + Revenue - Expenses) = {res}" | |
| 111 | + ), | |
| 112 | + location="ledger", | |
| 113 | + suggestion="an entry posted to a mistyped account or an unbalanced entry slipped through", | |
| 114 | + origin_pass=pass_name, | |
| 115 | + )) | |
| 116 | + return diags | |
added
core/journal.py
+90 −0
@@ -0,0 +1,90 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : journal.py | |
| 6 | +# Description : Compiled output types — accounts, journal lines/entries, compiled journal. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Journal types: what the AIC compiler PRODUCES (never what the LLM writes). | |
| 9 | + | |
| 10 | +Every journal line carries a provenance node id, so any posted figure can be | |
| 11 | +traced back through tax/FX derivations to the source economic event and its | |
| 12 | +document. | |
| 13 | +""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +import enum | |
| 17 | +from dataclasses import dataclass, field | |
| 18 | +from datetime import date | |
| 19 | +from decimal import Decimal | |
| 20 | + | |
| 21 | +from core.money import Money | |
| 22 | +from core.provenance import ProvenanceGraph | |
| 23 | + | |
| 24 | + | |
| 25 | +class AccountType(str, enum.Enum): | |
| 26 | + ASSET = "asset" | |
| 27 | + LIABILITY = "liability" | |
| 28 | + EQUITY = "equity" | |
| 29 | + REVENUE = "revenue" | |
| 30 | + EXPENSE = "expense" | |
| 31 | + | |
| 32 | + @property | |
| 33 | + def normal_side(self) -> "Side": | |
| 34 | + if self in (AccountType.ASSET, AccountType.EXPENSE): | |
| 35 | + return Side.DEBIT | |
| 36 | + return Side.CREDIT | |
| 37 | + | |
| 38 | + | |
| 39 | +class Side(str, enum.Enum): | |
| 40 | + DEBIT = "debit" | |
| 41 | + CREDIT = "credit" | |
| 42 | + | |
| 43 | + | |
| 44 | +@dataclass(frozen=True, slots=True) | |
| 45 | +class Account: | |
| 46 | + code: str | |
| 47 | + name: str | |
| 48 | + type: AccountType | |
| 49 | + | |
| 50 | + | |
| 51 | +@dataclass(frozen=True, slots=True) | |
| 52 | +class JournalLine: | |
| 53 | + account: Account | |
| 54 | + side: Side | |
| 55 | + amount: Money # always >= 0; direction is carried by `side` | |
| 56 | + memo: str = "" | |
| 57 | + provenance_id: str | None = None | |
| 58 | + | |
| 59 | + def signed(self) -> Decimal: | |
| 60 | + """Debit-positive signed amount (for balance math).""" | |
| 61 | + return self.amount.amount if self.side is Side.DEBIT else -self.amount.amount | |
| 62 | + | |
| 63 | + | |
| 64 | +@dataclass(frozen=True, slots=True) | |
| 65 | +class JournalEntry: | |
| 66 | + id: str | |
| 67 | + date: date | |
| 68 | + description: str | |
| 69 | + lines: tuple[JournalLine, ...] | |
| 70 | + source_event_id: str | |
| 71 | + policy_set: str = "" | |
| 72 | + policy_version: str = "" | |
| 73 | + reverses: str | None = None # id of the entry this one reverses, if any | |
| 74 | + | |
| 75 | + | |
| 76 | +@dataclass | |
| 77 | +class CompiledJournal: | |
| 78 | + """The result of one AIC compilation: entries + full provenance.""" | |
| 79 | + | |
| 80 | + entries: list[JournalEntry] = field(default_factory=list) | |
| 81 | + provenance: ProvenanceGraph = field(default_factory=ProvenanceGraph) | |
| 82 | + policy_set: str = "" | |
| 83 | + policy_version: str = "" | |
| 84 | + | |
| 85 | + def accounts(self) -> dict[str, Account]: | |
| 86 | + out: dict[str, Account] = {} | |
| 87 | + for entry in self.entries: | |
| 88 | + for line in entry.lines: | |
| 89 | + out[line.account.code] = line.account | |
| 90 | + return out | |
added
core/money.py
+121 −0
@@ -0,0 +1,121 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : money.py | |
| 6 | +# Description : Exact decimal Money type — floats are forbidden for amounts. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Money: exact decimal amounts with an ISO 4217 currency code. | |
| 9 | + | |
| 10 | +Rules (non-negotiable, see CLAUDE.md §5): | |
| 11 | +- Amounts are NEVER floats. Construction from float raises TypeError. | |
| 12 | +- Arithmetic across currencies raises CurrencyMismatchError. | |
| 13 | +- Rounding is explicit: a RoundingMode is always named by the caller | |
| 14 | + (policies decide the mode per jurisdiction; nothing is implicit). | |
| 15 | +""" | |
| 16 | +from __future__ import annotations | |
| 17 | + | |
| 18 | +import enum | |
| 19 | +from dataclasses import dataclass | |
| 20 | +from decimal import ROUND_HALF_EVEN, ROUND_HALF_UP, Decimal | |
| 21 | + | |
| 22 | +# ISO 4217 minor-unit exponents for the currencies AIR handles today. | |
| 23 | +# Extend as needed; unknown currencies default to 2 decimal places. | |
| 24 | +CURRENCY_EXPONENTS: dict[str, int] = { | |
| 25 | + "CAD": 2, "USD": 2, "EUR": 2, "GBP": 2, "AUD": 2, "CHF": 2, | |
| 26 | + "JPY": 0, "KRW": 0, | |
| 27 | +} | |
| 28 | + | |
| 29 | + | |
| 30 | +class RoundingMode(enum.Enum): | |
| 31 | + """Named rounding modes; the mode applied is always a policy decision.""" | |
| 32 | + | |
| 33 | + HALF_UP = "half_up" # 0.5 rounds away from zero (common on tax lines) | |
| 34 | + HALF_EVEN = "half_even" # banker's rounding | |
| 35 | + | |
| 36 | + @property | |
| 37 | + def decimal_mode(self) -> str: | |
| 38 | + return ROUND_HALF_UP if self is RoundingMode.HALF_UP else ROUND_HALF_EVEN | |
| 39 | + | |
| 40 | + | |
| 41 | +class CurrencyMismatchError(ValueError): | |
| 42 | + """Raised when arithmetic mixes two different currencies.""" | |
| 43 | + | |
| 44 | + | |
| 45 | +def _to_decimal(value: Decimal | int | str) -> Decimal: | |
| 46 | + if isinstance(value, float): | |
| 47 | + raise TypeError( | |
| 48 | + "Money amounts must not be floats. Pass a Decimal, int, or string " | |
| 49 | + "(e.g. Decimal('19.99') or '19.99')." | |
| 50 | + ) | |
| 51 | + if isinstance(value, Decimal): | |
| 52 | + return value | |
| 53 | + if isinstance(value, (int, str)): | |
| 54 | + return Decimal(value) | |
| 55 | + raise TypeError(f"Unsupported amount type: {type(value).__name__}") | |
| 56 | + | |
| 57 | + | |
| 58 | +@dataclass(frozen=True, slots=True) | |
| 59 | +class Money: | |
| 60 | + """An exact amount in a single currency. Immutable.""" | |
| 61 | + | |
| 62 | + amount: Decimal | |
| 63 | + currency: str | |
| 64 | + | |
| 65 | + def __post_init__(self) -> None: | |
| 66 | + object.__setattr__(self, "amount", _to_decimal(self.amount)) | |
| 67 | + if not (isinstance(self.currency, str) and len(self.currency) == 3 | |
| 68 | + and self.currency.isalpha() and self.currency.isupper()): | |
| 69 | + raise ValueError(f"Invalid ISO 4217 currency code: {self.currency!r}") | |
| 70 | + | |
| 71 | + # --- arithmetic (same-currency only) ------------------------------------- | |
| 72 | + def _check(self, other: "Money") -> None: | |
| 73 | + if self.currency != other.currency: | |
| 74 | + raise CurrencyMismatchError( | |
| 75 | + f"Cannot combine {self.currency} with {other.currency}; " | |
| 76 | + "convert explicitly through the FX pass first." | |
| 77 | + ) | |
| 78 | + | |
| 79 | + def __add__(self, other: "Money") -> "Money": | |
| 80 | + self._check(other) | |
| 81 | + return Money(self.amount + other.amount, self.currency) | |
| 82 | + | |
| 83 | + def __sub__(self, other: "Money") -> "Money": | |
| 84 | + self._check(other) | |
| 85 | + return Money(self.amount - other.amount, self.currency) | |
| 86 | + | |
| 87 | + def __neg__(self) -> "Money": | |
| 88 | + return Money(-self.amount, self.currency) | |
| 89 | + | |
| 90 | + def multiply(self, factor: Decimal | int | str) -> "Money": | |
| 91 | + """Unrounded multiplication (e.g. qty × unit price, rate × base). | |
| 92 | + | |
| 93 | + The result keeps full precision; call .quantized() when a policy | |
| 94 | + says the amount becomes a posted figure. | |
| 95 | + """ | |
| 96 | + return Money(self.amount * _to_decimal(factor), self.currency) | |
| 97 | + | |
| 98 | + # --- rounding ------------------------------------------------------------- | |
| 99 | + def exponent(self) -> int: | |
| 100 | + return CURRENCY_EXPONENTS.get(self.currency, 2) | |
| 101 | + | |
| 102 | + def quantized(self, mode: RoundingMode) -> "Money": | |
| 103 | + """Round to the currency's minor unit using an explicit, named mode.""" | |
| 104 | + quantum = Decimal(1).scaleb(-self.exponent()) | |
| 105 | + return Money(self.amount.quantize(quantum, rounding=mode.decimal_mode), | |
| 106 | + self.currency) | |
| 107 | + | |
| 108 | + # --- predicates / display --------------------------------------------------- | |
| 109 | + def is_zero(self) -> bool: | |
| 110 | + return self.amount == 0 | |
| 111 | + | |
| 112 | + def is_negative(self) -> bool: | |
| 113 | + return self.amount < 0 | |
| 114 | + | |
| 115 | + def __str__(self) -> str: | |
| 116 | + return f"{self.amount} {self.currency}" | |
| 117 | + | |
| 118 | + | |
| 119 | +def money(amount: Decimal | int | str, currency: str) -> Money: | |
| 120 | + """Convenience constructor: money('19.99', 'CAD').""" | |
| 121 | + return Money(_to_decimal(amount), currency) | |
added
core/provenance.py
+117 −0
@@ -0,0 +1,117 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : provenance.py | |
| 6 | +# Description : Immutable provenance graph — accounting SSA: every amount has one traceable origin. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Provenance graph: the accounting analogue of SSA form. | |
| 9 | + | |
| 10 | +In LLVM's SSA form every value is defined exactly once and every use points | |
| 11 | +back to its unique definition. AIR transposes this to money: every amount that | |
| 12 | +appears anywhere in a compilation (an invoice line, a tax amount, a converted | |
| 13 | +FX amount, a posted journal line) is a *node* defined exactly once, and every | |
| 14 | +derived amount points to the node(s) it was computed from, together with the | |
| 15 | +operation that produced it (e.g. "tax:GST@0.05", "fx:USD->CAD@1.3500"). | |
| 16 | + | |
| 17 | +The graph is append-only and nodes are immutable: corrections never rewrite | |
| 18 | +history, they add new nodes (mirroring reversal entries in the ledger). | |
| 19 | +""" | |
| 20 | +from __future__ import annotations | |
| 21 | + | |
| 22 | +import itertools | |
| 23 | +from dataclasses import dataclass, field | |
| 24 | +from typing import Iterator | |
| 25 | + | |
| 26 | +from core.money import Money | |
| 27 | + | |
| 28 | + | |
| 29 | +@dataclass(frozen=True, slots=True) | |
| 30 | +class ProvenanceNode: | |
| 31 | + """A single, immutable definition of an amount.""" | |
| 32 | + | |
| 33 | + id: str | |
| 34 | + kind: str # e.g. "invoice_line", "tax", "fx", "journal_line" | |
| 35 | + operation: str # human/machine readable derivation, e.g. "tax:GST@0.05" | |
| 36 | + amount: Money | |
| 37 | + inputs: tuple[str, ...] = () # ids of the nodes this amount was derived from | |
| 38 | + source_ref: str | None = None # external anchor: event id, document URI, ... | |
| 39 | + | |
| 40 | + | |
| 41 | +class ProvenanceError(KeyError): | |
| 42 | + """Raised when a node id is unknown or redefined (SSA violation).""" | |
| 43 | + | |
| 44 | + | |
| 45 | +@dataclass | |
| 46 | +class ProvenanceGraph: | |
| 47 | + """Append-only DAG of amount definitions.""" | |
| 48 | + | |
| 49 | + _nodes: dict[str, ProvenanceNode] = field(default_factory=dict) | |
| 50 | + _counter: itertools.count = field(default_factory=itertools.count) | |
| 51 | + | |
| 52 | + def define( | |
| 53 | + self, | |
| 54 | + kind: str, | |
| 55 | + operation: str, | |
| 56 | + amount: Money, | |
| 57 | + inputs: tuple[str, ...] = (), | |
| 58 | + source_ref: str | None = None, | |
| 59 | + ) -> ProvenanceNode: | |
| 60 | + """Define a new amount (exactly once — ids are generated, never reused).""" | |
| 61 | + for parent in inputs: | |
| 62 | + if parent not in self._nodes: | |
| 63 | + raise ProvenanceError(f"Unknown provenance input: {parent}") | |
| 64 | + node = ProvenanceNode( | |
| 65 | + id=f"prov_{next(self._counter):06d}", | |
| 66 | + kind=kind, | |
| 67 | + operation=operation, | |
| 68 | + amount=amount, | |
| 69 | + inputs=inputs, | |
| 70 | + source_ref=source_ref, | |
| 71 | + ) | |
| 72 | + self._nodes[node.id] = node | |
| 73 | + return node | |
| 74 | + | |
| 75 | + def get(self, node_id: str) -> ProvenanceNode: | |
| 76 | + try: | |
| 77 | + return self._nodes[node_id] | |
| 78 | + except KeyError: | |
| 79 | + raise ProvenanceError(f"Unknown provenance node: {node_id}") from None | |
| 80 | + | |
| 81 | + def trace(self, node_id: str) -> list[ProvenanceNode]: | |
| 82 | + """Full ancestry of a node (the node first, origins last), depth-first.""" | |
| 83 | + seen: set[str] = set() | |
| 84 | + out: list[ProvenanceNode] = [] | |
| 85 | + | |
| 86 | + def walk(nid: str) -> None: | |
| 87 | + if nid in seen: | |
| 88 | + return | |
| 89 | + seen.add(nid) | |
| 90 | + node = self.get(nid) | |
| 91 | + out.append(node) | |
| 92 | + for parent in node.inputs: | |
| 93 | + walk(parent) | |
| 94 | + | |
| 95 | + walk(node_id) | |
| 96 | + return out | |
| 97 | + | |
| 98 | + def __len__(self) -> int: | |
| 99 | + return len(self._nodes) | |
| 100 | + | |
| 101 | + def __iter__(self) -> Iterator[ProvenanceNode]: | |
| 102 | + return iter(self._nodes.values()) | |
| 103 | + | |
| 104 | + def to_dicts(self) -> list[dict[str, object]]: | |
| 105 | + """Serialize for audit export (amounts as strings, never floats).""" | |
| 106 | + return [ | |
| 107 | + { | |
| 108 | + "id": n.id, | |
| 109 | + "kind": n.kind, | |
| 110 | + "operation": n.operation, | |
| 111 | + "amount": str(n.amount.amount), | |
| 112 | + "currency": n.amount.currency, | |
| 113 | + "inputs": list(n.inputs), | |
| 114 | + "source_ref": n.source_ref, | |
| 115 | + } | |
| 116 | + for n in self._nodes.values() | |
| 117 | + ] | |
added
docs/adr/0001-implementation-language.md
+44 −0
@@ -0,0 +1,44 @@ | ||
| 1 | +<!-- | |
| 2 | +Projet : AIR — Accounting Intermediate Representation | |
| 3 | +Auteur : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +Fichier : 0001-implementation-language.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# ADR 0001 — Implementation language: strictly-typed Python first, Rust later if needed | |
| 9 | + | |
| 10 | +- **Status**: Accepted — 2026-08-05 | |
| 11 | +- **Context**: CLAUDE.md §4 requires choosing between Rust and strictly-typed | |
| 12 | + Python after comparative research (see docs/research/ledger-engines.md and | |
| 13 | + llvm-architecture.md). | |
| 14 | + | |
| 15 | +## Decision | |
| 16 | + | |
| 17 | +Phase 1–4 are implemented in **Python 3.11+ with strict typing** (Pydantic v2 | |
| 18 | +models, `mypy --strict`-compatible code, frozen dataclasses). A Rust port of | |
| 19 | +the hot core (AIC passes + invariant verifier) remains an explicit later | |
| 20 | +option once the spec stabilizes. | |
| 21 | + | |
| 22 | +## Rationale | |
| 23 | + | |
| 24 | +1. **Correctness needs here are logical, not mechanical.** The invariants that | |
| 25 | + matter (double-entry balance, exact decimals, determinism) are enforced by | |
| 26 | + design (Decimal everywhere, floats rejected at every boundary, verifier | |
| 27 | + after every pass) — not by the borrow checker. TigerBeetle-class problems | |
| 28 | + (throughput, crash-safety under a million TPS) are not Phase 1 problems. | |
| 29 | +2. **Iteration speed on a moving spec.** AIR/ALSL formats will churn during | |
| 30 | + Phases 1–3; Python + Pydantic gives schema evolution, JSON Schema | |
| 31 | + generation (`model_json_schema()`), and golden-test iteration far faster. | |
| 32 | +3. **Ecosystem adjacency.** Ingestion (Phase 4) is LLM/OCR tooling, which is | |
| 33 | + Python-native; the property-testing story (hypothesis) is mature. | |
| 34 | +4. **Exactness parity.** Python's `decimal.Decimal` provides the same | |
| 35 | + fixed-point guarantees as `rust_decimal`; both round via explicit modes. | |
| 36 | + | |
| 37 | +## Consequences | |
| 38 | + | |
| 39 | +- JSON Schema is generated from the Pydantic types today; when the schema | |
| 40 | + stabilizes, the direction flips (schema-first, generated types) per | |
| 41 | + CLAUDE.md §3.1. | |
| 42 | +- Performance-critical kernels (batch compilation, reconciliation) may move | |
| 43 | + to Rust behind the same AIR/ALSL contracts; the format, not the code, is | |
| 44 | + the interface (see docs/research/llvm-architecture.md, lesson 1). | |
added
docs/adr/0002-economic-event-model.md
+43 −0
@@ -0,0 +1,43 @@ | ||
| 1 | +<!-- | |
| 2 | +Projet : AIR — Accounting Intermediate Representation | |
| 3 | +Auteur : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +Fichier : 0002-economic-event-model.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# ADR 0002 — AIR adopts the REA economic-event model; existing standards are import/export mappings | |
| 9 | + | |
| 10 | +- **Status**: Accepted — 2026-08-05 | |
| 11 | +- **Context**: docs/research/accounting-data-standards.md surveyed XBRL-GL, | |
| 12 | + UBL/Peppol, ISO 20022 camt, OFX, plain-text accounting, REA (ISO 15944-4) | |
| 13 | + and ValueFlows. | |
| 14 | + | |
| 15 | +## Decision | |
| 16 | + | |
| 17 | +1. **AIR's core is an REA-style economic event** (Resources–Events–Agents, | |
| 18 | + McCarthy 1982; ISO/IEC 15944-4): AIR records that value moved between | |
| 19 | + agents — debits and credits are *derived views* computed by the compiler. | |
| 20 | + This is exactly AIR's thesis, so we adopt REA's conceptual vocabulary | |
| 21 | + (events, agents, resources; commitments planned for delivery/performance | |
| 22 | + obligations per IFRS 15). | |
| 23 | +2. **No existing wire format becomes the IR.** XBRL-GL is journal-level (too | |
| 24 | + late in the pipeline — it encodes the *output* of the decisions AIR wants | |
| 25 | + to compile), UBL is document-level (an *input*), camt.053 is bank-side | |
| 26 | + (an *input*), plain-text accounting is single-entity journal syntax. | |
| 27 | +3. **Standards become frontends and backends**: UBL/Peppol invoices and | |
| 28 | + camt.053 statements are ingestion frontends; XBRL-GL, UBL, and a | |
| 29 | + beancount/hledger text export are backend targets (the latter is | |
| 30 | + near-free and ideal for golden-test diffing). | |
| 31 | +4. **Perspective-neutrality** (from ValueFlows): one AIR event is | |
| 32 | + simultaneously the seller's sale and the buyer's purchase; which journal | |
| 33 | + is produced depends on the compiling entity's policy set. | |
| 34 | + | |
| 35 | +## Consequences | |
| 36 | + | |
| 37 | +- `EconomicEvent` (core/events.py, schemas/air-0.1.schema.json) carries | |
| 38 | + parties/items/payment/delivery/tax/fx/meta — no account codes, no | |
| 39 | + debit/credit anywhere in AIR itself. | |
| 40 | +- EN 16931's "one semantic model, multiple syntaxes" pattern guides the | |
| 41 | + backend layer: one CompiledJournal, many target payloads. | |
| 42 | +- A commitments field (REA duality/fulfillment) is planned for AIR v0.2 to | |
| 43 | + carry pending deliveries and revenue-recognition obligations. | |
added
docs/adr/0003-monetary-amounts.md
+40 −0
@@ -0,0 +1,40 @@ | ||
| 1 | +<!-- | |
| 2 | +Projet : AIR — Accounting Intermediate Representation | |
| 3 | +Auteur : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +Fichier : 0003-monetary-amounts.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# ADR 0003 — Monetary amounts: exact decimals everywhere, rounding is jurisdiction policy | |
| 9 | + | |
| 10 | +- **Status**: Accepted — 2026-08-05 | |
| 11 | +- **Context**: docs/research/ledger-engines.md (float hazards, TigerBeetle | |
| 12 | + fixed-point design) and docs/research/canada-gst-qst.md (statutory | |
| 13 | + rounding). | |
| 14 | + | |
| 15 | +## Decision | |
| 16 | + | |
| 17 | +1. **No floats, anywhere, ever.** In JSON/YAML, every amount, quantity, and | |
| 18 | + rate is a **string** ("19.99"); in code it is `decimal.Decimal` wrapped in | |
| 19 | + the `Money` type (core/money.py). Construction from `float` raises at | |
| 20 | + every boundary: `Money`, the Pydantic schema (`ExactDecimal`), and the | |
| 21 | + ALSL loader all reject floats independently. | |
| 22 | +2. **Rounding is a named, per-jurisdiction ALSL policy — not an engine | |
| 23 | + default.** Research falsified the assumption that banker's rounding is | |
| 24 | + universal: GST/QST rounding is **half-up** by statute (Excise Tax Act | |
| 25 | + s. 165.2(2): fractions < $0.005 disregarded, ≥ $0.005 deemed one cent; | |
| 26 | + Revenu Québec IN-203-V states the same for QST). `RoundingMode` therefore | |
| 27 | + supports `half_up` and `half_even`, and the ca-qc policy set selects | |
| 28 | + `half_up` with a source citation. | |
| 29 | +3. **Round late.** Intermediate computations (qty × unit price, base × rate, | |
| 30 | + amount × fx rate) keep full precision; quantization to the currency's | |
| 31 | + ISO 4217 minor unit happens once, when a figure becomes a posted amount — | |
| 32 | + and each quantization is recorded in the provenance graph's operation | |
| 33 | + string (e.g. `tax:QST@0.09975~half_up`). | |
| 34 | + | |
| 35 | +## Consequences | |
| 36 | + | |
| 37 | +- Cross-currency arithmetic is a hard error (`CurrencyMismatchError`); | |
| 38 | + conversion only happens in the FX pass, explicitly, with a cited rate. | |
| 39 | +- Determinism holds bit-for-bit: same document + same policies = identical | |
| 40 | + journal (property-tested in tests/property/). | |
added
docs/cli-reference.md
+87 −0
@@ -0,0 +1,87 @@ | ||
| 1 | +<!-- | |
| 2 | +Projet : AIR — Accounting Intermediate Representation | |
| 3 | +Auteur : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +Fichier : cli-reference.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# AIR CLI Reference | |
| 9 | + | |
| 10 | +Install with `pip install -e .` and the `air` command is on your PATH | |
| 11 | +(equivalently: `python -m sdk.cli`). | |
| 12 | + | |
| 13 | +Global concept: the **AIR home** (`--home <dir>`) is your managed books | |
| 14 | +directory — meta, hash-chained ledger, content-addressed document archive, | |
| 15 | +exports, and the review inbox. Most commands take `--home`; `--policies` | |
| 16 | +defaults to the home's configured policy set. | |
| 17 | + | |
| 18 | +## Lifecycle | |
| 19 | + | |
| 20 | +| Command | What it does | | |
| 21 | +|---|---| | |
| 22 | +| `air init --home books --policies alsl/policies/ca-qc-2026.yaml [--name x]` | Create a managed AIR home | | |
| 23 | +| `air status --home books` | Entries, chain validity, archives, exports | | |
| 24 | +| `air audit --home books` | Render + verify the hash-chained syscall audit log | | |
| 25 | + | |
| 26 | +## Compile & post | |
| 27 | + | |
| 28 | +| Command | What it does | | |
| 29 | +|---|---| | |
| 30 | +| `air compile doc.yaml --home books` | Compile AIR events → post to the native ledger; archives the document | | |
| 31 | +| `air compile doc.yaml --home books --optimize` | Same, with duplicate detection, netting, and payment fusion | | |
| 32 | +| `air compile doc.yaml --home books --report trial-balance --format markdown` | Post, then print statements | | |
| 33 | +| `air compile doc.yaml --home books --backend csv` | Export a universal CSV journal instead | | |
| 34 | +| `air compile doc.yaml --home books --backend qbo-export` | QuickBooks-shaped JSON export — **no QuickBooks account needed** | | |
| 35 | +| `air verify doc.yaml --policies <set>` | Compile without posting; print diagnostics | | |
| 36 | +| `air recompile old.yaml new.yaml --home books` | Incremental: diff by content fingerprint, post reversal + replacement entries only | | |
| 37 | + | |
| 38 | +## Statements | |
| 39 | + | |
| 40 | +| Command | What it does | | |
| 41 | +|---|---| | |
| 42 | +| `air report trial-balance --home books` | Also: `general-ledger`, `income-statement`, `balance-sheet` | | |
| 43 | +| `... --format text\|markdown\|csv\|json` | Any statement, four formats | | |
| 44 | + | |
| 45 | +## Ingestion (document → books) | |
| 46 | + | |
| 47 | +| Command | What it does | | |
| 48 | +|---|---| | |
| 49 | +| `air ingest invoice.txt --home books` | Extract events (offline mock extractor), validate against the AIR schema, route by confidence | | |
| 50 | +| `air ingest invoice.txt --home books --llm` | Use the Claude extractor (needs `pip install anthropic` + `ANTHROPIC_API_KEY`) | | |
| 51 | +| `air ingest ... --threshold 0.9` | Auto-approve threshold (default 0.85); schema-invalid always goes to a human | | |
| 52 | +| `air inbox --home books` | List extractions awaiting review | | |
| 53 | +| `air approve inbox_00000 --home books --approver name` | Stamp approver + timestamp, compile, post | | |
| 54 | +| `air reject inbox_00000 --home books --reason "duplicate"` | Reject (archived, never deleted) | | |
| 55 | + | |
| 56 | +## Bank reconciliation | |
| 57 | + | |
| 58 | +| Command | What it does | | |
| 59 | +|---|---| | |
| 60 | +| `air reconcile statement.xml --home books` | camt.053 / MT940 / CSV, auto-detected; exits 0 clean, 2 with differences | | |
| 61 | +| `... --cash-account 1000 --tolerance-days 3` | Matching knobs | | |
| 62 | + | |
| 63 | +## Agent surface (Python) | |
| 64 | + | |
| 65 | +```python | |
| 66 | +from sdk.syscalls import AirKernel | |
| 67 | + | |
| 68 | +k = AirKernel("books", actor="agent:alice") | |
| 69 | +k.create_economic_event({...}) # schema-validated draft | |
| 70 | +k.validate(); k.compile() # dry runs | |
| 71 | +k.post() # deterministic compile -> books (idempotent) | |
| 72 | +k.merge() # post with netting + fusion | |
| 73 | +k.reverse("je_evt_x") # contra entry, never an edit | |
| 74 | +k.close_period("2026-01") # later posts into it are refused (AIR-E700) | |
| 75 | +k.reconcile("statement.mt940") # bank matching | |
| 76 | +k.generate_report("balance-sheet", "markdown") | |
| 77 | +# every call above — success or refusal — is one hash-chained audit record | |
| 78 | +``` | |
| 79 | + | |
| 80 | +## Demo | |
| 81 | + | |
| 82 | +```bash | |
| 83 | +air init --home /tmp/demo --policies alsl/policies/ca-qc-2026.yaml | |
| 84 | +air compile tests/fixtures/demo_document.yaml --home /tmp/demo --report balance-sheet | |
| 85 | +python -m sdk.demo_agent --home /tmp/demo # scripted agent, syscalls only | |
| 86 | +air audit --home /tmp/demo | |
| 87 | +``` | |
added
docs/research/accounting-data-standards.md
+164 −0
@@ -0,0 +1,164 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : accounting-data-standards.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Research Report — Existing Accounting & Business Data Standards | |
| 9 | + | |
| 10 | +**Date of research / consultation of all sources: 2026-08-05** | |
| 11 | +**Purpose:** Survey prior art before designing the AIR format (CLAUDE.md §1.1, Phase 0). AIR models *economic events*, not journal entries; this report assesses which existing standards to align with, borrow from, or avoid. | |
| 12 | + | |
| 13 | +--- | |
| 14 | + | |
| 15 | +## 1. XBRL and XBRL-GL (Global Ledger) | |
| 16 | + | |
| 17 | +### 1.1 XBRL (financial reporting) | |
| 18 | + | |
| 19 | +XBRL (eXtensible Business Reporting Language) is the dominant global standard for *aggregated* financial **reporting** — regulator-facing filings (SEC, ESEF, EDINET). It is taxonomy-driven: facts are tagged against concepts defined in jurisdiction-specific taxonomies (US GAAP, IFRS). The SEC continues to publish annual taxonomy updates (2025 update confirmed). XBRL operates at the *report* level, downstream of everything AIR does. | |
| 20 | + | |
| 21 | +### 1.2 XBRL-GL (Global Ledger) | |
| 22 | + | |
| 23 | +XBRL-GL is a separate, generic, system-independent XML representation of **detailed ledger/transactional data** — journal entries, sub-ledger detail, master data — meant to bridge transactional systems and reporting standards. Its data model is a hierarchy of `accountingEntries → entryHeader → entryDetail`, organized in composable taxonomy modules (core `gl-cor`, business `gl-bus`, multicurrency `gl-muc`, US/UK extensions `gl-usk`, tax audit `gl-taf`). | |
| 24 | + | |
| 25 | +**Status (as of 2026-08-05):** The 2015 XBRL GL Taxonomy reached Recommendation status (stable) and remains listed among the specifications maintained by XBRL International, with a later Public Working Draft of an updated framework. However, real-world adoption is **narrow**: the most notable deployment is the Turkish government's mandated electronic bookkeeping (e-Defter) for tax purposes. Wikipedia and practitioner commentary note persistent barriers (producer incentives, lack of practical instruction sets). It is best described as *maintained but dormant/niche*, not a living ecosystem. | |
| 26 | + | |
| 27 | +**Sources (consulted 2026-08-05):** | |
| 28 | +- XBRL International — Global Ledger overview: https://www.xbrl.org/the-standard/what/global-ledger/ | |
| 29 | +- XBRL Specifications — Transactional Reporting (GL spec group): https://specifications.xbrl.org/transactional.html | |
| 30 | +- XBRL GL tag archive (status news): https://www.xbrl.org/tag/xbrl-gl/ | |
| 31 | +- XBRL Japan — What is XBRL GL: https://www.xbrl.or.jp/modules/pico7/index.php?content_id=8&ml_lang=en | |
| 32 | +- Wikipedia — XBRL GL (adoption, Turkey e-bookkeeping): https://en.wikipedia.org/wiki/XBRL_GL | |
| 33 | +- SEC 2025 taxonomy update: https://www.sec.gov/newsroom/whats-new/2503-2025-xbrl-taxonomies-update | |
| 34 | + | |
| 35 | +**Relevance to AIR:** | |
| 36 | +- **Borrow:** the idea of a system-independent transactional interchange layer sitting between ERPs and reporting; the modular taxonomy design (core + optional extension modules) is a good pattern for AIR schema layering. | |
| 37 | +- **Align (export only):** an XBRL-GL export backend is a credible long-term target (drill-down from XBRL reports to AIR provenance is exactly XBRL-GL's pitch). | |
| 38 | +- **Avoid:** adopting it as AIR's core format. It models *journal entries* (post-compilation artifacts in AIR terms), it is XML/taxonomy-heavy, tooling is scarce, and adoption never materialized. AIR's core must stay upstream of the journal entry. | |
| 39 | + | |
| 40 | +--- | |
| 41 | + | |
| 42 | +## 2. UBL (OASIS Universal Business Language) and Peppol / EN 16931 | |
| 43 | + | |
| 44 | +### 2.1 UBL | |
| 45 | + | |
| 46 | +UBL is an open, royalty-free, XML-based OASIS standard for business **documents**. **UBL 2.4** is the current OASIS Standard (published 2024-06-20), defining **94 document schemas** — Invoice, Credit Note, Debit Note, Self Billed Invoice/Credit Note, Order, Despatch Advice, Receipt Advice, Catalogue, Tender, transport documents, etc. Minor revisions are guaranteed backward compatible: documents conforming to UBL 2.1 remain valid under 2.4. The document model is built from reusable common aggregate components (Party, Item, TaxTotal/TaxSubtotal, MonetaryTotal, PaymentMeans, Delivery, AllowanceCharge), all with explicit currency-qualified amounts. | |
| 47 | + | |
| 48 | +### 2.2 Peppol BIS Billing 3.0 and EN 16931 | |
| 49 | + | |
| 50 | +Peppol (the pan-European — now global — e-procurement network) mandates **Peppol BIS Billing 3.0**, which uses **UBL 2.1** Invoice and CreditNote document types. BIS Billing 3.0 is a CIUS (Core Invoice Usage Specification) of **EN 16931**, the European semantic model for electronic invoices: compliance with BIS Billing implies EN 16931 compliance. Validation involves ~200 business rules (VAT consistency, mandatory fields, totals arithmetic). EN 16931 is significant because it is a *semantic* model with two syntax bindings (UBL and UN/CEFACT CII) — a semantics-first design AIR should emulate. | |
| 51 | + | |
| 52 | +**Sources (consulted 2026-08-05):** | |
| 53 | +- OASIS UBL 2.4 Standard: https://docs.oasis-open.org/ubl/UBL-2.4.html | |
| 54 | +- Peppol BIS Billing 3.0 specification: https://docs.peppol.eu/poacc/billing/3.0/bis/ | |
| 55 | +- Peppol document formats introduction (ionite, 2025): https://ionite.net/news-articles/2025-03-07_peppol_document_formats/ | |
| 56 | +- UBL invoice format guide / field reference: https://e-invoice.be/blog/ubl-format-guide | |
| 57 | +- Singapore BIS Billing (Peppol beyond EU): https://www.peppolguide.sg/billing/bis/ | |
| 58 | + | |
| 59 | +**Relevance to AIR:** | |
| 60 | +- **Borrow:** UBL/EN 16931's field-level vocabulary for invoices — party identification, tax category/subtotal breakdown, allowance/charge modeling, payment means codes, currency-qualified amount types. AIR's `Sale`/`Purchase` event items, tax blocks, and payment blocks should map cleanly onto these components so LLM extraction from Peppol invoices is near-lossless. | |
| 61 | +- **Align (import + export):** UBL invoices are a first-class *ingestion source* (structured, no OCR needed) and a natural export target. An AIR↔UBL mapping table belongs in the spec. | |
| 62 | +- **Avoid:** using UBL as the core IR. It is document-centric (an invoice is a *claim document*, not the economic event) and says nothing about ledger posting, provenance, or compilation. | |
| 63 | + | |
| 64 | +--- | |
| 65 | + | |
| 66 | +## 3. ISO 20022 (camt.053) and OFX — bank statement models | |
| 67 | + | |
| 68 | +### 3.1 ISO 20022 camt.053 | |
| 69 | + | |
| 70 | +`camt.053` (BankToCustomerStatement) is the ISO 20022 end-of-day bank statement message: a Group Header plus one or more Statements containing **balances** (opening/closing/available, coded `Bal/Tp`) and **entries** (`Ntry`), each with amount, credit/debit indicator, status, booking/value dates, bank transaction codes (domain/family/subfamily), and nested `EntryDetails/TransactionDetails` carrying end-to-end references, debtor/creditor parties and accounts, structured remittance information, purpose codes, and charge breakdowns. It is deeply hierarchical XML (~1,300 tags in v.001.02; later versions add more); this replaces the flat positional MT940 (SWIFT's ISO 20022 migration has made camt the strategic format). Related messages: `camt.052` (intraday report), `camt.054` (debit/credit notification — the reconciliation workhorse). | |
| 71 | + | |
| 72 | +### 3.2 OFX | |
| 73 | + | |
| 74 | +OFX (Open Financial Exchange, 1997) is the legacy North American consumer format for bank/card statement download (SGML then XML; `STMTTRN` records with `TRNTYPE`, `DTPOSTED`, `TRNAMT`, `FITID`, `NAME`/`MEMO`). Current release is **OFX 2.3** (split into OFX Banking 2.3 and OFX Tax). Since 2019 stewardship sits with the **Financial Data Exchange (FDX)** consortium; OFX is in maintenance mode while FDX's REST/JSON API is the modern successor for open-banking-style data sharing. In practice, aggregators (Plaid, Flinks) expose their own JSON models on top. | |
| 75 | + | |
| 76 | +**Sources (consulted 2026-08-05):** | |
| 77 | +- Betaalvereniging NL — camt.053 implementation guidelines (v1.1, 2026): https://www.betaalvereniging.nl/wp-content/uploads/2026/03/IG-Bank-to-Customer-Statement-CAMT-053-v1-1.pdf | |
| 78 | +- Bank of America — camt.053 reference guide: https://images.em.bankofamerica.com/GTS/ISO_20022/ReferenceGuideBanktoCustomerStatement(CAMT.053).pdf | |
| 79 | +- Finanssiala — ISO 20022 Account Statement Guide: https://www.finanssiala.fi/wp-content/uploads/2021/03/ISO-20022-Account-Statement-Guide-2020.pdf | |
| 80 | +- MT940 vs camt.053 comparison: https://invoicedataextraction.com/blog/mt940-camt053-bank-statement-format-guide | |
| 81 | +- FDX — OFX Work Group: https://financialdataexchange.org/about-fdx/ofx-work-group/ | |
| 82 | +- Wikipedia — Open Financial Exchange: https://en.wikipedia.org/wiki/Open_Financial_Exchange | |
| 83 | +- OFX→FDX evolution: https://ninth-wave.com/blog/the-evolution-of-data-sharing-part-1-from-ofx-foundation-to-today-fdx/ | |
| 84 | + | |
| 85 | +**Relevance to AIR:** | |
| 86 | +- **Borrow:** camt.053's separation of *entry* vs *transaction details*, its bank transaction code taxonomy (domain/family/subfamily) for classifying settlement events, and its end-to-end reference chain — directly useful for AIR's `Settlement` events and the Phase 6 reconciliation pass. | |
| 87 | +- **Align (import):** camt.053/camt.054 (and MT940 legacy, plus Plaid/Flinks JSON) are *ingestion sources* producing AIR `Settlement`/`BankEntry` events. OFX matters only as a legacy import for NA consumer data. | |
| 88 | +- **Avoid:** modeling AIR amounts or events on OFX's loose semantics (no double-entry, weak typing); avoid inheriting camt's XML verbosity — map its semantics, not its syntax. | |
| 89 | + | |
| 90 | +--- | |
| 91 | + | |
| 92 | +## 4. Plain-Text Accounting: ledger-cli, hledger, beancount | |
| 93 | + | |
| 94 | +All three share the core model: a text file of dated **transactions**, each a list of **postings** (account, amount+commodity) that must balance to zero — i.e., double-entry with multi-commodity support and first-class cost/price annotations (`@`, `@@`, `{cost}` lots). | |
| 95 | + | |
| 96 | +- **ledger-cli** (C++): the original; fastest; accounts created implicitly on use; very flexible expression language; loosest validation. | |
| 97 | +- **hledger** (Haskell): compatible middle ground; more structure, strict mode available, official web UI, strong CSV import rules. | |
| 98 | +- **beancount** (Python, v3 current): the strictest — mandatory `YYYY-MM-DD open Account` directives (typo-proof accounts), a small closed set of directives (`open`, `close`, `balance` assertions, `pad`, `price`, `commodity`, `event`, `document`, `note`, `custom`), full re-derivation of state from the file, and a Python plugin pipeline for arbitrary validation/transformation during load. Balance **assertions** and inventory/lot tracking are notable features. Active comparisons through 2025–2026 confirm all three are alive; beancount's philosophy is "assume data-entry errors, build guardrails." | |
| 99 | + | |
| 100 | +**Sources (consulted 2026-08-05):** | |
| 101 | +- plaintextaccounting.org (ecosystem hub): https://plaintextaccounting.org/ | |
| 102 | +- hledger FAQ (differences from ledger): https://hledger.org/faq.html | |
| 103 | +- Beancount vs hledger developer deep-dive: https://beancount.io/forum/t/beancount-vs-hledger-a-developers-deep-dive-after-using-both/34 | |
| 104 | +- Plain Text Accounting Showdown 2025 (Beancount v3 vs hledger vs Ledger): https://beancount.io/forum/t/the-ultimate-plain-text-accounting-showdown-2025-beancount-v3-vs-hledger-vs-ledger/81 | |
| 105 | +- Beancount technical comparison (performance, data integrity): https://beancount.io/blog/2025/07/22/beancounts-technical-edge-a-deep-dive-on-performance-python-api-and-data-integrity-vs-ledger-hledger-and-gnucash | |
| 106 | + | |
| 107 | +**Relevance to AIR:** | |
| 108 | +- **Borrow (heavily):** beancount's *strictness* philosophy — explicit account opening, balance assertions, deterministic re-derivation of all state from source data, plugin/pass pipeline (a direct analogue of AIC passes); the zero-sum posting invariant as the executable form of `Assets = Liabilities + Equity`; decimal-only amounts with explicit commodities; cost-basis/lot tracking ideas for FX and inventory. | |
| 109 | +- **Align (export):** a beancount/hledger text export backend is cheap and gives instant query/reporting tooling for golden tests and debugging — strongly recommended as a dev-facing backend alongside generic CSV. | |
| 110 | +- **Avoid:** their *transaction-as-source-of-truth* model. PTA files ARE the journal; AIR's source of truth is the economic event, with journals compiled. Also single-user, file-based — no concurrency, approval, or provenance model. | |
| 111 | + | |
| 112 | +--- | |
| 113 | + | |
| 114 | +## 5. REA Ontology (ISO 15944-4) and ValueFlows | |
| 115 | + | |
| 116 | +### 5.1 REA | |
| 117 | + | |
| 118 | +REA (Resources–Events–Agents), proposed by William E. McCarthy in 1982 ("The REA Accounting Model," *The Accounting Review*), models economic activity as: | |
| 119 | +- **Resources**: goods, services, rights, claims under agents' control; | |
| 120 | +- **Events**: phenomena that change resources (production, exchange, consumption, distribution), paired by **duality** (e.g., a Sale event dual to a CashReceipt event — give/take); | |
| 121 | +- **Agents**: identifiable parties who obtain, use, or dispose of resources. | |
| 122 | + | |
| 123 | +Key insight: debits, credits, and accounts are *derived views*, not primitives — the base data is the event graph. REA was standardized as **ISO/IEC 15944-4:2007** (Business Operational View — business transaction scenarios, accounting and economic ontology) and influenced IBM's financial reporting architecture. Later formalizations (REA2, Laurier/Kiehn/Polovina 2018) unify the exchange and conversion views. Extended REA adds **commitments** (promised future events, e.g., an order) and **contracts** (bundles of commitments) — exactly what AIR needs for pending delivery / revenue recognition (IFRS 15 performance obligations map naturally to commitments). | |
| 124 | + | |
| 125 | +### 5.2 ValueFlows | |
| 126 | + | |
| 127 | +ValueFlows is a modern RDF-based vocabulary built explicitly on REA/ISO 15944-4 for networked economies (used by hREA on Holochain, Bonfire, etc.). Three layers: **Knowledge** (resource types, recipes/rules), **Plan** (intents, commitments, offers/requests), **Observation** (actual economic events as they occur). Its accounting page states the AIR thesis almost verbatim: "A standard General Ledger, Balance Sheet, and Income Statement can be generated automatically from Valueflows data. No need to... post double-entries; those can all be created by a computer program on request." It also formalizes perspective-dependence: one agent's *purchase* is the counterparty's *sale* — one neutral event, multiple ledger views. | |
| 128 | + | |
| 129 | +**Sources (consulted 2026-08-05):** | |
| 130 | +- Wikipedia — Resources, Events, Agents (McCarthy 1982, ISO 15944-4): https://en.wikipedia.org/wiki/Resources,_Events,_Agents | |
| 131 | +- ValueFlows — Accounting concepts: https://www.valueflo.ws/concepts/accounting/ | |
| 132 | +- REA2: A unified formalisation of the REA ontology (Applied Ontology, 2018): https://journals.sagepub.com/doi/10.3233/AO-180198 | |
| 133 | +- REA, Triple-Entry Accounting and Blockchain (arXiv 2005.07802): https://arxiv.org/pdf/2005.07802 | |
| 134 | +- P2P Foundation — REA model overview: https://wiki.p2pfoundation.net/Resource-Event-Agent_Model | |
| 135 | +- From REA to hREA (decentralized implementations): https://happeningscommunity.substack.com/p/from-rea-to-hrea-a-journey-into-decentralized | |
| 136 | + | |
| 137 | +**Relevance to AIR:** | |
| 138 | +- **Align (core):** REA is the theoretical foundation AIR should explicitly claim. AIR's `EconomicEvent` already IS an REA event with resources (items, money) and agents (seller, buyer). Adopting REA vocabulary (duality, commitment, fulfillment, agent, resource) gives AIR 40 years of academic grounding plus an ISO citation. | |
| 139 | +- **Borrow:** duality pairing (Sale ↔ Payment) as the structural basis for AIR's provenance/SSA chain; commitments for pending deliveries and revenue recognition; ValueFlows' plan/observation split (quote/order = plan; invoice/payment = observation) and its perspective-neutral event with per-agent views (one AIR event compiles to different journals for each party). | |
| 140 | +- **Avoid:** ValueFlows' RDF/linked-data serialization and P2P-network scope creep — AIR needs a closed, versioned JSON Schema, deterministic compilation, and enterprise concerns (tax, approval, audit) that REA/VF leave out. | |
| 141 | + | |
| 142 | +--- | |
| 143 | + | |
| 144 | +## 6. Decisions / Recommendations for AIR | |
| 145 | + | |
| 146 | +1. **Adopt REA as AIR's conceptual foundation (YES).** AIR models economic events, not journal entries — this is precisely McCarthy's REA claim that debits/credits are derived views. State the REA/ISO 15944-4 lineage in the spec (`docs/spec/`), and adopt its vocabulary: `EconomicEvent`, `Agent`, `Resource`, `duality`, `Commitment`/`fulfills`. Add commitments to the schema early (orders, pending deliveries → IFRS 15/ASC 606 performance obligations). | |
| 147 | + | |
| 148 | +2. **Do NOT adopt XBRL-GL, UBL, camt, or PTA formats as the core IR.** Each models a different downstream/upstream artifact (journal entries, claim documents, bank entries, journal files). AIR stays event-centric; all of these become **mappings**: | |
| 149 | + - **Import (frontends):** UBL 2.1/2.4 + Peppol BIS Billing invoices; camt.053/054 (+ MT940, OFX legacy, Plaid/Flinks JSON) bank data. | |
| 150 | + - **Export (backends):** generic CSV (Phase 1), **beancount/hledger text** (recommended addition — near-free, enables golden-test diffing and instant reporting), XBRL-GL (later, for audit/tax interchange; note its niche adoption before investing). | |
| 151 | + | |
| 152 | +3. **Emulate EN 16931's semantics-first design:** define the AIR *semantic model* independently of syntax (JSON Schema as normative binding, others derivable), and reuse EN 16931/UBL field vocabulary for invoice-like events (tax subtotals per category, allowance/charge, payment means codes) so Peppol ingestion is lossless. | |
| 153 | + | |
| 154 | +4. **Steal beancount's discipline for AIC:** explicit account/entity declarations (no on-the-fly creation), balance assertions as a pass, deterministic full re-derivation from events, decimal-only amounts with explicit currency, and a plugin-style pass pipeline. The zero-sum posting rule is the executable `Assets = Liabilities + Equity` invariant to enforce after every pass. | |
| 155 | + | |
| 156 | +5. **Model settlements on camt semantics:** AIR bank/settlement events should carry ISO 20022-style bank transaction codes and end-to-end references to power the Phase 6 reconciliation pass; map camt's meaning, not its XML shape. | |
| 157 | + | |
| 158 | +6. **Perspective-neutral events (from ValueFlows):** one AIR event, per-agent compiled views (seller's journal vs buyer's journal). This should be a stated design principle even if v0.1 compiles for a single reporting entity. | |
| 159 | + | |
| 160 | +7. **Follow-up research needed:** ISO/IEC 15944-4 full text (paywalled — obtain for spec citations); XBRL-GL 2015 module reference if/when an export backend is scoped; FDX API model for future bank-feed frontends; UN/CEFACT CII as the second EN 16931 syntax. | |
| 161 | + | |
| 162 | +--- | |
| 163 | + | |
| 164 | +*End of report — consulted 2026-08-05 — Simon-Pierre Boucher — contact@spboucher.ai* | |
added
docs/research/accounting-standards.md
+132 −0
@@ -0,0 +1,132 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : accounting-standards.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Research: Accounting Standards as AIR Compilation Targets | |
| 9 | + | |
| 10 | +**Date of research:** 2026-08-05 | |
| 11 | +**Method:** 4 targeted web searches (revenue recognition, leases, foreign currency, ASPE) with follow-up on primary standard-setter sources. | |
| 12 | +**Purpose:** Ground the design of AIR backend "target profiles". In AIR, each accounting framework (IFRS, US GAAP, ASPE) is treated as a **compilation target**: the same `EconomicEvent` compiles to different journal entries depending on the selected standard profile, exactly as LLVM IR compiles to different machine code per target triple. | |
| 13 | + | |
| 14 | +--- | |
| 15 | + | |
| 16 | +## 1. Revenue recognition — IFRS 15 / ASC 606 | |
| 17 | + | |
| 18 | +IFRS 15 *Revenue from Contracts with Customers* (IASB) and ASC 606 (FASB) are converged standards built on the **same five-step model**: | |
| 19 | + | |
| 20 | +1. **Identify the contract** with a customer. | |
| 21 | +2. **Identify the performance obligations** — promises to transfer *distinct* goods or services (distinct = capable of being sold separately, or distinct in the context of the contract). | |
| 22 | +3. **Determine the transaction price** (including variable consideration, subject to constraint). | |
| 23 | +4. **Allocate the transaction price** to the performance obligations (relative standalone selling prices). | |
| 24 | +5. **Recognize revenue when (or as) each performance obligation is satisfied** — i.e., when **control** of the good or service transfers to the customer, *not* when payment is received. | |
| 25 | + | |
| 26 | +### Why this matters for AIR | |
| 27 | + | |
| 28 | +AIR's `Sale` event carries a `delivery: {status: pending, expected: ...}` field. Under IFRS 15/ASC 606, a sale that is *paid* but *not delivered* has **not** satisfied its performance obligation: | |
| 29 | + | |
| 30 | +- **Cash received, delivery pending** → the compiler must post a **contract liability / deferred revenue** (Dr Cash, Cr Deferred Revenue), not revenue. | |
| 31 | +- **On delivery** (a later `Delivery`/`Fulfillment` event referencing the original event in the provenance graph) → Dr Deferred Revenue, Cr Revenue. | |
| 32 | +- **Delivered, not yet invoiced/paid** → contract asset / unbilled receivable. | |
| 33 | + | |
| 34 | +Design consequences: | |
| 35 | +- The AIR schema must model **performance obligations as first-class line-item state** (satisfied / unsatisfied / partially satisfied over time), because revenue timing is a function of obligation satisfaction, not of the cash or invoice event. | |
| 36 | +- Multi-element sales (e.g., hardware + support contract) require **transaction-price allocation** in the compiler's revenue pass — allocation ratios are policy/target-profile data, never LLM output. | |
| 37 | +- Revenue timing rules belong to the **target profile** (IFRS vs ASPE differ; see §4), so the same `Sale` AIR event can legally compile to different entries per target. | |
| 38 | + | |
| 39 | +Sources (consulted 2026-08-05): | |
| 40 | +- IFRS Foundation — IFRS 15 Revenue from Contracts with Customers: https://www.ifrs.org/issued-standards/list-of-standards/ifrs-15-revenue-from-contracts-with-customers/ | |
| 41 | +- FASB — Revenue Recognition (Topic 606): https://www.fasb.org/revenue | |
| 42 | +- Certinia — ASC 606 and IFRS 15: 5 steps: https://www.certinia.com/resources/industry-101/complying-with-asc-606-and-ifrs-15/ | |
| 43 | +- DataStudios — IFRS 15 / ASC 606 rules, performance obligations, variable consideration: https://www.datastudios.org/post/revenue-recognition-ifrs-15-and-asc-606-rules-performance-obligations-variable-consideration-and | |
| 44 | +- GAAP Dynamics — Revenue recognition resources (ASC 606 & IFRS 15): https://www.gaapdynamics.com/insights/accounting-topics/revenue-recognition-accounting-resources-for-asc-606-and-ifrs-15/ | |
| 45 | + | |
| 46 | +--- | |
| 47 | + | |
| 48 | +## 2. Leases — IFRS 16 / ASC 842 (brief) | |
| 49 | + | |
| 50 | +Both standards put leases **on the balance sheet** for lessees: at commencement the lessee recognizes a **right-of-use (ROU) asset** and a **lease liability**, both measured at the present value of future lease payments (discounted at the rate implicit in the lease, else the incremental borrowing rate). | |
| 51 | + | |
| 52 | +Key divergences (they are *not* fully converged, unlike revenue): | |
| 53 | + | |
| 54 | +| Aspect | IFRS 16 | ASC 842 | | |
| 55 | +|---|---|---| | |
| 56 | +| Lessee classification | Single model — every lease → depreciation + interest (front-loaded) | Dual model — **finance** vs **operating**; operating lease → single straight-line lease cost | | |
| 57 | +| Low-value asset exemption | Yes (in addition to short-term) | No (short-term only) | | |
| 58 | +| Index/rate-linked payment changes | Remeasure the liability | No remeasurement; variable lease cost in period | | |
| 59 | +| Restoration/dismantling costs | Included in ROU asset | Separate ARO under ASC 410-20 | | |
| 60 | + | |
| 61 | +### Why this matters for AIR | |
| 62 | + | |
| 63 | +A single AIR `Lease` event compiles to **structurally different entries** per target profile — this is the strongest argument that AIR targets are genuine "backends", not just chart-of-account mappings. Lease compilation also requires the compiler to do **present-value math deterministically** (fixed decimal, documented rounding). Depth of lease support can wait for a later phase; the target-profile abstraction must accommodate it from day one. | |
| 64 | + | |
| 65 | +Sources (consulted 2026-08-05): | |
| 66 | +- IFRS Foundation — IFRS 16 Leases: https://www.ifrs.org/issued-standards/list-of-standards/ifrs-16-leases/ | |
| 67 | +- FASB — Leases (Topic 842): https://www.fasb.org/leases | |
| 68 | +- KPMG — Lease accounting: IFRS Accounting Standards vs US GAAP: https://kpmg.com/us/en/articles/2025/lease-accounting-ifrs-standards-us-gaap.html | |
| 69 | +- insightsoftware — Differences between ASC 842 & IFRS 16: https://insightsoftware.com/blog/what-are-the-differences-between-asc-842-ifrs-16/ | |
| 70 | +- Deloitte DART — IFRS/US GAAP comparison, Leases: https://dart.deloitte.com/USDART/home/publications/deloitte/additional-deloitte-guidance/roadmap-ifrs-us-gaap-comparison/chapter-5-broad-transactions/5-7-leases | |
| 71 | + | |
| 72 | +--- | |
| 73 | + | |
| 74 | +## 3. Foreign currency — IAS 21 (drives AIR's FX pass) | |
| 75 | + | |
| 76 | +IAS 21 *The Effects of Changes in Foreign Exchange Rates* (US GAAP analogue: ASC 830) defines the rules the AIR **FX pass** must implement: | |
| 77 | + | |
| 78 | +1. **Initial recognition:** a foreign-currency transaction is recorded in the entity's **functional currency** using the **spot rate at the transaction date** (the date the transaction first qualifies for recognition). | |
| 79 | +2. **Subsequent measurement at each reporting date:** | |
| 80 | + - **Monetary items** (cash, receivables, payables, loans — fixed/determinable currency amounts) are **remeasured at the closing rate**; differences go to **profit or loss** (unrealized FX gain/loss). | |
| 81 | + - **Non-monetary items at historical cost** keep the historical transaction-date rate — no retranslation. | |
| 82 | +3. **Settlement:** exchange differences arising on settlement of monetary items (rate at settlement vs rate at initial recognition / last remeasurement) are recognized in profit or loss — the **realized** FX gain/loss. | |
| 83 | + | |
| 84 | +### Why this matters for AIR | |
| 85 | + | |
| 86 | +- Every AIR `Money` value must carry its **currency**, and the FX pass attaches `{rate, rate_source, rate_date}` provenance nodes when converting to functional currency — the SSA-style provenance graph makes each converted amount traceable to a dated rate. | |
| 87 | +- The compiler must distinguish **three moments**: transaction-date translation (booking), period-end remeasurement of open monetary balances (unrealized), and settlement (realized). These are three distinct pass behaviors producing distinct, reversible entries. | |
| 88 | +- Realized vs unrealized gains post to **separate accounts**; period-end remeasurement entries are natural candidates for automatic reversal on the next period open (fits AIR's incremental compilation / contra-entry model). | |
| 89 | +- The functional currency is a **target/entity profile parameter**, never inferred by the LLM. | |
| 90 | + | |
| 91 | +See companion note `fx-handling.md` for rate sources (Bank of Canada Valet API) and CRA rules. | |
| 92 | + | |
| 93 | +Sources (consulted 2026-08-05): | |
| 94 | +- IFRS Foundation — IAS 21 The Effects of Changes in Foreign Exchange Rates: https://www.ifrs.org/issued-standards/list-of-standards/ias-21-the-effects-of-changes-in-foreign-exchange-rates/ | |
| 95 | +- IFRScommunity — Changes in Foreign Exchange Rates (IAS 21): https://ifrscommunity.com/knowledge-base/ias-21-effects-of-changes-in-foreign-exchange-rates/ | |
| 96 | +- Moore Global — IAS 21 overview: https://www.moore-global.com/services/ifrs/ias-21-the-effects-of-changes-in-foreign-exchange-rates/ | |
| 97 | +- DataStudios — Foreign currency under ASC 830 and IAS 21: https://www.datastudios.org/post/foreign-currency-transactions-and-translation-adjustments-under-u-s-gaap-asc-830-and-ias-21 | |
| 98 | + | |
| 99 | +--- | |
| 100 | + | |
| 101 | +## 4. Canadian ASPE — Accounting Standards for Private Enterprises | |
| 102 | + | |
| 103 | +**What it is:** ASPE is the Canadian GAAP framework for **private enterprises**, issued by the Accounting Standards Board (AcSB) and published in Part II of the CPA Canada Handbook – Accounting. Canadian **publicly accountable** enterprises must use IFRS (Part I); private enterprises may **choose** ASPE or IFRS. ASPE is recognized only in Canada. | |
| 104 | + | |
| 105 | +**Who uses it:** the vast majority of Canadian private companies (SMBs and larger private firms not seeking public/foreign capital), because it is simpler and cheaper to apply than IFRS. | |
| 106 | + | |
| 107 | +**Key high-level differences vs IFRS:** | |
| 108 | +- **Revenue:** ASPE (Section 3400) is a simpler, more flexible model — no mandatory IFRS 15-style five-step framework, though deferral of unearned revenue still applies. | |
| 109 | +- **Leases:** ASPE (Section 3065) retains the old capital/operating lease distinction — operating leases stay **off balance sheet**, unlike IFRS 16. | |
| 110 | +- **Goodwill:** ASPE allows amortization / impairment-on-indication; IFRS requires annual impairment testing without amortization. | |
| 111 | +- **Financial instruments:** ASPE permits cost-based measurement in many cases; IFRS leans on fair value. | |
| 112 | +- **PP&E:** ASPE is cost model only; IFRS permits revaluation. | |
| 113 | +- **Disclosures:** substantially lighter under ASPE. | |
| 114 | + | |
| 115 | +### Why this matters for AIR | |
| 116 | + | |
| 117 | +ASPE confirms the **target-profile** design: a Canadian SMB target (ASPE) and an IFRS target compile the *same* AIR events differently (leases and revenue timing being the clearest cases). Given AIR's initial CA-QC focus, the **ASPE profile is a natural first "real" standards target** alongside the generic CSV backend, with IFRS and US GAAP profiles layered on the same interface. | |
| 118 | + | |
| 119 | +Sources (consulted 2026-08-05): | |
| 120 | +- CPA Canada — Summary comparison of ASPE and IFRS: https://www.cpacanada.ca/en/business-and-accounting-resources/financial-and-non-financial-reporting/accounting-standards-for-private-enterprises-aspe/publications/summary-comparison-of-aspe-and-ifrs | |
| 121 | +- FRAS Canada (AcSB) — Accounting Standards for Private Enterprises: https://www.frascanada.ca/en/aspe | |
| 122 | +- BDO Canada — ASPE–IFRS: A Comparison series: https://www.bdo.ca/insights/accounting-knowledge-center/aspe-ifrs-a-comparison | |
| 123 | +- BDC — Accounting Standards for Private Enterprises (ASPE) glossary: https://www.bdc.ca/en/articles-tools/entrepreneur-toolkit/templates-business-guides/glossary/accounting-standards-for-financial-enterprises | |
| 124 | + | |
| 125 | +--- | |
| 126 | + | |
| 127 | +## 5. Decisions / follow-ups | |
| 128 | + | |
| 129 | +- **D1:** Model standards (IFRS / US GAAP / ASPE) as **target profiles** consumed by the posting pass; profile choice is entity configuration, never event data. → Feed into ADR on backend architecture. | |
| 130 | +- **D2:** AIR `Sale` must separate cash/invoice events from performance-obligation satisfaction (delivery) to support deferred revenue under all profiles. | |
| 131 | +- **D3:** FX pass implements IAS 21 semantics (spot at transaction date; monetary remeasurement at close; realized on settlement) with full rate provenance. | |
| 132 | +- **Follow-up:** golden tests for (a) paid-but-undelivered sale → deferred revenue, (b) unpaid FX receivable across a period end → unrealized then realized gain/loss, (c) same lease under ASPE vs IFRS profiles. | |
added
docs/research/bank-statement-formats.md
+447 −0
@@ -0,0 +1,447 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : bank-statement-formats.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Bank Statement Formats & Reconciliation Research | |
| 9 | + | |
| 10 | +Research for Phase 6 (bank reconciliation): parsing bank statements (camt.053, MT940) | |
| 11 | +and matching statement lines against ledger cash movements. All findings below are | |
| 12 | +sourced from web research performed on **2026-08-05**; nothing is from memory. | |
| 13 | + | |
| 14 | +--- | |
| 15 | + | |
| 16 | +## 1. ISO 20022 camt.053 — BankToCustomerStatement | |
| 17 | + | |
| 18 | +camt.053 is the ISO 20022 XML end-of-day bank account statement (the "MX" successor | |
| 19 | +to MT940). It is part of the Bank-to-Customer Cash Management message set maintained | |
| 20 | +by the ISO 20022 Registration Authority (iso20022.org). | |
| 21 | + | |
| 22 | +### 1.1 Document root and namespace | |
| 23 | + | |
| 24 | +- Root element: `<Document>` with a version-specific default namespace: | |
| 25 | + `urn:iso:std:iso:20022:tech:xsd:camt.053.001.NN` (e.g. `camt.053.001.02`, | |
| 26 | + `camt.053.001.08`). **The version is identified via the XML namespace**, so the | |
| 27 | + parser must read the namespace, not assume one. | |
| 28 | +- Inside: `<BkToCstmrStmt>` (BankToCustomerStatement), containing one `<GrpHdr>` | |
| 29 | + and one or more `<Stmt>` blocks (one per account/period). | |
| 30 | + | |
| 31 | +### 1.2 Structure (elements the AIR parser must handle) | |
| 32 | + | |
| 33 | +``` | |
| 34 | +Document @xmlns = urn:iso:std:iso:20022:tech:xsd:camt.053.001.NN | |
| 35 | +└── BkToCstmrStmt | |
| 36 | + ├── GrpHdr | |
| 37 | + │ ├── MsgId unique message id | |
| 38 | + │ └── CreDtTm file creation timestamp (ISO 8601) | |
| 39 | + └── Stmt [1..n] one per account statement | |
| 40 | + ├── Id statement id | |
| 41 | + ├── ElctrncSeqNb electronic sequence number (optional) | |
| 42 | + ├── CreDtTm statement creation timestamp | |
| 43 | + ├── Acct | |
| 44 | + │ ├── Id/IBAN or Id/Othr/Id account identifier (IBAN or domestic) | |
| 45 | + │ └── Ccy account currency (ISO 4217) | |
| 46 | + ├── Bal [1..n] balances | |
| 47 | + │ ├── Tp/CdOrPrtry/Cd OPBD=opening booked, CLBD=closing booked, | |
| 48 | + │ │ CLAV=closing available, OIBD/CIBD=interim (multi-page) | |
| 49 | + │ ├── Amt @Ccy decimal amount, currency as XML attribute | |
| 50 | + │ ├── CdtDbtInd CRDT | DBIT (sign of the balance) | |
| 51 | + │ └── Dt/Dt balance date | |
| 52 | + └── Ntry [0..n] statement entries (transactions) | |
| 53 | + ├── NtryRef entry reference (optional) | |
| 54 | + ├── Amt @Ccy ALWAYS positive decimal; currency attribute mandatory | |
| 55 | + ├── CdtDbtInd CRDT (money in) | DBIT (money out) — gives the sign | |
| 56 | + ├── RvslInd true if this entry reverses a previous one (optional) | |
| 57 | + ├── Sts BOOK (booked) | PDNG (pending) | INFO | |
| 58 | + ├── BookgDt/Dt (or /DtTm) booking (posting) date | |
| 59 | + ├── ValDt/Dt (or /DtTm) value date | |
| 60 | + ├── AcctSvcrRef bank's own reference (optional) | |
| 61 | + ├── BkTxCd bank transaction code (Domn/Fmly/SubFmlyCd or Prtry/Cd) | |
| 62 | + ├── NtryDtls [0..n] | |
| 63 | + │ ├── Btch batch info (NbOfTxs, TtlAmt) when 1 entry = n transactions | |
| 64 | + │ └── TxDtls [0..n] underlying transactions | |
| 65 | + │ ├── Refs/EndToEndId payer-assigned end-to-end reference (invoice/PO ref) | |
| 66 | + │ ├── Refs/TxId, InstrId, MndtId | |
| 67 | + │ ├── AmtDtls InstdAmt / TxAmt (useful for FX) | |
| 68 | + │ ├── RltdPties Dbtr / Cdtr names and accounts | |
| 69 | + │ └── RmtInf/Ustrd unstructured remittance info (free text) | |
| 70 | + │ RmtInf/Strd structured remittance (creditor reference etc.) | |
| 71 | + └── AddtlNtryInf free-text description of the entry | |
| 72 | +``` | |
| 73 | + | |
| 74 | +Key semantics: | |
| 75 | + | |
| 76 | +- **`Amt` carries no sign**; direction comes exclusively from `CdtDbtInd` | |
| 77 | + (`CRDT` = credit to the account = money received; `DBIT` = money out). | |
| 78 | +- The currency is the mandatory `Ccy` **attribute** on `Amt`. | |
| 79 | +- One `Ntry` may aggregate many underlying transactions (batch/lump-sum entries): | |
| 80 | + `NtryDtls/Btch` + repeated `TxDtls`. This is the structural basis for | |
| 81 | + one-to-many reconciliation matching. | |
| 82 | +- `EndToEndId` (max 35 chars) travels unaltered from the payment initiation | |
| 83 | + (pain.001) to the statement — it is the best deterministic matching key. | |
| 84 | + | |
| 85 | +### 1.3 Version differences (brief) | |
| 86 | + | |
| 87 | +- Common versions in the wild: `.02` (2009 base, still the most widely delivered | |
| 88 | + by banks, e.g. Nordea, Dutch banks), `.06`, `.08` (2019 — the CBPR+/SEPA-aligned | |
| 89 | + version banks are converging on), `.10`/`.11`/`.13` (latest, Feb 2025). | |
| 90 | +- Newer versions mostly **add optional fields without removing existing ones**; | |
| 91 | + the structural core (`GrpHdr` / `Stmt` / `Bal` / `Ntry`) is stable across versions. | |
| 92 | + Notable mechanical difference: in `.02` `Sts` is a simple code | |
| 93 | + (`<Sts>BOOK</Sts>`); in `.08+` it becomes a complex element | |
| 94 | + (`<Sts><Cd>BOOK</Cd></Sts>`). Party identification blocks also gain structure | |
| 95 | + in `.08+`. | |
| 96 | +- Practical parser strategy (used by open-source parsers): parse | |
| 97 | + namespace-tolerantly, target the common core, handle the `Sts` shape difference. | |
| 98 | + | |
| 99 | +Sources: iso20022.org Message Definition Report Part 2 (Bank-to-Customer Cash | |
| 100 | +Management), Payments Canada camt.053.001.08 usage guideline, Nordea | |
| 101 | +camt.053.001.02 standard, ValidateFin structural guide, darko-mijic camt.053 | |
| 102 | +parser spec (URLs in §7). | |
| 103 | + | |
| 104 | +--- | |
| 105 | + | |
| 106 | +## 2. MT940 — SWIFT Customer Statement Message | |
| 107 | + | |
| 108 | +MT940 is the legacy SWIFT FIN "MT" end-of-day customer statement: a line/tag-based | |
| 109 | +text format. Still extremely widespread in bank connectivity (EBICS, host-to-host, | |
| 110 | +e-banking exports) even though it is deprecated by SWIFT (see §3). | |
| 111 | + | |
| 112 | +### 2.1 Tag structure | |
| 113 | + | |
| 114 | +| Tag | Name | Format | Notes | | |
| 115 | +|---|---|---|---| | |
| 116 | +| `:20:` | Transaction Reference Number | 16x | Sender's reference for the message | | |
| 117 | +| `:25:` | Account Identification | 35x | Account number (option `P` adds BIC) | | |
| 118 | +| `:28C:` | Statement Number/Sequence | 5n[/5n] | e.g. `151/1` | | |
| 119 | +| `:60F:` | Opening Balance (First) | 1!a6!n3!a15d | D/C mark + date YYMMDD + currency + amount | | |
| 120 | +| `:61:` | Statement Line | see §2.2 | Repeats per transaction | | |
| 121 | +| `:86:` | Information to Account Owner | 6*65x | Optional; up to 6 lines × 65 chars; must follow its `:61:` | | |
| 122 | +| `:62F:` | Closing Balance (Final) | 1!a6!n3!a15d | Same layout as `:60F:` | | |
| 123 | +| `:60M:`/`:62M:` | Intermediate opening/closing balances | same | Used when a statement spans multiple messages | | |
| 124 | +| `:64:`/`:65:` | Closing available / forward available balance | same layout | Optional | | |
| 125 | + | |
| 126 | +Constraint: a `:86:` must be preceded by a `:61:` and belongs to it; when a | |
| 127 | +period spans several messages, all but the last carry `:62M:` and the last | |
| 128 | +carries `:62F:`. | |
| 129 | + | |
| 130 | +### 2.2 The `:61:` statement line — full format specification | |
| 131 | + | |
| 132 | +``` | |
| 133 | +:61: 6!n [4!n] 2a [1!a] 15d 1!a3!c 16x [//16x] [34x] | |
| 134 | + │ │ │ │ │ │ │ │ └─ supplementary details (optional, new line) | |
| 135 | + │ │ │ │ │ │ │ └─ bank reference (optional, after //) | |
| 136 | + │ │ │ │ │ │ └─ customer reference (16x; NONREF if none) | |
| 137 | + │ │ │ │ │ └─ transaction type code: 1 letter + 3 chars | |
| 138 | + │ │ │ │ │ S103 = via SWIFT MT103, NTRF = non-SWIFT transfer, | |
| 139 | + │ │ │ │ │ NMSC = misc, NCHK = cheque, NDDT = direct debit... | |
| 140 | + │ │ │ │ └─ amount: max 15 digits, COMMA as decimal separator, | |
| 141 | + │ │ │ │ no thousands separators, ALWAYS UNSIGNED | |
| 142 | + │ │ │ └─ funds code (optional, 3rd char of currency) | |
| 143 | + │ │ └─ debit/credit mark: D | C | RD (reversal of debit) | RC (reversal of credit) | |
| 144 | + │ └─ entry (booking) date MMDD (optional; year inferred from value date) | |
| 145 | + └─ value date YYMMDD | |
| 146 | +``` | |
| 147 | + | |
| 148 | +Key semantics: | |
| 149 | + | |
| 150 | +- **Amount is unsigned; the D/C mark gives direction** (D = debit = money out, | |
| 151 | + C = credit = money in; `RD`/`RC` mark reversals). | |
| 152 | +- **Decimal separator is a comma** (`1150,00`), never a dot; the currency is NOT | |
| 153 | + on the `:61:` line — it comes from the `:60F:`/`:62F:` balance lines. | |
| 154 | +- Value date is `YYMMDD` (2-digit year → pivot logic needed); entry date is | |
| 155 | + `MMDD` with the year inferred from the value date (beware year boundaries). | |
| 156 | +- The customer reference (`16x`) is the counterpart of camt's `EndToEndId`, but | |
| 157 | + is often `NONREF`; the free-text `:86:` line then carries the useful | |
| 158 | + description (bank-proprietary sub-formats exist inside `:86:`, e.g. German | |
| 159 | + `/GVC/` and `?20`-style subtags — treat `:86:` as opaque text in v1). | |
| 160 | +- Integrity check available to the parser: | |
| 161 | + `:60F: balance ± Σ(:61: lines) = :62F: balance` — AIR should verify this. | |
| 162 | + | |
| 163 | +Sources: Paiementor MT940 detailed analysis, National Bank of Canada MT940 user | |
| 164 | +guide, Citi MT940 export guide, Huntington developer portal (URLs in §7). | |
| 165 | + | |
| 166 | +--- | |
| 167 | + | |
| 168 | +## 3. Adoption status: camt.053 vs MT940 (MT→MX migration) | |
| 169 | + | |
| 170 | +- **22 November 2025**: SWIFT ended MT/ISO 20022 coexistence for cross-border | |
| 171 | + *payment instruction* messages (MT103/MT202 retired for FI-to-FI traffic). | |
| 172 | +- **Cash-management / reporting messages (MT9xx) got an extended timeline**: | |
| 173 | + MT940/MT942/MT950/MT900/MT910 are *deprecated and no longer maintained* by | |
| 174 | + SWIFT but not yet withdrawn; their removal in favour of | |
| 175 | + camt.052/camt.053/camt.054 is being phased in roughly **2027–2028**, and major | |
| 176 | + banks tell clients they must be able to *receive* camt messages by | |
| 177 | + **November 2027**. | |
| 178 | +- Practical consequence (2026): **both formats are current reality.** camt.053 | |
| 179 | + is the strategic/current standard; MT940 remains the most widely deployed | |
| 180 | + corporate statement format worldwide (bank portals, ERP integrations, EBICS). | |
| 181 | + → AIR must parse **both**, normalizing into one internal model (§6). | |
| 182 | + | |
| 183 | +Sources: SWIFT ISO 20022 programme pages, J.P. Morgan and Citi migration FAQs, | |
| 184 | +RedCompass Labs 2026 deadline analysis, PaymentExpert on the Nov-2025 cutover | |
| 185 | +(URLs in §7). | |
| 186 | + | |
| 187 | +--- | |
| 188 | + | |
| 189 | +## 4. Bank reconciliation matching practice | |
| 190 | + | |
| 191 | +How mainstream systems match statement lines to ledger (book) cash entries: | |
| 192 | + | |
| 193 | +- **Exact match first**: same amount, same currency, same/near date. This is the | |
| 194 | + universal baseline (QuickBooks Online "matches", Xero "suggested matches"). | |
| 195 | +- **Date tolerance windows**: book date and bank date rarely coincide (cheques | |
| 196 | + clear days later; card settlements lag 1–3 days). Treasury/ERP tools (e.g. | |
| 197 | + Oracle Fusion reconciliation) expose *configurable* date tolerance rules; | |
| 198 | + date tolerances exist mainly for instruments with clearing lag. | |
| 199 | +- **Amount tolerance**: used for FX rounding differences or bank fees embedded in | |
| 200 | + the statement line. QuickBooks does **not** natively do tolerance-based | |
| 201 | + matching; Oracle-class treasury tools do (with a write-off/adjustment posting | |
| 202 | + for the difference). Xero handles the fee case via "adjustments" during | |
| 203 | + reconciliation. | |
| 204 | +- **Reference matching**: strongest deterministic signal — cheque number, | |
| 205 | + invoice number in remittance text, and in camt.053 the `EndToEndId` / | |
| 206 | + `AcctSvcrRef`. Xero bank rules can condition on bank text fields, direction | |
| 207 | + ("Received"/"Spent") and amount; a rule sets contact/account/tax code. | |
| 208 | +- **One-to-many / many-to-one (batch) matching**: one bank deposit = many | |
| 209 | + customer receipts (or one payroll debit = many payslips). Xero supports | |
| 210 | + "Match > multiple items"; QuickBooks Online is weak here natively; camt.053 | |
| 211 | + expresses it structurally (`NtryDtls/Btch` + several `TxDtls`). Treasury | |
| 212 | + reconciliation engines support 1:1, 1:n, n:1 and n:m rule tiers, applied in | |
| 213 | + priority order (exact reference → exact amount+date → amount+date window → | |
| 214 | + aggregate/batch), each with confidence scoring. | |
| 215 | +- **Output convention**: after matching, both sides' residues are reported — | |
| 216 | + unmatched statement lines (bank has it, books don't: missing entry / fee / | |
| 217 | + fraud) and unmatched ledger entries (books have it, bank doesn't: outstanding | |
| 218 | + cheque / deposit in transit). This two-sided exception report *is* the | |
| 219 | + classical bank reconciliation statement. | |
| 220 | + | |
| 221 | +Sources: Oracle Fusion tolerance-rule docs, Xero reconciliation guides, | |
| 222 | +QuickBooks Online reconciliation guides (URLs in §7). | |
| 223 | + | |
| 224 | +--- | |
| 225 | + | |
| 226 | +## 5. Aggregator APIs (future ingestion frontends) | |
| 227 | + | |
| 228 | +### 5.1 Plaid (`/transactions/sync`, `/transactions/get`) | |
| 229 | + | |
| 230 | +JSON transaction objects; key fields: | |
| 231 | + | |
| 232 | +- `amount` — decimal number. **Sign convention (Plaid docs): "Positive values | |
| 233 | + when money moves out of the account; negative values when money moves in."** | |
| 234 | + I.e. a debit-card purchase is **positive**, a deposit/refund is **negative** | |
| 235 | + — the *inverse* of a naive signed-bank-balance convention. Two decimal places. | |
| 236 | +- `iso_currency_code` (ISO 4217; null when `unofficial_currency_code` is used). | |
| 237 | +- `date` (YYYY-MM-DD; posting date for posted transactions, occurrence date for | |
| 238 | + pending), plus `authorized_date` / `datetime` / `authorized_datetime`. | |
| 239 | +- `name` (raw description) and `merchant_name` (Plaid-enriched, cleaner). | |
| 240 | +- `pending` (bool — details may change at settlement; the posted transaction | |
| 241 | + arrives with a new `transaction_id` and a `pending_transaction_id` link). | |
| 242 | +- `transaction_id` (unique, case-sensitive), `payment_channel` | |
| 243 | + (`online` / `in store` / `other`), `personal_finance_category`. | |
| 244 | + | |
| 245 | +⚠ Plaid delivers `amount` as a JSON **number** → the AIR parser must decode it | |
| 246 | +via decimal-preserving parsing (e.g. `json.loads(..., parse_float=Decimal)`), | |
| 247 | +never through binary floats. | |
| 248 | + | |
| 249 | +### 5.2 Flinks (Canadian aggregator, `/GetAccountsDetail`) | |
| 250 | + | |
| 251 | +Transactions array per account; each item: | |
| 252 | + | |
| 253 | +```json | |
| 254 | +{ | |
| 255 | + "Date": "2025-01-31", | |
| 256 | + "Code": null, | |
| 257 | + "Description": "PAYROLL - Stripe Paycheck", | |
| 258 | + "Debit": 1000.4, | |
| 259 | + "Credit": 1500.25, | |
| 260 | + "Balance": 5105.6, | |
| 261 | + "Id": "94584aed-7c98-42a4-9836-9f8557db63f5" | |
| 262 | +} | |
| 263 | +``` | |
| 264 | + | |
| 265 | +- Direction is expressed by **separate `Debit` / `Credit` fields** (one | |
| 266 | + populated, the other null in real responses), not a signed amount — a third | |
| 267 | + sign convention to normalize. `Balance` is the running balance after the | |
| 268 | + transaction; `Id` is Flinks' unique transaction id. | |
| 269 | +- Same JSON-number caveat as Plaid: parse into Decimal, never float. | |
| 270 | + | |
| 271 | +Sources: Plaid Transactions API reference, Flinks GetAccountsDetail docs | |
| 272 | +(URLs in §7). | |
| 273 | + | |
| 274 | +--- | |
| 275 | + | |
| 276 | +## 6. Example fixtures (small, structurally correct) | |
| 277 | + | |
| 278 | +Both fixtures describe the same statement: CAD account, opening balance | |
| 279 | +25,000.00, one outgoing supplier payment of 1,150.00 on 2026-08-01, one | |
| 280 | +incoming customer payment of 3,449.93 on 2026-08-04, closing balance 27,299.93 | |
| 281 | +(25,000.00 − 1,150.00 + 3,449.93 = 27,299.93 ✓). | |
| 282 | + | |
| 283 | +### 6.1 camt.053 (version 053.001.02) | |
| 284 | + | |
| 285 | +```xml | |
| 286 | +<?xml version="1.0" encoding="UTF-8"?> | |
| 287 | +<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02"> | |
| 288 | + <BkToCstmrStmt> | |
| 289 | + <GrpHdr> | |
| 290 | + <MsgId>AIR-STMT-20260805-001</MsgId> | |
| 291 | + <CreDtTm>2026-08-05T06:00:00</CreDtTm> | |
| 292 | + </GrpHdr> | |
| 293 | + <Stmt> | |
| 294 | + <Id>STMT-2026-0151</Id> | |
| 295 | + <ElctrncSeqNb>151</ElctrncSeqNb> | |
| 296 | + <CreDtTm>2026-08-05T06:00:00</CreDtTm> | |
| 297 | + <Acct> | |
| 298 | + <Id><Othr><Id>00112233445</Id></Othr></Id> | |
| 299 | + <Ccy>CAD</Ccy> | |
| 300 | + </Acct> | |
| 301 | + <Bal> | |
| 302 | + <Tp><CdOrPrtry><Cd>OPBD</Cd></CdOrPrtry></Tp> | |
| 303 | + <Amt Ccy="CAD">25000.00</Amt> | |
| 304 | + <CdtDbtInd>CRDT</CdtDbtInd> | |
| 305 | + <Dt><Dt>2026-07-31</Dt></Dt> | |
| 306 | + </Bal> | |
| 307 | + <Bal> | |
| 308 | + <Tp><CdOrPrtry><Cd>CLBD</Cd></CdOrPrtry></Tp> | |
| 309 | + <Amt Ccy="CAD">27299.93</Amt> | |
| 310 | + <CdtDbtInd>CRDT</CdtDbtInd> | |
| 311 | + <Dt><Dt>2026-08-04</Dt></Dt> | |
| 312 | + </Bal> | |
| 313 | + <Ntry> | |
| 314 | + <NtryRef>BKREF001</NtryRef> | |
| 315 | + <Amt Ccy="CAD">1150.00</Amt> | |
| 316 | + <CdtDbtInd>DBIT</CdtDbtInd> | |
| 317 | + <Sts>BOOK</Sts> | |
| 318 | + <BookgDt><Dt>2026-08-01</Dt></BookgDt> | |
| 319 | + <ValDt><Dt>2026-08-01</Dt></ValDt> | |
| 320 | + <BkTxCd><Prtry><Cd>NTRF</Cd></Prtry></BkTxCd> | |
| 321 | + <NtryDtls> | |
| 322 | + <TxDtls> | |
| 323 | + <Refs><EndToEndId>INV-2026-0042</EndToEndId></Refs> | |
| 324 | + <RmtInf><Ustrd>PAYMENT ACME INC INVOICE INV-2026-0042</Ustrd></RmtInf> | |
| 325 | + </TxDtls> | |
| 326 | + </NtryDtls> | |
| 327 | + <AddtlNtryInf>Supplier payment Acme Inc.</AddtlNtryInf> | |
| 328 | + </Ntry> | |
| 329 | + <Ntry> | |
| 330 | + <NtryRef>BKREF002</NtryRef> | |
| 331 | + <Amt Ccy="CAD">3449.93</Amt> | |
| 332 | + <CdtDbtInd>CRDT</CdtDbtInd> | |
| 333 | + <Sts>BOOK</Sts> | |
| 334 | + <BookgDt><Dt>2026-08-04</Dt></BookgDt> | |
| 335 | + <ValDt><Dt>2026-08-04</Dt></ValDt> | |
| 336 | + <BkTxCd><Prtry><Cd>NTRF</Cd></Prtry></BkTxCd> | |
| 337 | + <NtryDtls> | |
| 338 | + <TxDtls> | |
| 339 | + <Refs><EndToEndId>E2E-SALE-7781</EndToEndId></Refs> | |
| 340 | + <RmtInf><Ustrd>CUSTOMER CUST 123 SALE 7781</Ustrd></RmtInf> | |
| 341 | + </TxDtls> | |
| 342 | + </NtryDtls> | |
| 343 | + <AddtlNtryInf>Customer payment, sale 7781</AddtlNtryInf> | |
| 344 | + </Ntry> | |
| 345 | + </Stmt> | |
| 346 | + </BkToCstmrStmt> | |
| 347 | +</Document> | |
| 348 | +``` | |
| 349 | + | |
| 350 | +(For `.08+` fixtures the only mechanical change in these fields is | |
| 351 | +`<Sts><Cd>BOOK</Cd></Sts>` and the namespace suffix.) | |
| 352 | + | |
| 353 | +### 6.2 MT940 | |
| 354 | + | |
| 355 | +``` | |
| 356 | +:20:AIR-STMT-0805 | |
| 357 | +:25:BOFCCAM2/00112233445 | |
| 358 | +:28C:151/1 | |
| 359 | +:60F:C260731CAD25000,00 | |
| 360 | +:61:2608010801D1150,00NTRFINV-2026-0042//BKREF001 | |
| 361 | +:86:PAYMENT ACME INC INVOICE INV-2026-0042 | |
| 362 | +:61:2608040804C3449,93NTRFE2E-SALE-7781//BKREF002 | |
| 363 | +:86:CUSTOMER CUST 123 SALE 7781 | |
| 364 | +:62F:C260804CAD27299,93 | |
| 365 | +``` | |
| 366 | + | |
| 367 | +Reading `:61:2608010801D1150,00NTRFINV-2026-0042//BKREF001`: | |
| 368 | +value date `260801` (2026-08-01), entry date `0801`, `D` = debit, amount | |
| 369 | +`1150,00` (comma decimal), transaction type `NTRF`, customer reference | |
| 370 | +`INV-2026-0042`, bank reference `BKREF001`. | |
| 371 | + | |
| 372 | +--- | |
| 373 | + | |
| 374 | +## 7. Design decisions for AIR | |
| 375 | + | |
| 376 | +1. **Common internal model.** Both parsers (and future Plaid/Flinks ingesters) | |
| 377 | + produce the same `BankTransaction`: | |
| 378 | + - `date` (booking date, ISO date; value date kept separately if present) | |
| 379 | + - `amount` — **signed decimal serialized as a string** (positive = money in | |
| 380 | + to the account, negative = money out; AIR convention, normalized from | |
| 381 | + CdtDbtInd / D-C mark / Plaid inverse sign / Flinks Debit-Credit columns) | |
| 382 | + - `currency` (ISO 4217; from `Amt@Ccy` in camt, from `:60F:` in MT940) | |
| 383 | + - `description` (from `AddtlNtryInf`/`RmtInf/Ustrd` or `:86:`) | |
| 384 | + - `reference` (from `EndToEndId` / `NtryRef` / `:61:` customer+bank refs; | |
| 385 | + `NONREF` normalized to null) | |
| 386 | + - plus provenance: source format, statement id, raw line/entry. | |
| 387 | +2. **Amounts are never floats.** camt/MT940 amounts are parsed from text into | |
| 388 | + `Decimal`; MT940 comma decimals converted textually (`1150,00` → `1150.00`); | |
| 389 | + aggregator JSON parsed with `parse_float=Decimal`. | |
| 390 | +3. **Statement integrity check at parse time**: opening balance + Σ signed | |
| 391 | + entries must equal closing balance (`:60F:`/`:62F:`, `OPBD`/`CLBD`); | |
| 392 | + mismatch is a compiler-grade diagnostic, not a warning. | |
| 393 | +4. **Matching algorithm (v1)**: match = **amount + currency exact**, with a | |
| 394 | + **configurable date tolerance window** (default e.g. ±3 days, policy-driven | |
| 395 | + via ALSL, never hard-coded); reference/`EndToEndId` equality is used first | |
| 396 | + as a higher-priority deterministic tier and as a tie-breaker when several | |
| 397 | + candidates share amount+date. One-to-many (batch) matching deferred to a | |
| 398 | + later iteration but the model keeps `TxDtls` multiplicity so it stays possible. | |
| 399 | +5. **Two-sided residue reporting**: the reconciliation result lists matched | |
| 400 | + pairs *and* unmatched items on both sides (unmatched bank lines, unmatched | |
| 401 | + ledger entries), mirroring standard reconciliation-statement practice. | |
| 402 | +6. **Version-tolerant camt parser**: read the namespace to detect the version, | |
| 403 | + target the stable core (§1.2), handle the `.02` vs `.08` `Sts` shape. | |
| 404 | +7. **Both formats are required** (§3): camt.053 is strategic, MT940 remains | |
| 405 | + ubiquitous until at least 2027-2028. | |
| 406 | + | |
| 407 | +--- | |
| 408 | + | |
| 409 | +## 8. Sources (all consulted 2026-08-05) | |
| 410 | + | |
| 411 | +### camt.053 / ISO 20022 | |
| 412 | +- ISO 20022 MDR Part 2 — Bank-to-Customer Cash Management (iso20022.org): https://www.iso20022.org/sites/default/files/2020-12/ISO20022_MDRPart2_BankToCustomerCashManagement_2020_2021_v1_ForSEGReview.pdf | |
| 413 | +- Payments Canada — Usage Guideline camt.053.001.08: https://www.payments.ca/sites/default/files/Bank%20to%20customer%20statement%20V08%20(camt.053.001.08)(pdf).pdf | |
| 414 | +- Nordea — camt.053.001.02 Account Statement Standard: https://www.nordea.com/en/doc/caar-camt-053-001-02-account-statement-standard.pdf | |
| 415 | +- Betaalvereniging Nederland — IG Bank-to-Customer Statement camt.053: https://www.betaalvereniging.nl/wp-content/uploads/2026/03/IG-Bank-to-Customer-Statement-CAMT-053-v1-1.pdf | |
| 416 | +- ValidateFin — Reading a camt.053 bank statement: https://validatefin.com/en/blog/camt053-bank-statement | |
| 417 | +- darko-mijic — iso-20022-camt-053-parser spec: https://github.com/darko-mijic/iso-20022-camt-053-parser/blob/main/docs/iso20022-camt-053-spec.md | |
| 418 | +- Huntington Developer Portal — CAMT.053: https://developer.huntington.com/enterprisepayments/docs/camt053 | |
| 419 | + | |
| 420 | +### MT940 | |
| 421 | +- Paiementor — SWIFT MT940 Customer Statement detailed analysis: https://www.paiementor.com/swift-mt940-customer-statement-detailed-analysis/ | |
| 422 | +- National Bank of Canada — MT940/MT101 user guide: https://www.nbc.ca/content/dam/bnc/outils-apps/entreprises/guides/user-guide-mt940-mt101.pdf | |
| 423 | +- Citi Handlowy — SWIFT MT940 Export User Guide: https://www.citibank.pl/poland/files/SWIFT_MT940_Export_User_Guide_EN.pdf | |
| 424 | +- Huntington Developer Portal — SWIFT MT940: https://developer.huntington.com/enterprisepayments/docs/swift-mt-940 | |
| 425 | +- Danske Bank — MT940 with structured :86: https://danskeci.com/-/media/pdf/danskeci-com/swift-mt/reconciliation/mt940_structured.pdf | |
| 426 | + | |
| 427 | +### MT → MX migration | |
| 428 | +- SWIFT — ISO 20022 for Financial Institutions: https://www.swift.com/standards/iso-20022/iso-20022-financial-institutions-focus-payments-instructions | |
| 429 | +- J.P. Morgan — ISO 20022 Migration guidance: https://www.jpmorgan.com/insights/payments/fx-cross-border/iso-20022-migration | |
| 430 | +- Citi — ISO 20022 Migration FAQ: https://www.citibank.com/tts/sa/iso-20022-migration/assets/docs/ISO-20022-FAQs.pdf | |
| 431 | +- RedCompass Labs — ISO 20022 deadlines in 2026 onward: https://www.redcompasslabs.com/insights/what-now-iso-20022-deadlines-in-2026-onwards/ | |
| 432 | +- PaymentExpert — Swift's ISO 20022 cutover (Nov 2025): https://paymentexpert.com/2025/11/21/swifts-iso-20022-cutover-the-end-of-mt-and-a-20-year-promise/ | |
| 433 | +- ING — FAQ Swift ISO 20022: https://www.ingwb.com/en/service/payments-and-collections/swift-iso20022/faq-swift-iso-20022 | |
| 434 | + | |
| 435 | +### Reconciliation practice | |
| 436 | +- Oracle Fusion Cloud Financials — Overview of Tolerance Rules: https://docs.oracle.com/en/cloud/saas/financials/25d/fairp/overview-of-tolerance-rules.html | |
| 437 | +- FHP Accounting — Xero bank reconciliation rules & cash coding: https://fhpaccounting.co.uk/xero-bank-reconciliation-like-a-pro-master-rules-cash-coding-and-error-prevention-techniques/ | |
| 438 | +- The IQ Suite — Bank reconciliation in QuickBooks Online / Xero guides: https://bankreconciler.app/blogQuickBooksReconciliation , https://bankreconciler.app/blogXeroReconciliation | |
| 439 | + | |
| 440 | +### Aggregator APIs | |
| 441 | +- Plaid — Transactions API reference: https://plaid.com/docs/api/products/transactions/ | |
| 442 | +- Flinks — /GetAccountsDetail: https://docs.flinks.com/api/connect/endpoints/account-linking/get-accounts-detail | |
| 443 | +- Flinks — Data types in GetAccountsDetail: https://help.flinks.com/support/solutions/articles/43000705311-data-types-in-getaccountsdetail | |
| 444 | + | |
| 445 | +--- | |
| 446 | + | |
| 447 | +*End of research note — AIR project — Simon-Pierre Boucher — contact@spboucher.ai* | |
added
docs/research/canada-gst-qst.md
+240 −0
@@ -0,0 +1,240 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : canada-gst-qst.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Canadian Sales Tax Rules — GST / QST (Focus: Quebec) | |
| 9 | + | |
| 10 | +**Research date (all sources consulted): 2026-08-05** | |
| 11 | +**Status: rates and rules verified against official government sources (CRA / canada.ca, Revenu Québec, Justice Laws Canada).** | |
| 12 | + | |
| 13 | +This document is the citable source required by CLAUDE.md §1 before coding any tax rule. | |
| 14 | +Per project rule §5.6, none of these rates may be hardcoded in the engine — they belong in | |
| 15 | +versioned ALSL policies, with this document as the cited source. | |
| 16 | + | |
| 17 | +--- | |
| 18 | + | |
| 19 | +## 1. Current rates (verified 2026-08-05) | |
| 20 | + | |
| 21 | +| Tax | Rate | In effect since | Official source | | |
| 22 | +|---|---|---|---| | |
| 23 | +| GST (federal Goods and Services Tax) | **5%** | January 1, 2008 | Revenu Québec, "Tables of GST and QST Rates" [1]; CRA rate table [2] | | |
| 24 | +| QST (Quebec Sales Tax) | **9.975%** | January 1, 2013 | Revenu Québec, "Tables of GST and QST Rates" [1]; CRA rate table [2] | | |
| 25 | +| Combined effective rate in Quebec | **14.975%** | January 1, 2013 | Revenu Québec, "Calculating the Taxes" [3] | | |
| 26 | + | |
| 27 | +Historical GST rates (for processing back-dated documents): 7% (1991-01-01 to 2006-06-30), | |
| 28 | +6% (2006-07-01 to 2007-12-31), 5% (2008-01-01 to present). Historical QST rates: 6.5% | |
| 29 | +(1994-05-13 to 1997-12-31), 7.5% (1998 to 2010), 8.5% (2011), 9.5% (2012), 9.975% | |
| 30 | +(2013-01-01 to present). Source: [1]. | |
| 31 | + | |
| 32 | +- [1] https://www.revenuquebec.ca/en/businesses/consumption-taxes/gsthst-and-qst/basic-rules-for-applying-the-gsthst-and-qst/tables-of-gst-and-qst-rates/ (consulted 2026-08-05) | |
| 33 | +- [2] https://www.canada.ca/en/revenue-agency/services/tax/businesses/topics/gst-hst-businesses/charge-collect-which-rate.html (consulted 2026-08-05) — table lists Quebec: GST 5%, PST(QST) 9.975% | |
| 34 | +- [3] https://www.revenuquebec.ca/en/businesses/consumption-taxes/gsthst-and-qst/collecting-gst-and-qst/calculating-the-taxes/ (consulted 2026-08-05) | |
| 35 | + | |
| 36 | +## 2. QST calculation base (relative to GST) | |
| 37 | + | |
| 38 | +**Confirmed: since January 1, 2013, the QST is calculated on the selling price NOT including | |
| 39 | +GST.** GST and QST are both computed on the same pre-tax base. Revenu Québec's official page | |
| 40 | +"Calculating the Taxes" [3] describes the two accepted methods: | |
| 41 | + | |
| 42 | +- **Two-step calculation**: compute 5% GST on the sale price, then compute 9.975% QST *on the | |
| 43 | + same sale price* (not on price + GST). | |
| 44 | +- **One-step calculation**: apply a single combined rate of **14.975%** to the sale price. | |
| 45 | + | |
| 46 | +Both methods yield the same total. (Before 2013 the QST was applied on the GST-included price; | |
| 47 | +that regime is obsolete and must never be used for current documents.) | |
| 48 | + | |
| 49 | +Cash-register tolerance (from [3]): the 9.975% rate may be rounded to 9.97% (and 14.975% to | |
| 50 | +14.97%) **only** if the cash register cannot process three-decimal rates. AIR/AIC uses exact | |
| 51 | +decimals, so this tolerance must NOT be used — always 0.09975 / 0.14975. | |
| 52 | + | |
| 53 | +Combined effective rate: 5% + 9.975% = **14.975%** of the pre-tax price. | |
| 54 | + | |
| 55 | +## 3. Rates across Canada (for completeness) | |
| 56 | + | |
| 57 | +From the CRA official table "Charge and collect the tax – Which rate to charge" [2], | |
| 58 | +consulted 2026-08-05: | |
| 59 | + | |
| 60 | +### HST participating provinces (single harmonized tax, replaces GST + provincial tax) | |
| 61 | +| Province | HST rate | Notes | | |
| 62 | +|---|---|---| | |
| 63 | +| Ontario (ON) | **13%** | | | |
| 64 | +| Nova Scotia (NS) | **14%** | **Decreased from 15% on April 1, 2025** (provincial portion cut to 9%). Transitional rules: GST/HST Notices 342 and 343 [2a] | | |
| 65 | +| New Brunswick (NB) | **15%** | | | |
| 66 | +| Prince Edward Island (PE) | **15%** | | | |
| 67 | +| Newfoundland and Labrador (NL) | **15%** | | | |
| 68 | + | |
| 69 | +### GST only (5%) | |
| 70 | +Alberta (AB), Northwest Territories (NT), Nunavut (NU), Yukon (YT). | |
| 71 | + | |
| 72 | +### GST + separate provincial sales tax (provincial tax NOT federally administered) | |
| 73 | +| Province | GST | Provincial tax | | |
| 74 | +|---|---|---| | |
| 75 | +| Quebec (QC) | 5% | QST 9.975% (value-added, administered by Revenu Québec) | | |
| 76 | +| British Columbia (BC) | 5% | PST 7% | | |
| 77 | +| Saskatchewan (SK) | 5% | PST 6% | | |
| 78 | +| Manitoba (MB) | 5% | RST 7% | | |
| 79 | + | |
| 80 | +- [2a] Nova Scotia transition: https://www.canada.ca/en/revenue-agency/services/forms-publications/publications/notice342.html and .../notice343.html (linked from [2], consulted 2026-08-05) | |
| 81 | + | |
| 82 | +Administration note: under a federal–Quebec agreement, **Revenu Québec administers the GST/HST | |
| 83 | +in Quebec** on behalf of the CRA; Quebec businesses register and file with Revenu Québec for | |
| 84 | +both taxes. Source: https://www.revenuquebec.ca/en/businesses/consumption-taxes/gsthst-and-qst/basic-rules-for-applying-the-gsthst-and-qst/ (consulted 2026-08-05). | |
| 85 | + | |
| 86 | +## 4. Place-of-supply rules (which province's tax applies) | |
| 87 | + | |
| 88 | +Official source: CRA, "GST/HST rates and place-of-supply rules" [4], consulted 2026-08-05. | |
| 89 | +Detailed technical references: GST/HST Memorandum 3-3 (Place of Supply) and Technical | |
| 90 | +Information Bulletin B-103. | |
| 91 | + | |
| 92 | +- **Zero-rated supplies** (e.g. basic groceries, prescription drugs, exports): 0% GST/HST | |
| 93 | + regardless of place of supply anywhere in Canada. | |
| 94 | +- **Goods (tangible personal property) sold**: the place of supply is the province where the | |
| 95 | + goods are **delivered or made available** to the recipient (legal delivery per the | |
| 96 | + agreement). If the supplier ships the goods or arranges shipping to an address in another | |
| 97 | + province, delivery is considered to occur in that destination province. CRA example: a BC | |
| 98 | + store delivering a mattress to a customer in Ontario charges 13% ON HST. If goods are sold | |
| 99 | + with agreed delivery to an address but never delivered, the place of supply is the province | |
| 100 | + where delivery was supposed to occur. | |
| 101 | +- **Goods leased** (> 3 months): each lease interval is a separate supply; specific rules per | |
| 102 | + interval. Short leases (≤ 3 months): generally where the goods are provided (CRA example: | |
| 103 | + camera rented in Nova Scotia → NS HST even if used across Canada). | |
| 104 | +- **Services — general rules** (apply unless a specific rule overrides): | |
| 105 | + 1. **Rule 1**: place of supply is the province of the **recipient's address** obtained by | |
| 106 | + the supplier in the normal course of business (e.g. Quebec supplier designs a website for | |
| 107 | + an Ontario company → ON HST 13%). | |
| 108 | + 2. **Rule 2a**: if no address is obtained and the Canadian part of the service is NOT | |
| 109 | + performed primarily (> 50%) in participating provinces → the non-participating province | |
| 110 | + where performed (GST only, plus QST if Quebec). | |
| 111 | + 3. **Rule 2b**: if performed primarily in participating provinces → the participating | |
| 112 | + province with the largest proportion of the service. | |
| 113 | + 4. **Rule 2c**: tie between participating provinces → the one with the highest HST rate. | |
| 114 | +- **Specific overriding rules** exist for: services related to real property (province where | |
| 115 | + the property is located), personal services, transportation, telecommunications, | |
| 116 | + computer-related services/Internet access, air navigation services, specified motor | |
| 117 | + vehicles (province of registration), etc. See [4] for the full list — implement these as | |
| 118 | + ALSL policies per category, not in the engine. | |
| 119 | +- **QST side**: the QST applies to supplies **made in Quebec** under mirroring place-of-supply | |
| 120 | + rules in the Quebec Sales Tax Act; Quebec GST/HST registrants must charge HST on their sales | |
| 121 | + made in participating provinces (Revenu Québec, "Basic Rules", consulted 2026-08-05). | |
| 122 | + | |
| 123 | +- [4] https://www.canada.ca/en/revenue-agency/services/tax/businesses/topics/gst-hst-businesses/charge-collect-place-supply.html (consulted 2026-08-05) | |
| 124 | + | |
| 125 | +## 5. Rounding rules (fractions of a cent) | |
| 126 | + | |
| 127 | +**Statutory rule — Excise Tax Act, s. 165.2 (verified directly on Justice Laws, 2026-08-05) [5]:** | |
| 128 | + | |
| 129 | +- **s. 165.2(2)**: where the tax payable includes a fraction of a cent, the fraction, | |
| 130 | + **if less than half a cent, may be disregarded**; **if equal to or greater than half a cent, | |
| 131 | + it is deemed to be one cent**. (Round-half-up at the cent, per tax amount.) | |
| 132 | +- **s. 165.2(1)**: when two or more taxable supplies at the same rate appear on one invoice, | |
| 133 | + the tax **may be calculated on the total consideration** (i.e., compute tax once on the | |
| 134 | + invoice subtotal, then round — instead of rounding line by line). | |
| 135 | + | |
| 136 | +**Revenu Québec guidance — publication IN-203-V, "General Information Concerning the QST and | |
| 137 | +the GST/HST" (current version 2026-03) [6][6a]:** identical rule for both taxes: "Only | |
| 138 | +fractions equal to or greater than one-half of a cent ($0.005) are counted as a whole cent | |
| 139 | +($0.01) of sales tax. If more than one good or service is being sold, you can calculate the | |
| 140 | +taxes on the total price of all the goods or services purchased before rounding off the | |
| 141 | +fractions." | |
| 142 | + | |
| 143 | +**Cash rounding (penny elimination)** is a separate, later step: only the **final cash total | |
| 144 | +after GST/HST is calculated** is rounded to the nearest $0.05; electronic/cheque payments | |
| 145 | +settle to the cent, and tax is always computed on the pre-rounding price (Government of | |
| 146 | +Canada, Budget 2012 "Eliminating the Penny" [7]). | |
| 147 | + | |
| 148 | +**Decision for AIR/AIC**: compute tax in exact decimal (no floats), round each tax (GST, QST) | |
| 149 | +to the cent with round-half-up, at the invoice-total level by default (s. 165.2(1)), with a | |
| 150 | +per-line option; model cash rounding as a distinct posting line, never inside tax calculation. | |
| 151 | +Note: this statutory rule is round-HALF-UP, not banker's rounding — the CLAUDE.md §4 mention | |
| 152 | +of banker's rounding must NOT be applied to Canadian GST/QST. | |
| 153 | + | |
| 154 | +- [5] https://laws-lois.justice.gc.ca/eng/acts/E-15/section-165.2.html (consulted 2026-08-05) | |
| 155 | +- [6] https://www.revenuquebec.ca/en/online-services/forms-and-publications/current-details/in-203-v/ (consulted 2026-08-05 — confirms current version 2026-03) | |
| 156 | +- [6a] Wording quoted from the IN-203-V publication text as retrieved via a mirrored copy of the official Revenu Québec PDF (ryan.com/contentassets/.../qc-in-203-v.pdf, consulted 2026-08-05). See "flags" in §8. | |
| 157 | +- [7] https://www.budget.canada.ca/2012/themes/theme2-eng.pdf (consulted 2026-08-05) | |
| 158 | + | |
| 159 | +## 6. Registration — small supplier threshold | |
| 160 | + | |
| 161 | +**Confirmed from Revenu Québec's official information site (2026-08-05) [8]:** a business must | |
| 162 | +register for the GST/HST and QST as soon as its sales of taxable goods and services (including | |
| 163 | +zero-rated sales) **exceed $30,000 in a single calendar quarter or total $30,000 over the past | |
| 164 | +four calendar quarters**. The calculation includes worldwide taxable and zero-rated sales, | |
| 165 | +including those of associates, and excludes GST/HST and QST amounts, financial services, | |
| 166 | +sales of capital property, and goodwill. Below that threshold, a "small supplier" is generally | |
| 167 | +not required to register (special rules exist, e.g. taxi/ride-sharing operators must register | |
| 168 | +regardless). | |
| 169 | + | |
| 170 | +Once registered: collect both taxes, file returns (usually combined GST-QST in Quebec), and | |
| 171 | +claim **input tax credits (ITCs)** for GST and **input tax refunds (ITRs)** for QST paid on | |
| 172 | +business inputs [8]. | |
| 173 | + | |
| 174 | +- [8] https://justepourtous.revenuquebec.ca/en/profiles/self-employed-workers/ (Revenu Québec official "Fair for all" site, consulted 2026-08-05); also covered in IN-203-V [6]. CRA reference page: "When to register for and start charging the GST/HST" (canada.ca) — see flag in §8. | |
| 175 | + | |
| 176 | +## 7. Worked example — taxable sale of 1,000.00 CAD in Quebec (CA-QC) | |
| 177 | + | |
| 178 | +Rates: GST 5% (0.05), QST 9.975% (0.09975), both applied on the pre-tax price (§2). | |
| 179 | +Rounding: half-up to the cent per ETA s. 165.2(2) (§5). | |
| 180 | + | |
| 181 | +``` | |
| 182 | +Subtotal (pre-tax) : 1,000.00 CAD | |
| 183 | +GST = 1,000.00 × 0.05 = 50.00000 → rounds to 50.00 CAD (exact, no fraction) | |
| 184 | +QST = 1,000.00 × 0.09975 = 99.75000 → rounds to 99.75 CAD (exact, no fraction) | |
| 185 | +Total = 1,000.00 + 50.00 + 99.75 = 1,149.75 CAD | |
| 186 | +``` | |
| 187 | + | |
| 188 | +Cross-check with the one-step combined rate: 1,000.00 × 0.14975 = 149.75 total tax. ✓ | |
| 189 | + | |
| 190 | +**Supplementary example where rounding actually triggers** (subtotal 19.99 CAD, QC): | |
| 191 | + | |
| 192 | +``` | |
| 193 | +GST = 19.99 × 0.05 = 0.99950 → fraction 0.50¢ ≥ half cent → 1.00 CAD (round up) | |
| 194 | +QST = 19.99 × 0.09975 = 1.99400 → fraction 0.40¢ < half cent → 1.99 CAD (round down) | |
| 195 | +Total = 19.99 + 1.00 + 1.99 = 22.98 CAD | |
| 196 | +``` | |
| 197 | + | |
| 198 | +These two cases are suitable as golden tests (exact-decimal arithmetic, Decimal type, | |
| 199 | +never float). | |
| 200 | + | |
| 201 | +## 8. Flags — items not fully confirmed from a primary official page | |
| 202 | + | |
| 203 | +1. **IN-203-V exact rounding wording**: the current-version landing page on revenuquebec.ca | |
| 204 | + (2026-03 version) was confirmed directly, but the PDF body text quoted in §5 was read from | |
| 205 | + a mirrored copy of the official publication (ryan.com), because revenuquebec.ca blocks | |
| 206 | + automated PDF retrieval. The rule is independently anchored in ETA s. 165.2 (verified | |
| 207 | + directly on laws-lois.justice.gc.ca). Risk: low. Action: a human should spot-check the | |
| 208 | + IN-203-V PDF once before shipping the QC tax pass. | |
| 209 | +2. **CRA "When to register" page**: canada.ca returned HTTP 403 to automated fetching for | |
| 210 | + this specific page; the $30,000 small supplier threshold was instead confirmed on Revenu | |
| 211 | + Québec's official justepourtous.revenuquebec.ca site (which administers GST/HST in Quebec) | |
| 212 | + and is described in IN-203-V. Risk: low. | |
| 213 | +3. **QST place-of-supply specifics** (Quebec Sales Tax Act mirroring rules, e.g. supplies | |
| 214 | + between QC and other provinces): the general principle is confirmed; the detailed QST | |
| 215 | + place-of-supply provisions were NOT researched in depth here. **Required follow-up research | |
| 216 | + before implementing cross-province QST logic.** | |
| 217 | +4. **PST/RST details for BC/SK/MB** (bases, exemptions): rates confirmed via the CRA table | |
| 218 | + only; these provinces' own tax authorities were not consulted. Out of scope for the QC-first | |
| 219 | + Tax pass; research before adding those jurisdictions. | |
| 220 | +5. Rates change over time (e.g. Nova Scotia April 1, 2025). All rates above carry effective | |
| 221 | + dates and must live in **versioned ALSL policies keyed by jurisdiction and effective-date | |
| 222 | + ranges**, never in engine code. | |
| 223 | + | |
| 224 | +--- | |
| 225 | + | |
| 226 | +## Source index (all consulted 2026-08-05) | |
| 227 | + | |
| 228 | +| # | Source | Publisher | URL | | |
| 229 | +|---|---|---|---| | |
| 230 | +| 1 | Tables of GST and QST Rates | Revenu Québec | https://www.revenuquebec.ca/en/businesses/consumption-taxes/gsthst-and-qst/basic-rules-for-applying-the-gsthst-and-qst/tables-of-gst-and-qst-rates/ | | |
| 231 | +| 2 | Charge and collect the tax – Which rate to charge | CRA / canada.ca | https://www.canada.ca/en/revenue-agency/services/tax/businesses/topics/gst-hst-businesses/charge-collect-which-rate.html | | |
| 232 | +| 3 | Calculating the Taxes | Revenu Québec | https://www.revenuquebec.ca/en/businesses/consumption-taxes/gsthst-and-qst/collecting-gst-and-qst/calculating-the-taxes/ | | |
| 233 | +| 4 | GST/HST rates and place-of-supply rules | CRA / canada.ca | https://www.canada.ca/en/revenue-agency/services/tax/businesses/topics/gst-hst-businesses/charge-collect-place-supply.html | | |
| 234 | +| 5 | Excise Tax Act, s. 165.2 | Justice Laws Canada | https://laws-lois.justice.gc.ca/eng/acts/E-15/section-165.2.html | | |
| 235 | +| 6 | IN-203-V, General Information Concerning the QST and the GST/HST (2026-03) | Revenu Québec | https://www.revenuquebec.ca/en/online-services/forms-and-publications/current-details/in-203-v/ | | |
| 236 | +| 7 | Budget 2012 — Eliminating the Penny | Government of Canada | https://www.budget.canada.ca/2012/themes/theme2-eng.pdf | | |
| 237 | +| 8 | Fair for all — Self-employed workers (registration threshold) | Revenu Québec | https://justepourtous.revenuquebec.ca/en/profiles/self-employed-workers/ | | |
| 238 | +| 9 | Basic Rules for Applying the GST/HST and QST | Revenu Québec | https://www.revenuquebec.ca/en/businesses/consumption-taxes/gsthst-and-qst/basic-rules-for-applying-the-gsthst-and-qst/ | | |
| 239 | + | |
| 240 | +*End of research document — AIR project — Simon-Pierre Boucher — contact@spboucher.ai* | |
added
docs/research/erp-apis.md
+194 −0
@@ -0,0 +1,194 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : erp-apis.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# ERP Accounting APIs — Research Report | |
| 9 | + | |
| 10 | +**Purpose:** Ground the design of AIR's `Backend` interface (`capabilities() / compile() / post() / reverse()`) in the reality of the ERP APIs we intend to target. | |
| 11 | +**Research method:** 9 targeted web searches/fetches, official developer documentation prioritized. | |
| 12 | +**Consultation date: 2026-08-05.** | |
| 13 | + | |
| 14 | +--- | |
| 15 | + | |
| 16 | +## 1. QuickBooks Online (QBO) — AIR's first real backend | |
| 17 | + | |
| 18 | +### 1.1 API model | |
| 19 | + | |
| 20 | +- REST + JSON, base URL `https://quickbooks.api.intuit.com/v3/company/{realmId}/...` (sandbox: `https://sandbox-quickbooks.api.intuit.com/...`). The `realmId` is the company ID, obtained during the OAuth flow. | |
| 21 | +- **Minor versions:** since **August 1, 2025**, minor versions 1–74 are deprecated; **minor version 75 is the base/default** and any `minorversion` parameter below 75 is ignored. Recommendation: pin requests explicitly (`?minorversion=75`) and validate schema compatibility against mv75. | |
| 22 | +- **Rate limits:** ~500 requests/minute per company (realm ID), max ~10 concurrent requests, and ~120 requests/minute on the Batch endpoint. Throttling returns HTTP 429 → backend must implement backoff + retry. | |
| 23 | + | |
| 24 | +### 1.2 Authentication | |
| 25 | + | |
| 26 | +- OAuth 2.0 Authorization Code flow via Intuit's identity platform. Access tokens expire after **1 hour**; refresh tokens must be used (and rotated/persisted). Scope: `com.intuit.quickbooks.accounting`. | |
| 27 | +- Free **sandbox companies** are available in the developer dashboard for full end-to-end testing (Phase 3 target). | |
| 28 | + | |
| 29 | +### 1.3 JournalEntry entity | |
| 30 | + | |
| 31 | +Reference: https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry | |
| 32 | + | |
| 33 | +- A `JournalEntry` is a header + array of `Line` objects. | |
| 34 | +- Each line: `Description`, `Amount` (always **positive**), `DetailType: "JournalEntryLineDetail"`, and a `JournalEntryLineDetail` object containing: | |
| 35 | + - `PostingType`: **`"Debit"` or `"Credit"`** — the debit/credit convention is an *explicit enum per line*, with an unsigned amount. | |
| 36 | + - `AccountRef` (`value` = account ID, `name` optional) — reference to the Account. | |
| 37 | + - Optional `Entity` (Customer/Vendor/Employee reference — required for lines on AR/AP accounts), class/department refs, `TaxCodeRef`/`TaxApplicableOn` for tax lines. | |
| 38 | +- Header: `TxnDate`, `DocNumber`, `PrivateNote`, `CurrencyRef` + `ExchangeRate` (multicurrency companies). | |
| 39 | +- Operations: create (POST), read, full update, sparse update, **delete** (`?operation=delete` with Id + SyncToken). QBO uses **optimistic concurrency via `SyncToken`** — every update/delete must supply the current SyncToken or fails. | |
| 40 | +- **No native "reverse" operation on JournalEntry**: reversal = either delete the entry, or post a mirror-image journal entry (swap Debit/Credit) — AIR should generate an explicit reversal entry to preserve the audit trail rather than deleting. | |
| 41 | + | |
| 42 | +### 1.4 Invoice and Payment entities | |
| 43 | + | |
| 44 | +- `Invoice`: header (CustomerRef, TxnDate, DueDate, CurrencyRef, DocNumber) + `Line[]` with `DetailType: "SalesItemLineDetail"` (`ItemRef`, `Qty`, `UnitPrice`, `TaxCodeRef`). Tax computed via `TxnTaxDetail` (and Automated Sales Tax in US companies). | |
| 45 | +- `Payment`: separate entity linked to invoices via **`LinkedTxn`** (`TxnId`, `TxnType: "Invoice"`). `LinkedTxn` on the Invoice side is read-only; linkage is written from the Payment side. | |
| 46 | +- This matters for AIR: a `Sale` economic event may compile to an Invoice + Payment pair rather than a raw JournalEntry — the backend's `capabilities()` should advertise which target document types it can emit. | |
| 47 | + | |
| 48 | +### 1.5 Idempotency | |
| 49 | + | |
| 50 | +- QBO does **not** support modern `Idempotency-Key` headers. Two mechanisms: | |
| 51 | + 1. **`requestid` query parameter** (a UUID unique per company): if the service receives a duplicate `requestid`, it returns the original response instead of re-executing. Intuit strongly recommends sending it on every mutating request — this is our primary idempotency mechanism. | |
| 52 | + 2. Fallback: enable `CustomTxnNumbers` and rely on `DocNumber` uniqueness (duplicate → error 6140), usable as a duplicate detector. | |
| 53 | + | |
| 54 | +Sources (consulted 2026-08-05): | |
| 55 | +- https://developer.intuit.com/app/developer/qbo/docs/api/accounting/all-entities/journalentry | |
| 56 | +- https://developer.intuit.com/app/developer/qbo/docs/workflows/manage-linked-transactions | |
| 57 | +- https://blogs.intuit.com/2015/04/06/15346/ (Request ID / idempotency) | |
| 58 | +- https://medium.com/intuitdev/changes-to-our-accounting-api-that-may-impact-your-application-c330bd1a06f5 (minor version 75) | |
| 59 | +- https://docs.codat.io/updates/250219-qbo-minor-versions-update/ | |
| 60 | +- https://satvasolutions.com/blog/quickbooks-online-api-guide (rate limits) | |
| 61 | +- https://truto.one/blog/how-to-integrate-with-the-quickbooks-online-api-2026-guide | |
| 62 | + | |
| 63 | +--- | |
| 64 | + | |
| 65 | +## 2. Xero Accounting API | |
| 66 | + | |
| 67 | +### 2.1 API model & auth | |
| 68 | + | |
| 69 | +- REST + JSON, base `https://api.xero.com/api.xro/2.0/...`. OAuth 2.0 only (OAuth 1.0a fully retired). | |
| 70 | +- **Tenant model:** one OAuth connection can grant access to multiple Xero organisations; **every request must carry the `Xero-Tenant-Id` header**. The backend must persist the (token, tenantId) pair per organisation. | |
| 71 | +- **Rate limits:** 60 calls/min per tenant, 5,000 calls/day per tenant, 10,000 calls/min app-wide. Daily per-tenant cap is the binding constraint for batch posting — favor batch endpoints (up to multiple elements per PUT/POST call). | |
| 72 | + | |
| 73 | +### 2.2 ManualJournals | |
| 74 | + | |
| 75 | +Reference: https://developer.xero.com/documentation/api/accounting/manualjournals | |
| 76 | + | |
| 77 | +- Fields: `Narration` (required), `Date`, `Status`, `JournalLines[]`. | |
| 78 | +- Each `JournalLine`: `LineAmount`, `AccountCode`, `Description`, `TaxType`, `Tracking` (analytic categories). | |
| 79 | +- **Debit/credit convention: signed amount** — **positive = debit, negative = credit**. No PostingType enum. At least one debit and one credit line; the journal must balance. | |
| 80 | +- `Status`: `DRAFT` or `POSTED` on creation; a posted manual journal is removed by setting status to **`VOIDED`** (drafts → `DELETED`). Voiding is the sanctioned "reversal" path, but AIR may still prefer an explicit contra-journal for traceability. | |
| 81 | +- Invoices: `Invoices` endpoint (Type ACCREC/ACCPAY, LineItems with LineAmountTypes Exclusive/Inclusive/NoTax), separate `Payments` endpoint. | |
| 82 | + | |
| 83 | +### 2.3 Idempotency | |
| 84 | + | |
| 85 | +- **Native support:** `Idempotency-Key` HTTP header (UUID) on POST/PUT — retries do not create duplicates. This is the cleanest idempotency model of all surveyed APIs. | |
| 86 | + | |
| 87 | +Sources (consulted 2026-08-05): | |
| 88 | +- https://developer.xero.com/documentation/api/accounting/manualjournals | |
| 89 | +- https://developer.xero.com/documentation/guides/oauth2/limits/ | |
| 90 | +- https://developer.xero.com/faq/limits | |
| 91 | +- https://xeroapi.github.io/xero-python/v1/accounting/index.html (idempotency_key parameter on ManualJournals operations) | |
| 92 | +- https://cdn.cdata.com/help/DXF/jdbc/pg_accountingtable-manualjournals.htm (sign convention, balance requirement) | |
| 93 | + | |
| 94 | +--- | |
| 95 | + | |
| 96 | +## 3. Odoo | |
| 97 | + | |
| 98 | +### 3.1 API model & auth | |
| 99 | + | |
| 100 | +- External API via **XML-RPC** (`/xmlrpc/2/common`, `/xmlrpc/2/object`) and **JSON-RPC** (`/jsonrpc`); a REST-ish surface exists since Odoo 17. All protocols hit the same ORM (`execute_kw(db, uid, password, model, method, args)`). | |
| 101 | +- **Deprecation:** `/xmlrpc`, `/xmlrpc/2` and `/jsonrpc` are scheduled for removal in **Odoo 22 (fall 2028)** / Online 21.1 (winter 2027), replaced by the **External JSON-2 API**. AIR's Odoo backend should abstract transport so the RPC layer can be swapped. | |
| 102 | +- Auth: database + login + API key (used as password in RPC). External API access requires a **Custom pricing plan** (not available on One App Free / Standard) — a real deployment constraint. | |
| 103 | + | |
| 104 | +### 3.2 Journal entries: `account.move` | |
| 105 | + | |
| 106 | +- One model for everything: journal entries, customer invoices, vendor bills, credit notes — discriminated by `move_type` (`entry`, `out_invoice`, `in_invoice`, `out_refund`, ...). | |
| 107 | +- Lines in **`line_ids`** (`account.move.line`), each with **separate `debit` and `credit` decimal fields** (one of the two is 0.0), `account_id`, `partner_id`, `name`. Invoice-style lines go through `invoice_line_ids`. | |
| 108 | +- Lifecycle via `state`: `draft` → `posted` → `cancel`. Post with **`action_post`** (`execute_kw(..., 'account.move', 'action_post', [[move_id]])`); back to draft with `button_draft`. | |
| 109 | +- Balance is validated at post time (debits = credits); to build temporarily unbalanced drafts pass context `{'check_move_validity': False}`. | |
| 110 | +- **Reversal:** first-class — `_reverse_moves` / reversal wizard creates a contra move with `reversed_entry_id` linking back to the original (good fit for AIR's provenance graph). | |
| 111 | + | |
| 112 | +### 3.3 Idempotency | |
| 113 | + | |
| 114 | +- **None built in.** RPC create calls are not idempotent; the backend must implement its own dedupe (e.g., store the AIR event ID in `ref`/a custom field and search before create). | |
| 115 | + | |
| 116 | +Sources (consulted 2026-08-05): | |
| 117 | +- https://www.odoo.com/documentation/19.0/developer/reference/external_rpc_api.html | |
| 118 | +- https://www.odoo.com/documentation/19.0/developer/reference/external_api.html (External JSON-2) | |
| 119 | +- https://github.com/odoo/odoo/blob/14.0/addons/account/models/account_move.py | |
| 120 | +- https://www.odoo.com/forum/help-1/creating-journal-entries-via-external-api-xml-rpc-130818 | |
| 121 | +- https://www.getknit.dev/blog/odoo-api-integration-guide-in-depth | |
| 122 | + | |
| 123 | +--- | |
| 124 | + | |
| 125 | +## 4. SAP S/4HANA (brief survey — defer details, complexity high) | |
| 126 | + | |
| 127 | +- **Modern path:** SOAP/OData APIs under the **Finance – Posting Integration (SAP_COM_0002)** communication scenario: *Journal Entry – Post* (synchronous and asynchronous) and *Journal Entry – Change*, plus OData read APIs (e.g., Operational Journal Entry Item – Read). Journal entries are complex documents: header (company code, document type, posting date) + typed item categories (G/L items, debtor items, creditor items, product tax items, withholding tax items). Debit/credit is expressed via signed amounts / debit-credit codes depending on the API. | |
| 128 | +- **Reversal:** a dedicated *Journal Entry – Reverse* service; requires `ReversalReason` and `ReversalReferenceDocument`, with **no line items** — reversal is a header-level operation referencing the original document. Sync and async variants exist. | |
| 129 | +- **Legacy path:** `BAPI_ACC_DOCUMENT_POST` (post) and `BAPI_ACC_DOCUMENT_CHECK` (validate-only — interesting analogue of AIR's `Validate()`), plus IDoc-based batch interfaces. On-premise landscapes still rely heavily on these. | |
| 130 | +- Complexity drivers: communication arrangements setup, document splitting, parallel ledgers, tax determination inside SAP, strict field controls per document type. **Decision: defer SAP backend; requires a dedicated deep research pass (OData vs SOAP vs BAPI, S/4HANA Cloud vs on-prem).** | |
| 131 | + | |
| 132 | +Sources (consulted 2026-08-05): | |
| 133 | +- https://community.sap.com/t5/technology-blog-posts-by-sap/apis-for-journal-entries-the-collection-updated-july-2025/ba-p/13565258 | |
| 134 | +- https://community.sap.com/t5/enterprise-resource-planning-blog-posts-by-sap/guidelines-for-api-journal-entry-post/ba-p/13421397 | |
| 135 | +- https://help.sap.com/docs/SAP_S4HANA_CLOUD/b978f98fc5884ff2aeb10c8fdeb8a43b/57b40036b71f4825adad70a0a5b91573.html (Journal Entry – Reverse) | |
| 136 | +- https://community.sap.com/t5/enterprise-resource-planning-blog-posts-by-sap/user-guide-for-journal-entry-post-api-on-s-4-hana-cloud/ba-p/13424193 | |
| 137 | + | |
| 138 | +--- | |
| 139 | + | |
| 140 | +## 5. NetSuite SuiteTalk & Sage Intacct (brief capability survey) | |
| 141 | + | |
| 142 | +### 5.1 NetSuite (SuiteTalk REST) | |
| 143 | + | |
| 144 | +- SuiteTalk REST Web Services: JSON, standard verbs (GET/POST/PATCH/DELETE) on records including **`journalEntry`** (`/services/rest/record/v1/journalEntry`). Legacy SOAP SuiteTalk still exists. | |
| 145 | +- Journal entry = header (subsidiary, currency, trandate) + `line` sublist; each line has account reference and **separate `debit` / `credit` fields** (Odoo-like convention). | |
| 146 | +- Auth: **Token-Based Authentication (OAuth 1.0-signed, via an Integration Record with consumer key/secret + token)** or **OAuth 2.0** (auth code and client-credentials machine-to-machine flows). Setup ceremony is significant (integration record, roles, permissions). | |
| 147 | +- Reversal: native support via reversal date fields on the journal entry record (creates a linked reversing entry). No native idempotency-key header; use `externalId` on records for dedupe/upsert. | |
| 148 | + | |
| 149 | +Sources (consulted 2026-08-05): | |
| 150 | +- https://www.moderntreasury.com/journal/how-to-authenticate-to-netsuites-suitetalk-rest-web-services-api | |
| 151 | +- https://yepcode.io/recipes/rest-api-to-oracle-netsuite-journal-entries/ | |
| 152 | +- https://www.brokenrubik.com/blog/netsuite-rest-api-guide | |
| 153 | + | |
| 154 | +### 5.2 Sage Intacct | |
| 155 | + | |
| 156 | +- **Two API surfaces:** legacy **XML API** (envelope: `<request><control><operation><content>`, two-level authentication: sender ID/password + company/user credentials) and a newer **REST API with OAuth 2.0** — all new objects/features ship on REST only. | |
| 157 | +- Journal entries: XML API `GLBATCH` object (journal entry is an owned element of a GL transaction/batch — header journal + `GLENTRY` lines with `TR_TYPE`/amount or debit-credit indicators); REST API exposes `general-ledger/journal-entries` create endpoint on the developer portal. | |
| 158 | +- Dimensions (department, location, class, project) are pervasive on lines — the AIR line model must carry analytic dimensions generically. | |
| 159 | + | |
| 160 | +Sources (consulted 2026-08-05): | |
| 161 | +- https://developer.intacct.com/api/general-ledger/journal-entries/ | |
| 162 | +- https://developer.sage.com/intacct/apis/intacct/1/intacct-openapi/groups/general-ledger/groups/journal-entries/tags/general_ledger_journal-entries/paths/create-general-ledger-journal-entry | |
| 163 | +- https://www.getknit.dev/blog/sage-intacct-api-integration-guide-in-depth | |
| 164 | + | |
| 165 | +--- | |
| 166 | + | |
| 167 | +## 6. Comparison matrix | |
| 168 | + | |
| 169 | +| Aspect | QBO | Xero | Odoo | SAP S/4HANA | NetSuite | Intacct | | |
| 170 | +|---|---|---|---|---|---|---| | |
| 171 | +| Protocol | REST/JSON | REST/JSON | XML-RPC/JSON-RPC (→ JSON-2) | OData/SOAP (+BAPI/IDoc) | REST/JSON (+SOAP) | XML + REST | | |
| 172 | +| Auth | OAuth 2.0 (1h tokens) | OAuth 2.0 + Xero-Tenant-Id | API key via RPC login | Comm. arrangement / OAuth | TBA (OAuth1) or OAuth 2.0 | 2-level XML creds / OAuth 2.0 | | |
| 173 | +| D/C convention | `PostingType` enum + unsigned amount | **signed** LineAmount (+debit / −credit) | separate `debit`/`credit` fields | item categories + D/C codes | separate `debit`/`credit` fields | GLENTRY type/amount | | |
| 174 | +| Idempotency | `requestid` query param | `Idempotency-Key` header | none (manual dedupe) | async msg IDs / none simple | `externalId` upsert | none simple | | |
| 175 | +| Reversal | delete or contra entry (manual) | VOID status | `_reverse_moves` (linked contra) | dedicated Reverse API (reason + ref) | native reversal date | contra entry | | |
| 176 | +| Rate limits | 500/min/realm | 60/min + 5k/day/tenant | server-dependent | gateway-dependent | account-tier concurrency | account-tier | | |
| 177 | +| Sandbox | free sandbox company | demo company | local/docker instance | trial/CAL complex | dev account (paid) | dev account | | |
| 178 | + | |
| 179 | +--- | |
| 180 | + | |
| 181 | +## 7. Implications for AIR backend interface | |
| 182 | + | |
| 183 | +The common `Backend` interface (`capabilities() / compile() / post() / reverse()`) must accommodate: | |
| 184 | + | |
| 185 | +1. **Debit/credit representation is backend-specific.** AIR's `CompiledJournal` should use one canonical internal convention (explicit `side: Debit|Credit` + unsigned `Decimal` amount is the safest, lossless superset); each backend's `compile()` maps it to signed amounts (Xero), PostingType enums (QBO), or paired debit/credit fields (Odoo, NetSuite). | |
| 186 | +2. **`capabilities()` must be rich**, declaring at minimum: supported document types (raw journal entry vs invoice/payment documents), multicurrency support, tax-line handling (does the target recompute tax — QBO AST, SAP — or accept our tax lines — Xero, Odoo), analytic dimensions (Xero Tracking, Intacct dimensions, Odoo analytic), draft-then-post vs direct post, max lines/batch sizes, and **native idempotency mechanism**. | |
| 187 | +3. **Idempotency is an adapter concern with a common contract.** The interface should require: `post(payload, idempotency_key)` where the key is derived from the AIR event ID + policy version. Adapters map it to `Idempotency-Key` (Xero), `requestid` (QBO), `externalId` (NetSuite), or a search-before-create fallback (Odoo, Intacct). Every backend MUST persist the (AIR event → target document ID) mapping in the `PostingReceipt`. | |
| 188 | +4. **`reverse()` must support three strategies**, selected via capabilities: (a) native reversal API (SAP, NetSuite, Odoo `_reverse_moves`), (b) status change/void (Xero VOIDED), (c) synthesized contra entry (QBO, Intacct, generic CSV). Regardless of mechanism, AIR always records a reversal event in the provenance graph; deletion of posted entries is forbidden even where the API allows it (QBO delete). | |
| 189 | +5. **Concurrency/version tokens**: receipts must carry target-side version tokens (QBO `SyncToken`) so later reverse/update operations don't fail on optimistic locking. | |
| 190 | +6. **Rate limiting and retries belong in a shared posting runtime** (token bucket per realm/tenant, exponential backoff on 429/503, batch endpoints where available: QBO Batch ≤ ~30 ops, Xero batched PUT/POST) — not reimplemented per backend. | |
| 191 | +7. **Multi-tenancy is first-class:** a backend connection = (credentials, tenant/realm/db identifier, environment sandbox|prod). QBO realmId, Xero tenantId, Odoo db, SAP company code must be part of the connection config, never of the compiled journal. | |
| 192 | +8. **API versioning must be pinned per backend** (QBO `minorversion=75`, Odoo transport migration to JSON-2 by 2028, Intacct XML→REST) and surfaced in `capabilities()` so golden tests can be version-scoped. | |
| 193 | +9. **Validate-before-post** is natively supported by some targets (SAP `BAPI_ACC_DOCUMENT_CHECK`, Odoo draft state, Xero DRAFT status) — the interface should expose an optional `dry_run`/draft mode used by AIR's Approval pass before final posting. | |
| 194 | +10. **Ordering of backend development confirmed:** generic CSV → QBO (best docs, free sandbox, simple entity model) → Xero (cleanest idempotency) → Odoo (open source, self-hostable test instance) → SAP last (deep dedicated research required). | |
added
docs/research/fx-handling.md
+96 −0
@@ -0,0 +1,96 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : fx-handling.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Research: Foreign Exchange Handling (Rate Sources, CRA Rules, System Design) | |
| 9 | + | |
| 10 | +**Date of research:** 2026-08-05 | |
| 11 | +**Method:** 4 targeted web searches (Bank of Canada Valet API / noon-rate history, CRA acceptable rates, CRA Folio S5-F4-C1, multi-currency system design). | |
| 12 | +**Purpose:** Define where AIR's FX pass gets its rates, which rate the Canadian tax authority accepts, and how the compiler must record FX in the ledger. Complements `accounting-standards.md` §3 (IAS 21 semantics). | |
| 13 | + | |
| 14 | +--- | |
| 15 | + | |
| 16 | +## 1. Bank of Canada exchange rates and the Valet API | |
| 17 | + | |
| 18 | +- The historical **noon rate** (and closing rate) was **discontinued**: legacy noon/closing rates were last updated **28 April 2017** and are frozen. Since 1 March 2017 the Bank publishes a single **daily average exchange rate** per currency pair. | |
| 19 | +- Daily average rates are published **once each business day by 16:30 ET**, for CAD against ~26 currencies. The Bank also publishes **monthly** and **annual** average rates. | |
| 20 | +- All Bank of Canada rates are **indicative** — averages of aggregated price quotes from financial institutions, not transactable rates. | |
| 21 | +- The **Valet API** is the Bank's free, no-registration web API for all of this data: | |
| 22 | + - Docs: https://www.bankofcanada.ca/valet/docs | |
| 23 | + - Example series: `FXUSDCAD` (USD→CAD daily average), e.g. `GET https://www.bankofcanada.ca/valet/observations/FXUSDCAD/json?start_date=2026-01-01` | |
| 24 | + - Group endpoints exist for all daily FX rates; RSS feeds per series (e.g. https://www.bankofcanada.ca/valet/fx_rss/FXUSDCAD). | |
| 25 | + | |
| 26 | +### AIR design consequences | |
| 27 | + | |
| 28 | +- Build a **rate provider service** in the kernel (`kernel/fx`) with the Valet API as the primary CAD source; cache observations locally so compilation is **deterministic and replayable** (a recompilation must use the same stored rate, never a live fetch). | |
| 29 | +- Every converted amount in the provenance graph carries `{rate, series (e.g. FXUSDCAD), rate_date, provider: bank_of_canada_valet, retrieved_at}`. | |
| 30 | +- **No business-day = no rate**: weekends/holidays require an explicit, documented convention (e.g. last published business-day rate) — a policy parameter, not a hard-coded choice. | |
| 31 | +- **Cross rates** (e.g. USD→EUR) must be derived through CAD from two BoC series when operating under Canadian tax rules (see §2). | |
| 32 | + | |
| 33 | +Sources (consulted 2026-08-05): | |
| 34 | +- Bank of Canada — Daily exchange rates: https://www.bankofcanada.ca/rates/exchange/daily-exchange-rates/ | |
| 35 | +- Bank of Canada — Valet API docs: https://www.bankofcanada.ca/valet/docs | |
| 36 | +- Bank of Canada — Valet API how-to guide: https://www.bankofcanada.ca/valet-api-how-to/ | |
| 37 | +- Bank of Canada — Legacy noon and closing rates: https://www.bankofcanada.ca/rates/exchange/legacy-noon-and-closing-rates/ | |
| 38 | +- Bank of Canada — Annual average exchange rates: https://www.bankofcanada.ca/rates/exchange/annual-average-exchange-rates/ | |
| 39 | + | |
| 40 | +--- | |
| 41 | + | |
| 42 | +## 2. CRA rules: which rate to use | |
| 43 | + | |
| 44 | +Primary source: **Income Tax Folio S5-F4-C1, Income Tax Reporting Currency** (CRA). | |
| 45 | + | |
| 46 | +- Default: convert a foreign amount using the **Bank of Canada rate in effect on the day the amount arises** (the "relevant spot rate" — day income is received, expense paid, transaction occurs). | |
| 47 | +- CRA will generally also accept **another published rate** if it is: widely available, verifiable, from an independent provider, market-recognized, used for financial reporting where applicable, and **used consistently year over year**. | |
| 48 | +- **Average rates** (annual/monthly BoC averages) may be accepted for **income items received throughout the year** (interest, dividends, employment income, recurring revenue) — but **not** for capital transactions (purchase/sale of assets or investments), which require the **transaction-date rate**. | |
| 49 | +- Where the statutory **"relevant spot rate"** definition applies (e.g. functional-currency elections under s. 261), the CRA has **no discretion** to accept anything other than the Bank of Canada rate; for non-CAD ↔ non-CAD conversions, the rate is **derived via CAD** from the two BoC series. | |
| 50 | + | |
| 51 | +### AIR design consequences | |
| 52 | + | |
| 53 | +- Rate-selection policy is **ALSL policy data**, not engine code: `use: transaction_date_rate` as the default, with an opt-in `annual_average` policy scoped to eligible recurring-income event types and jurisdiction `CA`. The compiler enforces that capital-type events always use the transaction-date rate. | |
| 54 | +- Consistency requirement → the chosen rate source/method must be **versioned in the policy** and stable across a fiscal year; changing it is a policy-version event that shows up in the audit trail. | |
| 55 | + | |
| 56 | +Sources (consulted 2026-08-05): | |
| 57 | +- CRA — Income Tax Folio S5-F4-C1, Income Tax Reporting Currency: https://www.canada.ca/en/revenue-agency/services/tax/technical-information/income-tax/income-tax-folios-index/series-5-international-residency/series-5-international-residency-folio-4-foreign-currency/income-tax-folio-s5-f4-c1-income-tax-reporting-currency.html | |
| 58 | +- TaxTips.ca — Reporting foreign amounts on a Canadian return: https://www.taxtips.ca/filing/reporting-foreign-transactions.htm | |
| 59 | +- Bank of Canada — Exchange rates hub: https://www.bankofcanada.ca/rates/exchange/ | |
| 60 | +- Open Government Portal — Daily average FX rates dataset: https://open.canada.ca/data/en/dataset/bb66787c-9509-456d-bd72-5016abdf39c5 | |
| 61 | + | |
| 62 | +--- | |
| 63 | + | |
| 64 | +## 3. How accounting systems record FX | |
| 65 | + | |
| 66 | +Standard multi-currency ledger practice (consistent with IAS 21 / ASC 830): | |
| 67 | + | |
| 68 | +1. **Dual-amount recording:** every line stores the **transaction-currency amount** (as invoiced) *and* the **functional-currency amount** (converted at the transaction-date spot rate). The functional currency is the currency of the entity's primary economic environment — an entity-level configuration. | |
| 69 | +2. **Realized FX gain/loss** — arises on **settlement**: the difference between the functional-currency value at booking and at payment. Example: invoice €10,000 booked at 1.10, collected at 1.12 → realized gain. Posted to a dedicated P&L account (Dr/Cr Realized FX Gain/Loss). | |
| 70 | +3. **Unrealized FX gain/loss** — arises at **period-end revaluation** of open monetary balances (unpaid AR/AP, foreign-currency cash, loans): system compares booking rate to closing rate and posts the difference to Unrealized FX Gain/Loss. These are "paper" movements; systems typically **reverse them at the next period open** or track them cumulatively per open item. | |
| 71 | +4. **Separation for reporting:** realized and unrealized are kept in **separate accounts** for clean reporting, tax treatment, and compliance. | |
| 72 | + | |
| 73 | +### AIR design consequences | |
| 74 | + | |
| 75 | +- AIR `Money` = `{amount: Decimal, currency}`; the **FX pass** (not the LLM, not the event author) produces the functional-currency leg and appends the conversion to the provenance graph (Invoice → FX → Settlement chain, per the SSA-style traceability requirement). | |
| 76 | +- Three deterministic compiler behaviors, matching §3 of `accounting-standards.md`: | |
| 77 | + - **Booking:** spot rate at transaction date (BoC daily average for CA entities). | |
| 78 | + - **Period-end pass:** revalue open monetary items at closing rate → unrealized entries, auto-reversal on period open (fits incremental compilation / contra-entry machinery). | |
| 79 | + - **Settlement:** compute realized gain/loss against the booked rate, per open item. | |
| 80 | +- **Never floats**: rates and amounts are fixed-point decimals; rounding rules (banker's rounding, per-jurisdiction) are documented policy parameters. | |
| 81 | +- Golden tests: FX purchase booked/paid across a rate move (realized), open FX receivable across a period end (unrealized + reversal), CAD-only path (FX pass is a no-op). | |
| 82 | + | |
| 83 | +Sources (consulted 2026-08-05): | |
| 84 | +- Withum — Accounting for Foreign Exchange Transactions: https://www.withum.com/resources/accounting-for-foreign-exchange-transactions/ | |
| 85 | +- Corporate Finance Institute — Foreign Exchange Gain/Loss: https://corporatefinanceinstitute.com/resources/accounting/foreign-exchange-gain-loss/ | |
| 86 | +- Rillet — Multi-Currency Accounting: A Modern Guide: https://www.rillet.com/blog/multi-currency-accounting-guide | |
| 87 | +- Beancount.io — FX gains and losses: a practical multi-currency guide: https://beancount.io/blog/2026/05/03/foreign-exchange-gain-loss-multi-currency-accounting-small-business-guide | |
| 88 | + | |
| 89 | +--- | |
| 90 | + | |
| 91 | +## 4. Decisions / follow-ups | |
| 92 | + | |
| 93 | +- **D1:** Bank of Canada **Valet API** is the canonical rate source for Canadian entities; rates are fetched once, stored immutably, and replayed on recompilation. | |
| 94 | +- **D2:** Rate-selection rules (transaction-date vs average; weekend fallback) live in **ALSL policies**, versioned; the engine only enforces provenance and the capital-transaction restriction. | |
| 95 | +- **D3:** Ledger schema stores transaction-currency and functional-currency amounts on every line, with realized and unrealized FX in separate accounts. | |
| 96 | +- **Follow-up:** research rounding rules for GST/QST on converted amounts before implementing the Tax pass × FX pass interaction; research MT940/camt.053 statement currencies for the reconciliation phase. | |
added
docs/research/ledger-engines.md
+216 −0
@@ -0,0 +1,216 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : ledger-engines.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Double-Entry Ledger Engines and Event-Sourcing Accounting Patterns | |
| 9 | + | |
| 10 | +**Research date:** 2026-08-05 | |
| 11 | +**Purpose:** Survey the state of the art in programmable double-entry ledgers (TigerBeetle, Formance, Modern Treasury, Stripe Ledger, Increase, Square Books) and the foundational design patterns (Martin Fowler's Accounting Patterns, event sourcing) to inform the design of AIR's core (event model, provenance graph, compilation to journal entries, audit log). | |
| 12 | + | |
| 13 | +--- | |
| 14 | + | |
| 15 | +## 1. Martin Fowler's Accounting Patterns | |
| 16 | + | |
| 17 | +Fowler's *Analysis Patterns* (chapter 6) and his more up-to-date follow-up paper *Accounting Patterns* (martinfowler.com/apsupp/accounting.pdf, which supersedes the book chapter) define the canonical object model that nearly every modern ledger engine re-derives: | |
| 18 | + | |
| 19 | +- **Accounting Event** — something that occurs in business operations with financial significance. It is *not* an entry; it is the raw fact that *triggers* entries. This is precisely AIR's `EconomicEvent` concept: the event is the input, the journal entry is compiled output. | |
| 20 | +- **Posting Rule** — the bridge between events and account impacts. A rule inspects an event and determines which accounts to debit/credit and by how much. Fowler treats posting rules as first-class, configurable objects (with variants: Individual Instance Method, Posting Rule Execution, Posting Rules for Many Accounts). This is the direct analogue of AIR's ALSL policies + the AIC posting pass. | |
| 21 | +- **Accounting Transaction** — a *multi-legged* grouping of entries that must balance around a single business occurrence (two-legged is the special case; real transactions with taxes, fees and FX are multi-legged). | |
| 22 | +- **Entry** — an individual amount posted to an account with a direction (debit/credit). | |
| 23 | +- **Account** — aggregates entries by category (assets, liabilities, equity, revenue, expense). | |
| 24 | + | |
| 25 | +**Corrections without mutation.** Fowler defines three adjustment patterns, all of which *add* entries rather than modify them: | |
| 26 | +1. **Reversal Adjustment** — post offsetting (contra) entries to undo the original, then post the correct version. | |
| 27 | +2. **Replacement Adjustment** — reverse and record corrected amounts in one operation. | |
| 28 | +3. **Difference Adjustment** — post only the variance between original and corrected values. | |
| 29 | + | |
| 30 | +Fowler emphasizes maintaining an "event trace": rather than modifying original entries, the system creates additional entries that preserve historical accuracy while achieving corrected balances — audit trail preserved, data integrity risks of in-place modification avoided. This maps 1:1 to AIR's planned incremental recompilation with auto-generated contra entries when a source document changes. | |
| 31 | + | |
| 32 | +Sources: | |
| 33 | +- https://martinfowler.com/apsupp/accounting.pdf (primary, consulted 2026-08-05) | |
| 34 | +- https://martinfowler.com/eaaDev/Account.html (consulted 2026-08-05) | |
| 35 | +- https://martinfowler.com/apsupp/apchap6.pdf (consulted 2026-08-05) | |
| 36 | +- Reference implementations: https://github.com/knoguchi/acc, https://github.com/grahambrooks/accounting-pattern (consulted 2026-08-05) | |
| 37 | + | |
| 38 | +--- | |
| 39 | + | |
| 40 | +## 2. TigerBeetle — a database whose schema *is* double-entry | |
| 41 | + | |
| 42 | +TigerBeetle is a purpose-built OLTP financial database. Its central claim: the debit/credit model is *minimal and complete* — two entity types (**accounts**, **transfers**) plus one invariant (**every debit has an equal and opposite credit**) can model any exchange of value in any domain. | |
| 43 | + | |
| 44 | +Key design properties relevant to AIR: | |
| 45 | + | |
| 46 | +- **Debits/credits as first-class primitives.** There is no generic row model; the fixed schema has ledgers, accounts, and transfers only. Business meaning is encoded in account/transfer codes, not free-form columns. | |
| 47 | +- **Strict append-only immutability.** No `UPDATE`, no `DELETE`, no `ALTER SCHEMA`. Once a transfer is recorded it can never be erased; **reversals are implemented as separate transfers**, yielding a full, auditable log of business events and "effortless reconciliation." | |
| 48 | +- **Integer (fixed-point) amounts.** All fields are fixed-size; amounts are unsigned integers (128-bit), with the currency scale defined per ledger. No floats anywhere in the hot path. | |
| 49 | +- **Invariant enforcement at write time.** Accounts can be flagged so their balance may never go negative (`debits_must_not_exceed_credits` / inverse); violating transfers are *rejected*, not logged-then-fixed. Because transfers apply serially to current state, an accepted transfer is a proof the balance permitted it. | |
| 50 | +- **Linked transfers (atomic chains).** A chain of transfers either all commit or none do — the primitive for multi-legged transactions (sale + tax + fee). | |
| 51 | +- **Two-phase transfers** (pending → post/void) model holds/authorizations, i.e., the lifecycle stages AIR tracks in its provenance graph (Invoice → Payment → Settlement). | |
| 52 | +- Externally validated: Jepsen tested TigerBeetle 0.16.11 for strict serializability. | |
| 53 | + | |
| 54 | +**Lesson for AIR:** push balance invariants into the *storage/compilation layer* as hard rejections with diagnostics, not as after-the-fact checks; model every state change (including corrections) as a new immutable record. | |
| 55 | + | |
| 56 | +Sources: | |
| 57 | +- https://docs.tigerbeetle.com/concepts/debit-credit/ (consulted 2026-08-05) | |
| 58 | +- https://docs.tigerbeetle.com/coding/financial-accounting/ (consulted 2026-08-05) | |
| 59 | +- https://docs.tigerbeetle.com/coding/data-modeling/ (consulted 2026-08-05) | |
| 60 | +- https://tigerbeetle.com/#debit-credit (consulted 2026-08-05) | |
| 61 | +- https://jepsen.io/analyses/tigerbeetle-0.16.11 (consulted 2026-08-05) | |
| 62 | +- https://softwaremill.com/whats-interesting-about-tigerbeetle/ (consulted 2026-08-05) | |
| 63 | + | |
| 64 | +--- | |
| 65 | + | |
| 66 | +## 3. Formance Ledger and Numscript — programmable money movement | |
| 67 | + | |
| 68 | +Formance Ledger (open source, Go) is a programmable double-entry accounting database: immutable, tamper-evident transaction log, built-in concurrency control, multi-asset support, and a DSL — **Numscript** — for modeling money movements. | |
| 69 | + | |
| 70 | +- **Numscript = declarative DSL for transactions.** A financial transaction is defined as a series of discrete value movements between abstract accounts (`send [USD/2 100] (source = @world destination = @users:1234)`). It replaces error-prone imperative code with readable, declarative, *templatizable* scripts. This is the closest existing analogue to ALSL: a small, deterministic language whose only job is to describe postings. | |
| 71 | +- **Determinism and conservation of money.** Numscript programs are deterministic, always terminate with predictable output, and the language semantics guarantee no accidental creation or destruction of money and no currency-rounding leaks (allocations by percentage handle remainders explicitly). Execution is atomic: all postings commit or none. | |
| 72 | +- **Hash-chained log.** The ledger chains transaction logs together (each transaction produces a hash over its data plus the previous hash), blockchain-style, giving tamper evidence — the exact mechanism AIR plans for its audit log. | |
| 73 | +- Formance also published a useful formal model of double-entry for engineers (accounts, postings, balance as fold over postings). | |
| 74 | + | |
| 75 | +**Lesson for AIR:** ALSL should aspire to Numscript-grade properties — declarative, deterministic, total (always terminates), money-conserving by construction, with explicit remainder handling in splits. | |
| 76 | + | |
| 77 | +Sources: | |
| 78 | +- https://github.com/formancehq/ledger (consulted 2026-08-05) | |
| 79 | +- https://docs.formance.com/modules/ledger/introduction (consulted 2026-08-05) | |
| 80 | +- https://www.formance.com/blog/engineering/numscript (consulted 2026-08-05) | |
| 81 | +- https://www.formance.com/blog/engineering/defining-double-entry (consulted 2026-08-05) | |
| 82 | +- https://docs-archive.formance.com/numscript/ (consulted 2026-08-05) | |
| 83 | + | |
| 84 | +--- | |
| 85 | + | |
| 86 | +## 4. Ledgers at scale: Modern Treasury, Stripe, Increase, Square | |
| 87 | + | |
| 88 | +### 4.1 Modern Treasury ("How to Scale a Ledger", parts I–VI) | |
| 89 | + | |
| 90 | +Modern Treasury sells Ledgers-as-an-API and documented its design in a six-part series. Core guarantees they identify as *the* product: | |
| 91 | + | |
| 92 | +- **Immutability** — all changes recorded so any past state can be retrieved; history must be replayable. | |
| 93 | +- **Double-entry enforcement** — every movement names both source and destination; the API models everything with three core objects: **Account, Entry, Transaction** (Fowler's model, again). | |
| 94 | +- **Concurrency controls** — prevent double-spending; entries can operate in *authorizing* mode (balance checks enforced synchronously, like TigerBeetle) or *recording* mode (log-first), chosen per Entry. | |
| 95 | +- **Idempotency** — writes carry idempotency keys so client retries cannot double-post; combined with immutability this keeps data "pristine" under network failure. | |
| 96 | +- **Efficient aggregation** — balances are derived, so they precompute/aggregate (Account Categories graph for roll-ups) rather than sum entries per read; scaling double-entry is hard precisely because history is immutable and must remain replayable at billions of transactions. | |
| 97 | + | |
| 98 | +### 4.2 Stripe Ledger | |
| 99 | + | |
| 100 | +Stripe's internal Ledger tracks and validates money movement across its Global Payments and Treasury Network: | |
| 101 | + | |
| 102 | +- An **immutable log of events**; producer systems are modeled as **state machines** whose behavior is expressed as *logical fund flows* — movements of balances between accounts. | |
| 103 | +- **Double-entry as mathematical proof of correctness**: all platform activity is mapped to one common data structure, and traditional accounting principles validate the flows. | |
| 104 | +- Scale/quality numbers: ~**5 billion events/day**; 99.99% of dollar volume fully ingested and verified within four days; data-quality platform achieves **99.9999%+ "explainability" of money movement** — i.e., every cent is accounted for or flagged. | |
| 105 | +- Key insight: the ledger is a *verification and reconciliation layer over heterogeneous producer systems*, not the systems themselves — very close to AIR's role as a common IR above heterogeneous ERPs. | |
| 106 | + | |
| 107 | +### 4.3 Increase | |
| 108 | + | |
| 109 | +Increase (banking-API bank) exposes a **Bookkeeping API**: clients create bookkeeping accounts (e.g., one per customer with `compliance_category: customer_balance`, plus `commingled_cash` accounts mapped to real bank accounts) and post **bookkeeping entry sets** whose entries **must sum to match the transaction amount** — balance enforced at the API boundary. It demonstrates the "compliance-labeled chart of accounts + balanced entry set" pattern for FBO/commingled-funds tracking. | |
| 110 | + | |
| 111 | +### 4.4 Square Books | |
| 112 | + | |
| 113 | +Square's internal service "Books" is an **immutable double-entry accounting database service**; their stated rationale: double-entry forces you to record not just *what* financial state change occurred but *why*, and all transactions must balance to zero. | |
| 114 | + | |
| 115 | +Sources: | |
| 116 | +- https://www.moderntreasury.com/journal/how-to-scale-a-ledger-part-i (consulted 2026-08-05) | |
| 117 | +- https://www.moderntreasury.com/journal/how-to-scale-a-ledger-part-ii (consulted 2026-08-05) | |
| 118 | +- https://www.moderntreasury.com/journal/how-to-scale-a-ledger-part-v (immutability & double-entry; consulted 2026-08-05) | |
| 119 | +- https://www.moderntreasury.com/journal/how-to-scale-a-ledger-part-vi (concurrency & performance; consulted 2026-08-05) | |
| 120 | +- https://www.moderntreasury.com/journal/behind-the-scenes-how-we-built-ledgers-for-high-throughput (consulted 2026-08-05) | |
| 121 | +- https://www.moderntreasury.com/journal/why-we-built-ledgers (consulted 2026-08-05) | |
| 122 | +- https://stripe.dev/blog/ledger-stripe-system-for-tracking-and-validating-money-movement (consulted 2026-08-05) | |
| 123 | +- https://www.fintechwrapup.com/p/deep-dive-ledger-stripes-system-for (analysis; consulted 2026-08-05) | |
| 124 | +- https://increase.com/documentation/bookkeeping (consulted 2026-08-05) | |
| 125 | +- https://developer.squareup.com/blog/books-an-immutable-double-entry-accounting-database-service/ (consulted 2026-08-05) | |
| 126 | + | |
| 127 | +--- | |
| 128 | + | |
| 129 | +## 5. Event sourcing for financial systems | |
| 130 | + | |
| 131 | +Event sourcing is the natural persistence model for accounting because accounting *invented* it: a bank ledger records every transaction and the balance is a derived sum. | |
| 132 | + | |
| 133 | +- **Append-only event store.** Events are only ever appended; never updated or deleted. Every state change is an immutable fact. The store is the single source of truth. | |
| 134 | +- **Projections / read models.** Current state (account balances, trial balance, aging reports) is a *materialized view built by replaying events*. Projections are updated asynchronously and must be kept consistent with (and rebuildable from) the event store. For AIR: the compiled journal, the general ledger, and every financial report are projections of the `EconomicEvent` stream + policy versions. | |
| 135 | +- **Corrections via new events, never mutation.** A wrong event is not edited; a compensating/reversal event is appended (matching Fowler's Reversal/Difference Adjustments and TigerBeetle's reversal transfers). Replaying the full stream still yields the corrected state, and the mistake itself remains visible for audit. | |
| 136 | +- **Why finance loves it:** regulatory audit trails, fraud detection, time-travel ("what was the balance on March 31 as known on April 5?" — bi-temporal queries), recovery, and replay-based testing. | |
| 137 | +- **Pitfalls to plan for:** schema/versioning of events (AIR's `air_version` + explicit migrations), projection lag (eventual consistency of read models), and event-store growth (snapshots). | |
| 138 | + | |
| 139 | +### 5.1 Hash-chained, tamper-evident audit logs | |
| 140 | + | |
| 141 | +Standard construction (used by Formance, Trillian/transparency.dev, and accepted by financial examiners): | |
| 142 | + | |
| 143 | +1. **Append-only storage** with a monotonic sequence number per row. | |
| 144 | +2. **Hash chaining**: each entry stores `prev_hash` and `own_hash = SHA-256(prev_hash || seq || canonical_encoding(fields))`. A verifier replays the chain from genesis and reports the first break (gap, predecessor mismatch, or row-hash mismatch). | |
| 145 | +3. **Merkle trees** on top for efficient inclusion proofs (chain verification alone is O(n)); periodically **anchor the root** somewhere hard to alter retrospectively (HSM-signed storage, WORM storage, external transparency log). | |
| 146 | +4. **Write-path discipline**: atomic append + **idempotent retries** (outbox pattern) so network retries never create duplicate or missing entries; run **continuous verification** against the live chain, not only at export time. | |
| 147 | + | |
| 148 | +Note the distinction: the *audit log* records who did what and when (SDK syscall journal); *event sourcing* makes domain events the primary source of truth. AIR needs both, and they are separate artifacts. | |
| 149 | + | |
| 150 | +Sources: | |
| 151 | +- https://www.techinterview.org/post/3233465463/system-design-event-sourcing/ (consulted 2026-08-05) | |
| 152 | +- https://medium.com/@alxkm/event-sourcing-explained-benefits-challenges-and-use-cases-d889dc96fc18 (consulted 2026-08-05) | |
| 153 | +- https://dev.to/tyson_cung/event-sourcing-from-scratch-why-your-database-has-been-lying-to-you-2eec (consulted 2026-08-05) | |
| 154 | +- https://dzone.com/articles/event-sourcing-guide-when-to-use-avoid-pitfalls (consulted 2026-08-05) | |
| 155 | +- https://www.designgurus.io/answers/detail/how-do-you-design-tamperevident-audit-logs-merkle-trees-hashing (consulted 2026-08-05) | |
| 156 | +- https://www.designgurus.io/answers/detail/how-do-you-enforce-immutability-and-appendonly-audit-trails (consulted 2026-08-05) | |
| 157 | +- https://finqub.io/learn/tamper-evident-audit-trail/ (what examiners accept: hash chains, signed exports, WORM; consulted 2026-08-05) | |
| 158 | +- https://transparency.dev/ (Trillian open-source append-only log; consulted 2026-08-05) | |
| 159 | +- https://www.architecture-weekly.com/p/building-your-own-ledger-database (consulted 2026-08-05) | |
| 160 | + | |
| 161 | +--- | |
| 162 | + | |
| 163 | +## 6. Amounts: never floats; fixed-point/decimal and banker's rounding | |
| 164 | + | |
| 165 | +### 6.1 Why floats are forbidden | |
| 166 | + | |
| 167 | +- IEEE-754 binary floating point cannot represent most decimal fractions exactly: `0.1` is stored as an approximation. Tiny representation errors **compound** over repeated operations; a worked example: $100 deposited daily at 6% interest compounded daily drifts ~**$1.40 from the exact answer over one year** using doubles. | |
| 168 | +- In financial systems, small numeric errors become real discrepancies: balances that do not reconcile, reports that do not match fills, results that **depend on operation order** (float addition is not associative — fatal for AIR's determinism guarantee: same AIR + same policies must always produce identical entries). | |
| 169 | +- Even the contrarian take (evanjones.ca, "You can use floating-point numbers for money") concedes it only works with extreme care and explicit rounding at every step — exactly the discipline that decimal types give you for free. Industry consensus and every ledger surveyed here (TigerBeetle, Formance, Modern Treasury, Stripe) use integers or decimals. | |
| 170 | + | |
| 171 | +### 6.2 The correct representations | |
| 172 | + | |
| 173 | +- **Fixed-point integers with explicit scale**: store minor units (cents) or scaled integers (`amount: i128, scale: u8`) — TigerBeetle's approach (unsigned 128-bit integers, scale defined per ledger/asset). | |
| 174 | +- **Decimal types**: `rust_decimal` (Rust), `decimal.Decimal` (Python), `decimal` (C#) — base-10 storage, exact cent representation, configurable rounding modes. | |
| 175 | +- For AIR: JSON Schema should carry amounts as **strings or {amount, currency} objects with decimal strings**, never JSON numbers (JSON parsers commonly decode numbers as float64). | |
| 176 | + | |
| 177 | +### 6.3 Banker's rounding (round-half-to-even) | |
| 178 | + | |
| 179 | +- Ties (exactly .5) round to the nearest **even** digit: 2.5 → 2, 3.5 → 4. Over many operations this removes the systematic upward bias of round-half-up, minimizing cumulative error. It is IEEE-754's default mode and `decimal` libraries support it (`ROUND_HALF_EVEN`). | |
| 180 | +- Caveat: a rounding mode only behaves as intended if the underlying value is stored exactly in the first place — another reason floats are out. | |
| 181 | +- **Jurisdictional warning for AIR**: rounding rules for taxes are *legal*, not stylistic — e.g., tax authorities may mandate round-half-up per line or per invoice total (CRA/Revenu Québec rules for GST/QST rounding must be verified separately; see the tax research doc before coding). Therefore the rounding mode must be a **per-jurisdiction ALSL policy parameter**, never hard-coded in the engine, and every rounding step should be a node in the provenance graph (Numscript's explicit remainder allocation is the model: when splitting an amount, the remainder cent is assigned deterministically and visibly). | |
| 182 | + | |
| 183 | +Sources: | |
| 184 | +- https://rustyeddy.com/software/numeric-types-financial-software/ (fixed-point in financial software; consulted 2026-08-05) | |
| 185 | +- https://medium.com/@sohail_saifii/the-floating-point-standard-thats-silently-breaking-financial-software-7f7e93430dbb (consulted 2026-08-05) | |
| 186 | +- https://evertpot.com/currencies-floats/ (consulted 2026-08-05) | |
| 187 | +- https://www.evanjones.ca/floating-point-money.html (contrarian view, noted and rejected for AIR; consulted 2026-08-05) | |
| 188 | +- https://legalclarity.org/what-is-bankers-rounding-and-how-does-it-work/ (consulted 2026-08-05) | |
| 189 | + | |
| 190 | +--- | |
| 191 | + | |
| 192 | +## 7. Design lessons for AIR | |
| 193 | + | |
| 194 | +1. **Append-only event store as the single source of truth.** `EconomicEvent`s are immutable facts, only ever appended (PostgreSQL event store per CLAUDE.md §4). The general ledger, balances, and reports are *projections* rebuilt by replaying events through the compiler — like Stripe's Ledger sitting above producer systems. No `UPDATE`/`DELETE` on events or on compiled entries, ever (TigerBeetle discipline). | |
| 195 | + | |
| 196 | +2. **Corrections as compensating entries, never mutation.** When a source document changes or an error is found, append a new event; the AIC recompiles the delta and emits **reversal (contra) entries + corrected entries** automatically (Fowler's Reversal/Difference Adjustments; TigerBeetle reversal transfers; Modern Treasury immutability). The mistake stays visible in history — that is a feature. | |
| 197 | + | |
| 198 | +3. **Balance invariant enforced at every stage, as a hard rejection.** `Assets = Liabilities + Equity` (equivalently: every transaction's legs sum to zero) is checked after *every* compiler pass and at posting time; violations abort compilation with clang-quality diagnostics. Follow TigerBeetle: reject-at-write, don't detect-after-the-fact. Multi-legged transactions must be atomic (linked-transfer semantics: all legs or none). Add per-account invariants (e.g., cash accounts may not go negative) as ALSL policies. | |
| 199 | + | |
| 200 | +4. **Provenance / traceability as a first-class graph.** Every compiled entry links back to: source event → source document + OCR/LLM scores → policy versions applied → each intermediate transformation (tax computation, FX conversion, rounding step). Stripe's "99.9999% explainability" is the benchmark; Fowler's "event trace" and Square Books' "record *why*, not just *what*" are the pattern. Rounding and remainder allocation are explicit provenance nodes (Numscript model). | |
| 201 | + | |
| 202 | +5. **Idempotency keys on every mutating syscall.** `CreateEconomicEvent`, `Post`, `Reverse`, etc. accept a client idempotency key; retries are safe and never double-post (Modern Treasury; Increase entry sets). Backend `post()` must be idempotent per receipt as well — required for Phase 3 (QuickBooks OAuth/posting). | |
| 203 | + | |
| 204 | +6. **Hash-chained audit log, separate from the event store.** Every SDK syscall appends a row with `seq`, `prev_hash`, `own_hash = SHA-256(prev_hash || seq || canonical_json)`; continuous verification recomputes the chain; periodically anchor a Merkle root to WORM/external storage (Formance log-chaining; Trillian; examiner-accepted practice per FinQub). | |
| 205 | + | |
| 206 | +7. **Amounts are decimals/fixed-point end-to-end.** Decimal strings in JSON schemas, `rust_decimal`/`Decimal` in code, integer minor units in storage. Rounding mode (banker's vs half-up) is a per-jurisdiction ALSL parameter, never engine-coded. Determinism (§5 of CLAUDE.md) is unachievable with floats. | |
| 207 | + | |
| 208 | +8. **Keep the core model minimal: Account, Entry (debit/credit), Transaction, Event, Posting Rule.** Every system surveyed — from Fowler (1996) to TigerBeetle (2026) — converges on these five objects. AIR's innovation is not the ledger model; it is the *compiler* (deterministic posting rules as versioned ALSL policies) and the *IR* (events, not entries, as the interchange format). Do not reinvent the ledger core; adopt it exactly and spend the novelty budget on passes, diagnostics, and backends. | |
| 209 | + | |
| 210 | +9. **Two-phase / lifecycle-aware events.** Pending → posted/voided states (TigerBeetle two-phase transfers; Modern Treasury authorizing vs recording modes) map to AIR's approval queue: events below the LLM-confidence threshold sit in `pending` and reserve nothing until approved. | |
| 211 | + | |
| 212 | +10. **Balances are derived but must be cheap.** Plan projection/aggregation tables (Modern Treasury Account Categories) so trial balances and per-account balances do not require replaying the full event stream on every read; snapshots + incremental projection updates. | |
| 213 | + | |
| 214 | +--- | |
| 215 | + | |
| 216 | +*End of research document — consulted 2026-08-05.* | |
added
docs/research/llm-structured-extraction.md
+116 −0
@@ -0,0 +1,116 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : llm-structured-extraction.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# Research: LLM Structured Extraction (2025–2026 Best Practices) | |
| 9 | + | |
| 10 | +**Date of research:** 2026-08-05 | |
| 11 | +**Method:** 4 targeted web searches (Claude structured outputs, OpenAI structured outputs, confidence scoring + human-in-the-loop, OCR+LLM invoice pipelines). | |
| 12 | +**Purpose:** Design AIR's ingestion layer (`ingestion/`). **Core AIR principle restated:** the LLM only ever produces **AIR `EconomicEvent` documents** — never journal entries. Every LLM output is validated against the AIR JSON Schema, and low-confidence extractions are routed to a human approval queue before compilation. | |
| 13 | + | |
| 14 | +--- | |
| 15 | + | |
| 16 | +## 1. Schema-constrained generation — Claude API | |
| 17 | + | |
| 18 | +The Claude Developer Platform supports **structured outputs** (public beta since late 2025, beta header `anthropic-beta: structured-outputs-2025-11-13`; initially Claude Sonnet 4.5 and Opus 4.1). Rather than merely prompting for JSON, the platform **compiles the JSON Schema into a grammar and constrains token generation during inference**, so responses are guaranteed to conform. | |
| 19 | + | |
| 20 | +Two modes: | |
| 21 | +- **JSON outputs mode** — supply the schema via the `output_format` parameter; the response text is guaranteed schema-valid JSON. This is the fit for extraction tasks like "invoice PDF text → AIR Sale event". | |
| 22 | +- **Strict tool use** — add `strict: true` to tool definitions; tool-call parameters exactly match the tool's input schema. This is the fit for the AIR **SDK-agent** path, where an agent calls `CreateEconomicEvent(...)` as a tool and the arguments are a schema-guaranteed AIR event. | |
| 23 | + | |
| 24 | +Established pre-structured-outputs pattern (still relevant as fallback for models/versions without the beta): define a single tool whose input schema is the extraction target and force the model to call it (tool use for extraction). | |
| 25 | + | |
| 26 | +Sources (consulted 2026-08-05): | |
| 27 | +- Anthropic — Structured outputs on the Claude Developer Platform (blog): https://claude.com/blog/structured-outputs-on-the-claude-developer-platform | |
| 28 | +- Claude Platform docs — Structured outputs: https://platform.claude.com/docs/en/build-with-claude/structured-outputs | |
| 29 | +- Claude docs mirror — Structured outputs: https://docs.claude.com/en/docs/build-with-claude/structured-outputs | |
| 30 | + | |
| 31 | +## 2. Schema-constrained generation — OpenAI | |
| 32 | + | |
| 33 | +OpenAI **Structured Outputs**: supply a JSON Schema via `response_format: {type: "json_schema", strict: true}` (or strict function calling). With `strict: true` the model **cannot** emit output violating the schema — required fields present, types correct, enum values valid. OpenAI reports ~100% schema compliance in evals vs ~86% for plain function calling and lower for raw JSON mode (JSON mode only guarantees *valid JSON*, not schema conformance). Supported on gpt-4o-2024-08-06 and later snapshots, gpt-4o-mini, o1 family and successors. | |
| 34 | + | |
| 35 | +**Takeaway for AIR:** both major providers now offer grammar-level schema enforcement. The AIR JSON Schema (source of truth in `schemas/`) can be passed directly as the constrained-decoding schema, making the ingestion layer **provider-agnostic**: one canonical schema → Claude `output_format` / strict tools, OpenAI `json_schema`, plus local re-validation. | |
| 36 | + | |
| 37 | +Sources (consulted 2026-08-05): | |
| 38 | +- OpenAI — Introducing Structured Outputs in the API: https://openai.com/index/introducing-structured-outputs-in-the-api/ | |
| 39 | +- OpenAI docs — Structured model outputs: https://developers.openai.com/api/docs/guides/structured-outputs | |
| 40 | + | |
| 41 | +## 3. Validation: schema conformance is necessary, not sufficient | |
| 42 | + | |
| 43 | +Constrained decoding guarantees **shape**, not **truth**. 2025–2026 practice is to build validation in from the start (e.g. Pydantic models generated from the schema) and layer semantic checks after parse. For AIR, post-parse validation in the ingestion pipeline (before anything reaches the compiler): | |
| 44 | + | |
| 45 | +1. **Schema validation** (defense in depth — never trust the provider's guarantee alone; also covers `air_version` compatibility). | |
| 46 | +2. **Semantic validators**: decimal amounts parse exactly (no floats), `qty × unit_price` consistent with line totals, gross vs net consistent, dates plausible, currency codes ISO 4217, jurisdiction codes known, referenced entities (`customer:...`, `company:...`) resolvable. | |
| 47 | +3. **Cross-document checks**: duplicate detection (hash + similarity vs previously ingested events) before the event enters the queue. | |
| 48 | +4. Only *then* the event enters Validate → (approval) → Compile. **The LLM's output is always an AIR event — a description of an economic event — never a journal entry; account selection, tax computation, and posting are the deterministic compiler's job.** | |
| 49 | + | |
| 50 | +Sources (consulted 2026-08-05): | |
| 51 | +- Vellum — Document data extraction in 2026: LLMs vs OCRs: https://www.vellum.ai/blog/document-data-extraction-llms-vs-ocrs | |
| 52 | +- Cleanlab — Real-time error detection for LLM structured outputs (benchmark): https://cleanlab.ai/blog/tlm-structured-outputs-benchmark/ | |
| 53 | + | |
| 54 | +## 4. Confidence scoring and human-in-the-loop approval queues | |
| 55 | + | |
| 56 | +Current practice in document-AI products (Box Extract, LandingAI, IDP platforms): | |
| 57 | + | |
| 58 | +- **Field-level confidence scores**, not just document-level. Common techniques: self-consistency (sample multiple extractions and measure agreement — variance ⇒ low confidence), token log-probabilities where exposed, calibrated model-as-judge scoring (e.g. Cleanlab TLM), and OCR engine character/word confidences propagated to dependent fields. | |
| 59 | +- **Threshold routing**: fields/documents **above** threshold flow straight through; **below** threshold they are routed to a **human review queue**. Reviewers verify or correct, and the queue size is proportional to actual uncertainty — that is what makes HITL tractable (review only the uncertain extractions, not everything). | |
| 60 | +- **Pre-fill even when unsure**: show the extracted value to the reviewer — correcting a pre-filled value is faster than typing from scratch. | |
| 61 | +- **Feedback loop**: corrections are logged and become evaluation/tuning data; thresholds are tuned per field criticality. | |
| 62 | + | |
| 63 | +### AIR mapping | |
| 64 | + | |
| 65 | +AIR's schema already reserves `meta.llm: {model, confidence, reasoning_hash}` and `timestamps.approved` / `approver`. Concretely: | |
| 66 | + | |
| 67 | +- Store **per-field confidence** (e.g. `meta.llm.field_confidence: {total: 0.99, tax.codes: 0.71, ...}`) in addition to the overall score, plus the OCR score (`meta.source.ocr_score`). | |
| 68 | +- **Approval policy is ALSL data**, not code: thresholds per event type, amount band, and field criticality (e.g. any event > X CAD, or `tax.jurisdiction` confidence < 0.9 ⇒ human approval). Below-threshold events sit in the approval queue with `timestamps.approved: null`; the compiler **refuses to compile unapproved events** whose policy requires approval. | |
| 69 | +- Every approval/correction is an **audit-log entry** (hash-chained), and a human correction produces a new event version with provenance to the original extraction — the reviewer's identity lands in `approver`. | |
| 70 | + | |
| 71 | +Sources (consulted 2026-08-05): | |
| 72 | +- Box — Confidence scores for Box Extract API: https://blog.box.com/confidence-scores-box-extract-api-know-when-rely-your-extractions | |
| 73 | +- LandingAI — Building human-in-the-loop review workflows for document AI: https://landing.ai/llms/building-human-in-the-loop-review-workflows-for-document-ai | |
| 74 | +- DEV — Human in the loop: using confidence scores for reliable document extraction: https://dev.to/iterationlayer/human-in-the-loop-using-confidence-scores-to-build-reliable-document-extraction-3pnb | |
| 75 | +- Databricks — What is Human-in-the-Loop (HITL)?: https://www.databricks.com/blog/human-in-the-loop | |
| 76 | +- Subhajit Bhar — Confidence scoring in document extraction: https://subhajitbhar.com/blog/idp/glossary/confidence-scoring-document-extraction/ | |
| 77 | + | |
| 78 | +## 5. Invoice / document extraction pipelines (OCR + LLM) | |
| 79 | + | |
| 80 | +2025–2026 consensus is a **hybrid architecture**: | |
| 81 | + | |
| 82 | +- **OCR / layout stage first** (or native PDF text extraction when the PDF has a text layer): specialized OCR + layout-analysis models (transformer-based document models) handle text recovery, tables, and segmentation, and yield **character/word confidence scores** that pure LLM vision calls don't expose reliably. | |
| 83 | +- **LLM stage second**: semantic interpretation of the recovered text/layout into the target schema — the LLM is best at understanding ("3 chairs paid by Visa"), OCR/layout models at faithful transcription. For invoices specifically, hybrid splits are common (deterministic extraction for header fields, LLM for messy line items). | |
| 84 | +- **Validation stage third**: schema + business-rule validation (see §3), then confidence-based routing (see §4). | |
| 85 | +- Production systems combining specialized table extraction, layout analysis, and LLM semantic understanding get the best accuracy, at higher engineering cost — appropriate for AIR since extraction errors become financial records. | |
| 86 | + | |
| 87 | +### AIR ingestion pipeline (Phase 4 target) | |
| 88 | + | |
| 89 | +``` | |
| 90 | +PDF/image/email | |
| 91 | + → OCR + layout (per-field ocr confidence) [meta.source.ocr_score] | |
| 92 | + → LLM extraction, schema-constrained to AIR [meta.llm.{model, confidence, field_confidence, reasoning_hash}] | |
| 93 | + → schema + semantic validation (reject/repair) | |
| 94 | + → duplicate detection | |
| 95 | + → confidence routing: ≥ threshold → auto-approve per policy | |
| 96 | + < threshold → human approval queue | |
| 97 | + → approved AIR event → AIC compiler (deterministic) → journal entries | |
| 98 | +``` | |
| 99 | + | |
| 100 | +The LLM's role **ends** at the AIR event. No prompt, agent, or extraction step ever emits debits/credits; determinism, tax rules, and the double-entry invariant live entirely in the compiler. | |
| 101 | + | |
| 102 | +Sources (consulted 2026-08-05): | |
| 103 | +- arXiv — Automated invoice data extraction using LLM and OCR (2511.05547): https://arxiv.org/abs/2511.05547 | |
| 104 | +- Unstract — A 2026 guide to AI invoice data extraction: https://unstract.com/blog/ai-invoice-processing-and-data-extraction/ | |
| 105 | +- Unstract — Invoice OCR in 2026: from document to accounting systems: https://unstract.com/blog/best-ocr-for-invoice-processing-invoice-ocr/ | |
| 106 | +- AIMultiple — Invoice OCR benchmark: LLMs vs OCRs: https://aimultiple.com/invoice-ocr | |
| 107 | +- Virtido — Document intelligence with LLMs (2026): https://virtido.com/blog/document-intelligence-llm-extraction-guide | |
| 108 | + | |
| 109 | +--- | |
| 110 | + | |
| 111 | +## 6. Decisions / follow-ups | |
| 112 | + | |
| 113 | +- **D1:** Ingestion is provider-agnostic around one canonical AIR JSON Schema; use grammar-constrained structured outputs (Claude `output_format`/strict tools; OpenAI `json_schema` strict) with mandatory local re-validation. | |
| 114 | +- **D2:** Per-field confidence + OCR score stored in `meta`; approval thresholds are versioned ALSL policies; compiler refuses unapproved events that policy flags. | |
| 115 | +- **D3:** Hybrid OCR→LLM→validate→route pipeline; corrections feed an eval set for regression-testing extraction quality. | |
| 116 | +- **Follow-up:** benchmark constrained vs unconstrained extraction accuracy on anonymized invoice fixtures (`tests/fixtures/`) before Phase 4; verify the current status of Anthropic's structured-outputs beta (header/model list may have changed since late 2025) at implementation time. | |
added
docs/research/llvm-architecture.md
+204 −0
@@ -0,0 +1,204 @@ | ||
| 1 | +<!-- | |
| 2 | +Project : AIR — Accounting Intermediate Representation | |
| 3 | +Author : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +File : llvm-architecture.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# LLVM Compiler Architecture — Research Notes | |
| 9 | + | |
| 10 | +**Purpose.** AIR ("Accounting Intermediate Representation") deliberately borrows its architecture from LLVM. This document records what LLVM actually does — IR structure, SSA form, the pass manager, target backends, TableGen, and diagnostics philosophy — from primary sources, then maps each concept to AIR. | |
| 11 | + | |
| 12 | +**Sources consulted on 2026-08-05** (see full list at the end): | |
| 13 | + | |
| 14 | +1. LLVM Language Reference Manual — https://llvm.org/docs/LangRef.html | |
| 15 | +2. LLVM New Pass Manager — https://llvm.org/docs/NewPassManager.html | |
| 16 | +3. LLVM Target-Independent Code Generator — https://llvm.org/docs/CodeGenerator.html | |
| 17 | +4. Clang: Expressive Diagnostics — https://clang.llvm.org/diagnostics.html | |
| 18 | +5. Chris Lattner, "LLVM", *The Architecture of Open Source Applications* — https://aosabook.org/en/v1/llvm.html | |
| 19 | + | |
| 20 | +--- | |
| 21 | + | |
| 22 | +## 1. The big picture: a three-phase design with the IR as the only interface | |
| 23 | + | |
| 24 | +LLVM implements the classic three-phase compiler model — **frontend → optimizer → backend** — but with one decisive twist: the intermediate representation (LLVM IR) is *fully specified* and is *the only interface* between phases (Lattner, AOSA). Consequences: | |
| 25 | + | |
| 26 | +- **M + N instead of M × N.** Supporting M source languages and N target architectures requires M frontends plus N backends, not M×N compilers. Any frontend that emits valid IR gains every backend for free. | |
| 27 | +- **Three isomorphic forms.** The IR exists as (1) in-memory data structures, (2) a dense on-disk binary ("bitcode"), and (3) a human-readable textual assembly. All three are semantically equivalent, so any stage can be serialized, inspected, diffed, and replayed. | |
| 28 | +- **Everything is a library.** Optimizer passes, code generators, and analyses ship as composable libraries; clients pick which passes run and in what order. This is what allowed LLVM to be reused far beyond `clang` (JITs, GPU stacks, etc.). | |
| 29 | + | |
| 30 | +> Contrast with GCC, where backends historically walked frontend ASTs — a tangling LLVM explicitly avoids. The IR being self-contained is the whole point. | |
| 31 | + | |
| 32 | +## 2. LLVM IR structure and SSA form | |
| 33 | + | |
| 34 | +Source: LangRef (consulted 2026-08-05). | |
| 35 | + | |
| 36 | +### 2.1 Containment hierarchy | |
| 37 | + | |
| 38 | +``` | |
| 39 | +Module — one "translation unit"; linkable with other modules | |
| 40 | + ├── global variables — always accessed through pointers | |
| 41 | + ├── functions (define) — a control-flow graph of basic blocks | |
| 42 | + │ └── basic blocks — straight-line instruction lists | |
| 43 | + │ └── instructions — typed operations; each block ends with exactly | |
| 44 | + │ one *terminator* (br, ret, switch, ...) | |
| 45 | + └── named metadata | |
| 46 | +``` | |
| 47 | + | |
| 48 | +- **Identifiers**: global symbols are prefixed `@` (functions, globals), local values `%` (registers, types). Unnamed temporaries are auto-numbered (`%0`, `%1`, ...) — compilers can mint temporaries without symbol-table clashes. | |
| 49 | +- **Entry block**: the first basic block of a function has no predecessors and may not contain PHI nodes. | |
| 50 | +- **Types**: every value has a static type; every instruction result is typed. There are no untyped "bags of bytes" at the IR level. | |
| 51 | +- **Metadata** (`!name !N`) attaches to instructions, functions, and globals to carry debug info, provenance, and optimization hints *without changing execution semantics*. This is the sanctioned channel for "extra facts about a value." | |
| 52 | + | |
| 53 | +### 2.2 SSA — Static Single Assignment | |
| 54 | + | |
| 55 | +LLVM IR is an SSA-based representation: **each value is defined exactly once**, and every use must be *dominated* by its definition (i.e., the definition provably executes before any use on every path). The verifier rejects IR where "the definition of `%x` does not dominate all of its uses." | |
| 56 | + | |
| 57 | +Why it matters: | |
| 58 | + | |
| 59 | +- Def-use chains are explicit and immutable → dataflow analyses become cheap and local. | |
| 60 | +- A value's provenance is unambiguous: you can always walk from any use back to the single defining instruction. | |
| 61 | +- Merging control-flow paths is explicit (PHI nodes) rather than implicit mutation. | |
| 62 | + | |
| 63 | +### 2.3 The verifier | |
| 64 | + | |
| 65 | +A dedicated **verification pass** checks well-formedness (typing, dominance, terminator rules) after parsing, and the optimizer re-validates before emitting bitcode. Malformed IR is a hard error, not a warning. The invariant is structural: *no pass is allowed to leave the IR in an invalid state, even transiently across pass boundaries.* | |
| 66 | + | |
| 67 | +## 3. The (new) Pass Manager | |
| 68 | + | |
| 69 | +Source: NewPassManager doc (consulted 2026-08-05). | |
| 70 | + | |
| 71 | +### 3.1 Pass kinds by IR unit | |
| 72 | + | |
| 73 | +Passes are stratified by the IR unit they operate on: **Module → CGSCC (call-graph SCC) → Function → Loop**. A pass declares its level; **adaptors** (`createFunctionToLoopPassAdaptor`, etc.) let an outer-level pipeline embed inner-level passes. Grouping passes at the same level improves cache locality (run all function passes on one function before moving to the next). | |
| 74 | + | |
| 75 | +### 3.2 Analyses vs transformations | |
| 76 | + | |
| 77 | +The design splits work into two disjoint kinds: | |
| 78 | + | |
| 79 | +- **Analysis passes** compute facts about the IR *without modifying it* (dominator trees, alias analysis, ...). Results are **cached** by an `AnalysisManager` and reused as long as they are valid. | |
| 80 | +- **Transformation passes** modify the IR. After running, each returns a **`PreservedAnalyses`** set declaring which cached analyses are still valid: `all()` (nothing changed), `none()` (invalidate everything), or a selective set. The manager invalidates accordingly; analyses may also implement custom `invalidate()` logic. | |
| 81 | + | |
| 82 | +A deliberate constraint: an analysis manager only serves results *at the same IR level* as the requesting pass (a function pass cannot freely poke module-level analyses) — this prevents quadratic compile times and keeps the door open for concurrency. | |
| 83 | + | |
| 84 | +### 3.3 Pipeline construction and ordering | |
| 85 | + | |
| 86 | +- A `PassBuilder` assembles standard pipelines (e.g., `buildPerModuleDefaultPipeline()`); ordering is explicit and centrally defined, not emergent. | |
| 87 | +- Extension points (`registerPipelineStartEPCallback()`, etc.) let clients inject passes at defined seams without forking the pipeline. | |
| 88 | +- Pipelines are scriptable as text: `opt -passes='function(pass1,pass2),module-pass' file.ll -S` — i.e., a pass schedule is *data*, reproducible and versionable. | |
| 89 | +- **Pass instrumentation** hooks allow logging/timing/verification around every pass execution (this is how `-verify-each` style checking is done). | |
| 90 | + | |
| 91 | +## 4. Backends: the target-independent code generator | |
| 92 | + | |
| 93 | +Source: CodeGenerator doc (consulted 2026-08-05). | |
| 94 | + | |
| 95 | +### 4.1 Separation of algorithm and target description | |
| 96 | + | |
| 97 | +The code generator splits into (a) **target-independent algorithms** (instruction selection, scheduling, register allocation, code emission) and (b) **abstract target description interfaces** that each concrete target implements: | |
| 98 | + | |
| 99 | +| Class | Describes | | |
| 100 | +|---|---| | |
| 101 | +| `TargetMachine` | Entry point; virtual accessors to everything below | | |
| 102 | +| `DataLayout` | Memory layout, alignment, pointer size, endianness (the one non-extensible class) | | |
| 103 | +| `TargetLowering` | Which IR operations the target supports natively, and how to legalize the rest | | |
| 104 | +| `TargetRegisterInfo` / `TargetInstrInfo` | Register file and instruction set | | |
| 105 | +| `TargetFrameLowering` / `TargetSubtarget` | Stack conventions; chip-specific features and latencies | | |
| 106 | + | |
| 107 | +The pipeline: **instruction selection → scheduling → SSA machine optimizations → register allocation → prologue/epilogue → late peepholes → MC emission** (assembly or object file via `MCStreamer`/`MCInst`). | |
| 108 | + | |
| 109 | +### 4.2 SelectionDAG and GlobalISel (conceptual) | |
| 110 | + | |
| 111 | +- **SelectionDAG**: the IR of each block is expanded into a dataflow DAG, then *legalized* — unsupported types are promoted/expanded, unsupported operations are expanded, promoted, or handled by custom target callbacks — then pattern-matched against target instructions and scheduled. Key idea: **a generic program representation is progressively lowered until everything in it is expressible on the concrete target**, with the target declaring its capabilities rather than the core knowing about targets. | |
| 112 | +- **GlobalISel**: a newer, more modular instruction-selection framework operating across block boundaries; same philosophy, different machinery. | |
| 113 | + | |
| 114 | +### 4.3 TableGen: declarative target descriptions | |
| 115 | + | |
| 116 | +Rather than hand-writing C++ for every target detail, LLVM targets are largely described **declaratively** in `.td` (TableGen) files: registers, instruction definitions, calling conventions, addressing modes, selection patterns. TableGen compiles these records into generated C++. Benefits called out in the docs: | |
| 117 | + | |
| 118 | +- drastically less boilerplate; the architecture spec lives in one place; | |
| 119 | +- pattern fragments are expanded automatically, type inference propagates constraints, commutative variants are derived without duplication; | |
| 120 | +- cross-cutting changes update all backends mechanically. | |
| 121 | + | |
| 122 | +The lesson: **domain rules belong in a declarative, checkable description language, not scattered through imperative engine code.** | |
| 123 | + | |
| 124 | +## 5. Diagnostics philosophy (Clang) | |
| 125 | + | |
| 126 | +Source: clang.llvm.org/diagnostics.html (consulted 2026-08-05). | |
| 127 | + | |
| 128 | +Clang treats error messages as a first-class product: | |
| 129 | + | |
| 130 | +- **Exact location**: line *and column*, with a caret (`^`) pointing at the precise token — "even inside of a string" — not merely where the parser gave up. | |
| 131 | +- **Ranges**: the operands/expressions involved are underlined, so the shape of the problem is visible without re-reading the line. | |
| 132 | +- **Cause, not ceremony**: messages state the inferred facts that matter (e.g., the actual types of both sides of an operator), skipping the obvious. | |
| 133 | +- **Fix-it hints**: where the correction is unambiguous, the compiler proposes the exact edit (e.g., insert a missing `typename`). | |
| 134 | +- **Readable types**: typedefs are preserved, with `aka` unwrapping when helpful; template type diffing prints only what differs. | |
| 135 | +- **Context chains**: errors inside macros automatically show the instantiation chain. | |
| 136 | + | |
| 137 | +The philosophy: a diagnostic must give **location + cause + suggested fix**, in the user's own vocabulary. | |
| 138 | + | |
| 139 | +--- | |
| 140 | + | |
| 141 | +## 6. Transposition to AIR | |
| 142 | + | |
| 143 | +| LLVM | AIR | Notes | | |
| 144 | +|---|---|---| | |
| 145 | +| LLVM IR (module/function/block/instruction) | AIR `EconomicEvent` documents | The universal, fully-specified exchange format; the *only* interface between understanding (LLM) and rule application (AIC) | | |
| 146 | +| SSA form + dominance verifier | Provenance graph: every amount has a single traceable origin | Invoice → Tax → Payment → FX → Settlement → Write-off | | |
| 147 | +| Pass manager (analysis + transformation passes) | AIC pass pipeline: validation, tax, FX, fraud, approval, optimization | Balance invariant `Assets = Liabilities + Equity` checked after every pass, like `-verify-each` | | |
| 148 | +| Target backends behind abstract `Target*` interfaces | ERP backends behind a common `Backend` interface | CSV, QuickBooks, Xero, Odoo, SAP; `capabilities()` ≈ `TargetLowering` legality | | |
| 149 | +| TableGen `.td` target descriptions | ALSL declarative policy/tax/norm rules | No rates or thresholds in engine code | | |
| 150 | +| Clang diagnostics | AIC compilation diagnostics | Location + cause + fix-it, pointing at the offending AIR field | | |
| 151 | + | |
| 152 | +### 6.1 LLVM IR → AIR economic events | |
| 153 | + | |
| 154 | +LLVM's central bet — a **well-specified IR that is the only interface** — is exactly AIR's bet. An `EconomicEvent` is the common currency between M ingestion frontends (invoice OCR, email, bank feeds, POS, APIs, each possibly LLM-driven) and N accounting backends (ERPs and reporting frameworks): **M + N adapters instead of M × N point-to-point integrations**. Direct implications to adopt: | |
| 155 | + | |
| 156 | +- **Formal schema first** (JSON Schema as the LangRef equivalent); anything that fails validation is rejected outright, like IR the verifier refuses. | |
| 157 | +- **Isomorphic forms**: a canonical serialized form (JSON/YAML documents at rest), an in-memory typed object model (Pydantic/Rust types), and a human-readable rendering — all provably equivalent, so any stage of compilation can be dumped, diffed, and replayed. | |
| 158 | +- **`air_version` in every document**, mirroring how bitcode compatibility is managed explicitly. | |
| 159 | +- **Metadata channel**: OCR scores, LLM model/confidence, `reasoning_hash`, policy version, and approver live in `meta`, exactly as LLVM metadata carries provenance *without affecting semantics* — the compiler's output must be identical whether or not metadata is present (determinism), while remaining fully traceable. | |
| 160 | + | |
| 161 | +### 6.2 SSA → single-origin amounts and the provenance graph | |
| 162 | + | |
| 163 | +SSA's rule — *each value defined exactly once, every use dominated by its definition* — transposes to money: **every monetary amount in the system has exactly one defining node in an immutable provenance graph**, and any derived amount (a tax line, an FX-converted total, a settlement allocation, a write-off) must reference the node it was derived from. "Dominance" becomes: *an amount cannot be consumed by a downstream step (Payment, Settlement) before the step that defines it (Invoice, Tax) exists in the graph.* An AIR verifier — the analogue of LLVM's verification pass — rejects any compilation unit containing an amount with zero or multiple origins, a dangling reference, or a use-before-definition, with the same hard-error posture: malformed AIR never enters the pipeline. Like PHI nodes making control-flow merges explicit, merges of money (netting, batch settlement of many invoices by one payment) are explicit graph nodes, never in-place mutation. | |
| 164 | + | |
| 165 | +### 6.3 Pass manager → AIC pipeline | |
| 166 | + | |
| 167 | +- **Stratified pass kinds**: LLVM's Module/CGSCC/Function/Loop levels map to AIR scopes — entity-wide passes (period close, reconciliation), batch passes (fusion, netting, duplicate detection), and per-event passes (validation, tax, FX, classification) — with adaptors to run event-level passes inside batch pipelines. | |
| 168 | +- **Analyses vs transformations**: AIC should separate *analyses* (duplicate-hash index, FX rate table lookup, policy applicability, running trial balance) from *transformations* (adding tax lines, reclassifying Expense→Asset, generating reversals). Analyses are cached by an analysis manager and invalidated via a `PreservedAnalyses`-style declaration when a transformation touches the events they cover — this is what makes **incremental recompilation** (recompile only the delta when an invoice changes) tractable. | |
| 169 | +- **Explicit, scriptable ordering**: the pipeline (OCR → Classification → Tax → FX → Fraud → Approval → Optimizations → Posting) is defined centrally by a PassBuilder equivalent and expressible as data (versioned config), with extension points instead of ad-hoc insertion. | |
| 170 | +- **Verify-each invariant**: LLVM instrumentation can run the verifier after every pass; AIC makes this mandatory, not optional — **`Assets = Liabilities + Equity` (and per-document debit=credit) is checked after every pass**, and any violation aborts compilation with a precise diagnostic. No pass may break the invariant even transiently across pass boundaries. | |
| 171 | + | |
| 172 | +### 6.4 Backends → ERP generators | |
| 173 | + | |
| 174 | +LLVM's split between target-independent algorithms and per-target descriptions maps directly onto AIR's `Backend` interface: | |
| 175 | + | |
| 176 | +- `capabilities()` ≈ `TargetLowering` legality declarations: which account types, tax treatments, multi-currency features, and posting granularities the target ERP supports. | |
| 177 | +- **Legalization** ≈ lowering a compiled journal to what the target can express: if an ERP can't represent a compound entry, expand it into multiple simple entries; if it lacks a dedicated tax field, promote tax to explicit journal lines — the core never special-cases ERPs; each backend declares and handles its own gaps, exactly like LegalizeOps expansion/promotion/custom hooks. | |
| 178 | +- `compile()/post()/reverse()` ≈ instruction selection + MC emission + the ability to undo: `post()` returns a `PostingReceipt` the way MC emission produces a concrete artifact. | |
| 179 | +- Development order (generic CSV first, then QBO, Xero, Odoo, SAP) mirrors how a simple reference backend validates the target-independent core before hard targets. | |
| 180 | + | |
| 181 | +### 6.5 TableGen → ALSL | |
| 182 | + | |
| 183 | +TableGen is the strongest single precedent for ALSL: **all target-specific facts live in declarative, versioned `.td` descriptions compiled into the engine, never hand-coded in the core**. For AIR: every tax rate, capitalization threshold, jurisdiction rule, and accounting-policy choice lives in ALSL policies (versioned, testable, diffable), and the AIC engine contains zero fiscal constants. TableGen's derived conveniences (pattern expansion, type inference, auto-derived variants) suggest ALSL should support rule composition and inheritance (e.g., a `CA-QC` policy extending a `CA` base) rather than copy-paste rules. Starting ALSL as a constrained YAML subset before a full DSL parallels how TableGen is a small language with generated artifacts, not a general-purpose one. | |
| 184 | + | |
| 185 | +### 6.6 Diagnostics → clang-quality compile errors | |
| 186 | + | |
| 187 | +AIC diagnostics adopt Clang's triad — **location, cause, fix-it**: | |
| 188 | + | |
| 189 | +- **Location**: JSON-pointer/path into the offending AIR document and field (`events[3].items[0].unit_price`), the accounting analogue of line+column+caret. | |
| 190 | +- **Cause with inferred facts**: state what the compiler actually computed, e.g. "entry unbalanced by 0.01 CAD: debits 1150.00, credits 1149.99 — after Tax pass (QST rounding)" — the analogue of printing the inferred operand types. | |
| 191 | +- **Fix-it hints**: "jurisdiction `CA-QC` requires tax codes [GST, QST]; event declares only [GST] — add QST or change jurisdiction." | |
| 192 | +- **Context chains**: like macro-expansion notes, a diagnostic on a derived amount shows its provenance chain (which pass, which ALSL policy and version, which source document and OCR/LLM confidence produced the value). | |
| 193 | + | |
| 194 | +--- | |
| 195 | + | |
| 196 | +## 7. Sources | |
| 197 | + | |
| 198 | +All consulted 2026-08-05: | |
| 199 | + | |
| 200 | +1. **LLVM Language Reference Manual** — LLVM Project. https://llvm.org/docs/LangRef.html — IR hierarchy, identifiers, SSA/dominance, three isomorphic forms, metadata, verification pass. | |
| 201 | +2. **Using the New Pass Manager** — LLVM Project. https://llvm.org/docs/NewPassManager.html — pass kinds, AnalysisManager caching, `PreservedAnalyses` invalidation, adaptors, PassBuilder pipelines, instrumentation. | |
| 202 | +3. **The LLVM Target-Independent Code Generator** — LLVM Project. https://llvm.org/docs/CodeGenerator.html — codegen phases, abstract target classes, SelectionDAG legalization, GlobalISel, TableGen-driven target descriptions, MC layer. | |
| 203 | +4. **Clang: Expressive Diagnostics** — LLVM Project. https://clang.llvm.org/diagnostics.html — caret diagnostics, ranges, fix-it hints, typedef preservation, template diffing, macro expansion notes. | |
| 204 | +5. **Chris Lattner, "LLVM"**, in *The Architecture of Open Source Applications*, vol. 1. https://aosabook.org/en/v1/llvm.html — three-phase design, M+N retargetability, IR as sole fully-specified interface, library-based modularity. | |
added
docs/spec/air-spec-v0.1.md
+240 −0
@@ -0,0 +1,240 @@ | ||
| 1 | +<!-- | |
| 2 | +Projet : AIR — Accounting Intermediate Representation | |
| 3 | +Auteur : Simon-Pierre Boucher | |
| 4 | +Contact : contact@spboucher.ai | |
| 5 | +Fichier : air-spec-v0.1.md | |
| 6 | +--> | |
| 7 | + | |
| 8 | +# AIR Specification v0.1 — The Language of Accounting | |
| 9 | + | |
| 10 | +AIR (Accounting Intermediate Representation) is to accounting what LLVM IR is | |
| 11 | +to compilation: a universal intermediate language. AIR describes **economic | |
| 12 | +events** — what actually happened in the world — and a deterministic compiler | |
| 13 | +(AIC) lowers them into journal entries under versioned rule sets (ALSL). | |
| 14 | + | |
| 15 | +The core contract: | |
| 16 | + | |
| 17 | +> **An LLM (or any frontend) may only ever produce AIR. It never produces a | |
| 18 | +> journal entry.** The compiler applies taxes, standards, and policies | |
| 19 | +> deterministically, traceably, and testably. | |
| 20 | + | |
| 21 | +AIR is **self-sufficient**: with the native backend, AIR is a complete | |
| 22 | +standalone accounting system (hash-chained ledger + general ledger, trial | |
| 23 | +balance, income statement, balance sheet in text/markdown/csv/json). Third | |
| 24 | +party systems (QuickBooks, Xero, Odoo, SAP...) are optional export targets. | |
| 25 | + | |
| 26 | +## 1. Document form | |
| 27 | + | |
| 28 | +An AIR document is YAML or JSON validated against | |
| 29 | +`schemas/air-0.1.schema.json`. Three isomorphic forms exist (as in LLVM): | |
| 30 | +in-memory typed objects (`core/events.py`), canonical JSON, and readable YAML. | |
| 31 | + | |
| 32 | +```yaml | |
| 33 | +air_version: "0.1" | |
| 34 | +events: | |
| 35 | + - id: evt_01H8XGJWBWBAQ4Z1 | |
| 36 | + type: Sale | |
| 37 | + date: 2026-07-20 | |
| 38 | + parties: {seller: "company:acme", buyer: "customer:cust_123"} | |
| 39 | + items: | |
| 40 | + - {sku: chair-std, qty: "3", unit_price: {amount: "333.33", currency: CAD}} | |
| 41 | + payment: {method: card.visa, immediate: true} | |
| 42 | + tax: {jurisdiction: CA-QC, codes: [GST, QST]} | |
| 43 | + meta: | |
| 44 | + source: {kind: invoice_pdf, uri: "s3://...", ocr_score: "0.97"} | |
| 45 | + llm: {model: "claude-fable-5", confidence: "0.93"} | |
| 46 | + policy_version: "2026.08" | |
| 47 | +``` | |
| 48 | + | |
| 49 | +### 1.1 Lexical rules | |
| 50 | + | |
| 51 | +- **Amounts, quantities, rates are strings** (`"333.33"`). Bare numbers would | |
| 52 | + be parsed as binary floats and are **rejected** (ADR 0003). | |
| 53 | +- `id` is unique per document (ULIDs recommended). | |
| 54 | +- Dates are ISO 8601. | |
| 55 | +- Unknown fields are rejected (`extra="forbid"`): the schema is the contract. | |
| 56 | + | |
| 57 | +### 1.2 Event types (v0.1) | |
| 58 | + | |
| 59 | +| Type | Meaning | Posting shape (native profile) | | |
| 60 | +|---|---|---| | |
| 61 | +| `Sale` | value delivered to a buyer | DR cash/AR total · CR revenue · CR tax payables | | |
| 62 | +| `Purchase` | value received from a vendor | DR expense/asset (+ non-recoverable tax) · DR recoverable taxes · CR cash/AP | | |
| 63 | +| `Refund` | reversal of a sale | mirror of Sale | | |
| 64 | +| `PaymentReceived` | settles a receivable | DR cash · CR AR · ± realized FX | | |
| 65 | +| `PaymentSent` | settles a payable | DR AP · CR cash · ± realized FX | | |
| 66 | +| `OwnerContribution` | equity injection | DR cash · CR owner capital | | |
| 67 | +| `LoanReceived` | debt proceeds | DR cash · CR loan payable | | |
| 68 | + | |
| 69 | +Semantic notes: | |
| 70 | + | |
| 71 | +- **No account, no debit, no credit appears in AIR.** Events are | |
| 72 | + perspective-neutral (ADR 0002): the same event compiles to the seller's or | |
| 73 | + buyer's books depending on the compiling entity's policy set. | |
| 74 | +- `tax.jurisdiction` names *where* the supply takes place; rates live only in | |
| 75 | + ALSL policies. | |
| 76 | +- `fx.rate` is **observed input data** (e.g. a Bank of Canada Valet daily | |
| 77 | + rate on the transaction date), never a constant of the system. | |
| 78 | + | |
| 79 | +## 2. Provenance — accounting SSA | |
| 80 | + | |
| 81 | +Every amount in a compilation is defined exactly once as a node in an | |
| 82 | +append-only DAG (`core/provenance.py`). Derivations record their operation: | |
| 83 | + | |
| 84 | +``` | |
| 85 | +prov_000000 event_subtotal subtotal:Sale 999.99 CAD <- evt_... | |
| 86 | +prov_000001 tax tax:GST@0.05~half_up 50.00 CAD <- prov_000000 | |
| 87 | +prov_000002 tax tax:QST@0.09975~half_up 99.75 CAD <- prov_000000 | |
| 88 | +prov_000005 journal_line post:credit:2320 99.75 CAD <- prov_000002 | |
| 89 | +``` | |
| 90 | + | |
| 91 | +From any posted line one can walk back to the source event, its document URI, | |
| 92 | +its OCR/LLM confidence, and the exact policy and rounding mode applied. | |
| 93 | + | |
| 94 | +## 3. The AIC pipeline | |
| 95 | + | |
| 96 | +``` | |
| 97 | +ValidationPass -> ClassificationPass -> TaxPass -> FxPass -> PostingPass | |
| 98 | +``` | |
| 99 | + | |
| 100 | +After **every** pass the verifier checks (a) each entry balances per currency | |
| 101 | +and (b) the accounting equation | |
| 102 | +`Assets = Liabilities + Equity + (Revenue − Expenses)`. Any violation aborts | |
| 103 | +with clang-style diagnostics (code, location, cause, `help:` fix), e.g.: | |
| 104 | + | |
| 105 | +``` | |
| 106 | +error[AIR-E400]: no tax policy in set 'ca-qc' matches jurisdiction 'CA-BC' | |
| 107 | + --> event evt_x, field tax.jurisdiction | |
| 108 | + pass: tax | |
| 109 | + help: add an ALSL tax policy for this jurisdiction or mark the event tax.exempt: true | |
| 110 | +``` | |
| 111 | + | |
| 112 | +Determinism: same document + same policy set = byte-identical journal. No | |
| 113 | +LLM, network, or clock inside the compiler (property-tested). | |
| 114 | + | |
| 115 | +### 3.1 Diagnostic codes | |
| 116 | + | |
| 117 | +| Code | Meaning | | |
| 118 | +|---|---| | |
| 119 | +| AIR-E100/E101/E102 | empty entry / unbalanced entry / accounting equation violated | | |
| 120 | +| AIR-E200..E203 | duplicate id / mixed currencies / no amount / negative subtotal | | |
| 121 | +| AIR-W200/W201 | taxable event without tax context / refund without related_event | | |
| 122 | +| AIR-W300 | multiple classification policies matched | | |
| 123 | +| AIR-E400/E401, AIR-W400 | no tax policy for jurisdiction / unsupported base / requested code not produced | | |
| 124 | +| AIR-E500..E502 | missing fx rate / settlement without related event / related event unusable | | |
| 125 | +| AIR-E600/E601, AIR-W600 | missing account role / no posting rule / stated gross differs from computed total | | |
| 126 | + | |
| 127 | +## 4. ALSL v0.1 | |
| 128 | + | |
| 129 | +A YAML subset (full DSL later). A policy set carries: `functional_currency`, | |
| 130 | +`rounding` (mode + source), `accounts` (role → code/name/type), and | |
| 131 | +`policies` (kind `tax` or `classification`). Loader strictness: bare-float | |
| 132 | +rates are rejected; **tax policies without a source citation are rejected**. | |
| 133 | +See `alsl/policies/ca-qc-2026.yaml` for the reference set (GST 5%, | |
| 134 | +QST 9.975% on the pre-GST price, half-up per ETA s. 165.2(2) — sources in | |
| 135 | +docs/research/canada-gst-qst.md, verified 2026-08-05). | |
| 136 | + | |
| 137 | +## 5. Backends | |
| 138 | + | |
| 139 | +Common interface (`backends/base.py`): `capabilities()`, | |
| 140 | +`compile(journal) -> TargetPayload`, `post(payload, idempotency_key)`, | |
| 141 | +`reverse(receipt)`. Posted entries are never deleted — reversal appends | |
| 142 | +contra entries (or uses the target's native mechanism). | |
| 143 | + | |
| 144 | +- **native** — AIR standalone: append-only, hash-chained JSONL ledger with | |
| 145 | + idempotent posting and built-in statements (kernel/ledger.py, reporting.py). | |
| 146 | + Managed by the **AIR home** (kernel/workspace.py): one directory holding | |
| 147 | + meta, ledger, content-addressed document archive, and exports — the whole | |
| 148 | + book of record is reproducible from it. | |
| 149 | +- **generic_csv** — universal flat-file journal export. | |
| 150 | +- **quickbooks** — QBO JournalEntry payloads (requestid idempotency, | |
| 151 | + minorversion=75, DocNumber ≤ 21 chars, contra-entry reversal). QuickBooks | |
| 152 | + is OPTIONAL: the backend is developed and tested entirely offline against | |
| 153 | + a mock transport that simulates QBO's documented behaviors (idempotent | |
| 154 | + replay, duplicate-DocNumber error 6140); `qbo-export` produces | |
| 155 | + QBO-shaped JSON with no account at all. The real HTTP transport activates | |
| 156 | + only when the user supplies OAuth credentials. | |
| 157 | +- Planned per docs/research/erp-apis.md: Xero (Idempotency-Key header), Odoo | |
| 158 | + (account.move), beancount text export, SAP (deferred). | |
| 159 | + | |
| 160 | +## 6. Ingestion — the only place an LLM appears | |
| 161 | + | |
| 162 | +``` | |
| 163 | +source text -> Extractor -> AIR schema validation -> confidence routing | |
| 164 | + | | | |
| 165 | + auto-approved human inbox | |
| 166 | + \\ / | |
| 167 | + deterministic compiler | |
| 168 | +``` | |
| 169 | + | |
| 170 | +The extractor (offline mock, or Claude with schema-constrained structured | |
| 171 | +output) emits **AIR events only** — never journal entries, accounts, or | |
| 172 | +debits/credits. Every extraction is validated against the AIR schema; | |
| 173 | +schema-invalid output ALWAYS routes to the human inbox, as does anything | |
| 174 | +below the confidence threshold (default 0.85). Approval stamps the approver | |
| 175 | +and timestamp into event meta (`ingestion/approval.py`); even approved | |
| 176 | +documents then go through the deterministic compiler. The inbox lives in the | |
| 177 | +AIR home (`inbox/pending|approved|rejected/`) — nothing is deleted. | |
| 178 | + | |
| 179 | +## 7. Agent syscalls & the audit log | |
| 180 | + | |
| 181 | +AI agents never touch the ledger, the compiler internals, or the files. Their | |
| 182 | +entire surface is the kernel syscall interface (`sdk/syscalls.py`): | |
| 183 | + | |
| 184 | +| Syscall | Effect | | |
| 185 | +|---|---| | |
| 186 | +| `CreateEconomicEvent` | schema-validate + stage a draft in `<home>/drafts/` | | |
| 187 | +| `Validate` / `Compile` | run the deterministic compiler on the drafts, no posting | | |
| 188 | +| `Post` | compile + append to the books (idempotent by draft content hash); refuses closed periods (`AIR-E700`); archives the drafts | | |
| 189 | +| `Reverse` | append the exact contra of a posted entry — never edits | | |
| 190 | +| `ClosePeriod` | close `YYYY-MM`; later posts into it are refused | | |
| 191 | +| `GenerateReport` | any statement, any format | | |
| 192 | +| `Merge` | post the drafts through the optimization passes (below) | | |
| 193 | +| `Reconcile` | match a bank statement against the books' cash movements | | |
| 194 | + | |
| 195 | +Every syscall — successful or refused — appends one record to the | |
| 196 | +**hash-chained audit log** (`<home>/audit.jsonl`, kernel/audit.py): actor, | |
| 197 | +syscall, parameter digest, outcome, timestamp, `prev_hash`, `hash`. Editing, | |
| 198 | +deleting, or reordering any record breaks `air audit` verification. A refused | |
| 199 | +action is evidence too — denials are logged like successes. | |
| 200 | + | |
| 201 | +## 8. Optimization passes & bank reconciliation | |
| 202 | + | |
| 203 | +Opt-in (`compile --optimize`, or the `Merge` syscall); the invariant is still | |
| 204 | +verified after every pass, and optimized lines carry provenance nodes pointing | |
| 205 | +to every line they absorbed (`aic/passes/optimize.py`): | |
| 206 | + | |
| 207 | +- **DuplicateDetectionPass** — identical economic content under different ids | |
| 208 | + → warning `AIR-W800`; nothing is dropped, a human decides. | |
| 209 | +- **NettingPass** — a Refund referencing a Sale in the same compilation nets | |
| 210 | + against it: one net entry (or none when fully offset; note `AIR-N801`). | |
| 211 | + Refunds exceeding their sale are left alone. | |
| 212 | +- **FusionPass** — same-day payments with an identical posting shape fuse into | |
| 213 | + one batch entry, amounts summed per line (note `AIR-N800`). 50 payments → | |
| 214 | + 1 entry — which then matches the bank's single batch deposit one-to-one. | |
| 215 | + | |
| 216 | +**Reconciliation** (`kernel/reconcile.py`, formats per | |
| 217 | +docs/research/bank-statement-formats.md): camt.053 (namespace-agnostic, | |
| 218 | +sign from `CdtDbtInd`, reference from `EndToEndId`), MT940 (`:61:` lines, | |
| 219 | +comma decimals, D/C/RD/RC marks, opening±movements=closing integrity check), | |
| 220 | +and CSV, all normalized to signed-Decimal `BankTransaction` records. Matching | |
| 221 | +is exact on (signed amount, currency) within a configurable date-tolerance | |
| 222 | +window; unmatched items on both sides are reported, never dropped. | |
| 223 | + | |
| 224 | +## 9. Versioning & incremental compilation | |
| 225 | + | |
| 226 | +`air_version` is mandatory; schema migrations are explicit. | |
| 227 | + | |
| 228 | +Changed source documents recompile as **deltas** (`aic/incremental.py`, | |
| 229 | +CLI `recompile old.yaml new.yaml`). Events are compared by content | |
| 230 | +fingerprint (SHA-256 over canonical JSON); the delta contains: | |
| 231 | + | |
| 232 | +- an exact contra **reversal entry** (`rev_je_<event>`, `reverses` link) for | |
| 233 | + every changed or removed event; | |
| 234 | +- a **replacement entry** with a deterministic revision-suffixed id | |
| 235 | + (`je_<event>_r<fp8>`) for every changed event; | |
| 236 | +- a normal entry for every added event; nothing for unchanged events. | |
| 237 | + | |
| 238 | +Posted history is never mutated. The delta itself must satisfy the | |
| 239 | +double-entry invariant, and posting `v1 + delta` yields byte-identical | |
| 240 | +balances to compiling `v2` directly (tested in tests/test_incremental.py). | |
added
ingestion/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
ingestion/approval.py
+134 −0
@@ -0,0 +1,134 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : approval.py | |
| 6 | +# Description : Human approval queue — file-backed inbox inside the AIR home. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Approval queue: the human-in-the-loop stage. | |
| 9 | + | |
| 10 | +Low-confidence or schema-invalid extractions land here instead of the ledger. | |
| 11 | +The queue lives inside the AIR home: | |
| 12 | + | |
| 13 | + <home>/inbox/pending/<item>.json | |
| 14 | + <home>/inbox/approved/<item>.json | |
| 15 | + <home>/inbox/rejected/<item>.json | |
| 16 | + | |
| 17 | +Approving stamps the approver + timestamp into every event's meta (full | |
| 18 | +traceability: who let this into the books) and returns the AirDocument ready | |
| 19 | +for compilation. Nothing is ever deleted — items move between folders. | |
| 20 | +""" | |
| 21 | +from __future__ import annotations | |
| 22 | + | |
| 23 | +import json | |
| 24 | +from dataclasses import dataclass | |
| 25 | +from datetime import datetime | |
| 26 | +from pathlib import Path | |
| 27 | +from typing import Any | |
| 28 | + | |
| 29 | +from core.events import AirDocument | |
| 30 | +from ingestion.pipeline import IngestionOutcome | |
| 31 | + | |
| 32 | +_AUTHOR = "Simon-Pierre Boucher <contact@spboucher.ai>" | |
| 33 | + | |
| 34 | + | |
| 35 | +@dataclass(frozen=True, slots=True) | |
| 36 | +class InboxItem: | |
| 37 | + id: str | |
| 38 | + status: str # pending | approved | rejected | |
| 39 | + confidence: str | |
| 40 | + reasons: list[str] | |
| 41 | + validation_errors: list[str] | |
| 42 | + events: list[dict[str, Any]] | |
| 43 | + source_excerpt: str | |
| 44 | + | |
| 45 | + | |
| 46 | +class ApprovalQueue: | |
| 47 | + def __init__(self, home: str | Path): | |
| 48 | + self.root = Path(home) / "inbox" | |
| 49 | + for status in ("pending", "approved", "rejected"): | |
| 50 | + (self.root / status).mkdir(parents=True, exist_ok=True) | |
| 51 | + | |
| 52 | + # -- intake ------------------------------------------------------------------ | |
| 53 | + def submit(self, outcome: IngestionOutcome, source_text: str) -> str: | |
| 54 | + """File a needs-review outcome into the pending inbox. Returns item id.""" | |
| 55 | + events = ( | |
| 56 | + [json.loads(e.model_dump_json(exclude_none=True)) | |
| 57 | + for e in outcome.document.events] | |
| 58 | + if outcome.document is not None | |
| 59 | + else outcome.extraction.events | |
| 60 | + ) | |
| 61 | + item_id = f"inbox_{len(list(self.root.rglob('*.json'))):05d}" | |
| 62 | + payload = { | |
| 63 | + "_author": _AUTHOR, | |
| 64 | + "id": item_id, | |
| 65 | + "status": "pending", | |
| 66 | + "confidence": str(outcome.extraction.confidence), | |
| 67 | + "extractor": outcome.extraction.extractor, | |
| 68 | + "reasons": outcome.reasons, | |
| 69 | + "validation_errors": outcome.validation_errors, | |
| 70 | + "events": events, | |
| 71 | + "source_excerpt": source_text[:2000], | |
| 72 | + } | |
| 73 | + (self.root / "pending" / f"{item_id}.json").write_text( | |
| 74 | + json.dumps(payload, indent=2, default=str), encoding="utf-8" | |
| 75 | + ) | |
| 76 | + return item_id | |
| 77 | + | |
| 78 | + # -- inspection --------------------------------------------------------------- | |
| 79 | + def _load(self, status: str) -> list[InboxItem]: | |
| 80 | + items = [] | |
| 81 | + for path in sorted((self.root / status).glob("*.json")): | |
| 82 | + data = json.loads(path.read_text(encoding="utf-8")) | |
| 83 | + items.append(InboxItem( | |
| 84 | + id=data["id"], status=data["status"], | |
| 85 | + confidence=data["confidence"], | |
| 86 | + reasons=data.get("reasons", []), | |
| 87 | + validation_errors=data.get("validation_errors", []), | |
| 88 | + events=data.get("events", []), | |
| 89 | + source_excerpt=data.get("source_excerpt", ""), | |
| 90 | + )) | |
| 91 | + return items | |
| 92 | + | |
| 93 | + def pending(self) -> list[InboxItem]: | |
| 94 | + return self._load("pending") | |
| 95 | + | |
| 96 | + # -- decisions ---------------------------------------------------------------- | |
| 97 | + def _move(self, item_id: str, new_status: str, | |
| 98 | + mutate: dict[str, Any]) -> dict[str, Any]: | |
| 99 | + source = self.root / "pending" / f"{item_id}.json" | |
| 100 | + if not source.exists(): | |
| 101 | + raise KeyError(f"no pending inbox item '{item_id}'") | |
| 102 | + data = json.loads(source.read_text(encoding="utf-8")) | |
| 103 | + data["status"] = new_status | |
| 104 | + data.update(mutate) | |
| 105 | + target = self.root / new_status / f"{item_id}.json" | |
| 106 | + target.write_text(json.dumps(data, indent=2, default=str), | |
| 107 | + encoding="utf-8") | |
| 108 | + source.unlink() | |
| 109 | + return data | |
| 110 | + | |
| 111 | + def approve(self, item_id: str, approver: str, | |
| 112 | + approved_at: datetime | None = None) -> AirDocument: | |
| 113 | + """Human approval: stamp approver + timestamp, return the document. | |
| 114 | + | |
| 115 | + The returned document still goes through the deterministic compiler — | |
| 116 | + approval authorizes compilation, it never writes to the ledger itself. | |
| 117 | + """ | |
| 118 | + when = approved_at.isoformat() if approved_at else None | |
| 119 | + data = self._move(item_id, "approved", | |
| 120 | + {"approver": approver, "approved_at": when}) | |
| 121 | + events = [] | |
| 122 | + for event in data["events"]: | |
| 123 | + meta = dict(event.get("meta") or {}) | |
| 124 | + meta["approver"] = approver | |
| 125 | + timestamps = dict(meta.get("timestamps") or {}) | |
| 126 | + if when: | |
| 127 | + timestamps["approved"] = when | |
| 128 | + if timestamps: | |
| 129 | + meta["timestamps"] = timestamps | |
| 130 | + events.append({**event, "meta": meta}) | |
| 131 | + return AirDocument.model_validate({"events": events}) | |
| 132 | + | |
| 133 | + def reject(self, item_id: str, reason: str = "") -> None: | |
| 134 | + self._move(item_id, "rejected", {"rejection_reason": reason}) | |
added
ingestion/extractor.py
+233 −0
@@ -0,0 +1,233 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : extractor.py | |
| 6 | +# Description : Extractors — source text to AIR event data. Mock (offline) and Claude (optional). | |
| 7 | +# ============================================================================= | |
| 8 | +"""Extractors: understanding, separated from rule application. | |
| 9 | + | |
| 10 | +An extractor reads source text (an invoice, an email, OCR output) and | |
| 11 | +produces candidate AIR event data plus a confidence score. It NEVER produces | |
| 12 | +journal entries — that is the deterministic compiler's job (CLAUDE.md core | |
| 13 | +principle). | |
| 14 | + | |
| 15 | +Two implementations: | |
| 16 | +- MockExtractor (default for tests/offline): deterministic parser for a | |
| 17 | + simple key:value fixture format. No network, no API key, ever. | |
| 18 | +- ClaudeExtractor (optional): Claude API with schema-constrained structured | |
| 19 | + output. Only constructed when the user supplies/holds an Anthropic API key; | |
| 20 | + nothing in the test suite touches it. | |
| 21 | + | |
| 22 | +Design follows docs/research/llm-structured-extraction.md: schema-validated | |
| 23 | +output, confidence scoring, low confidence routes to human approval. | |
| 24 | +""" | |
| 25 | +from __future__ import annotations | |
| 26 | + | |
| 27 | +import abc | |
| 28 | +import json | |
| 29 | +from dataclasses import dataclass, field | |
| 30 | +from decimal import Decimal, InvalidOperation | |
| 31 | +from typing import Any | |
| 32 | + | |
| 33 | + | |
| 34 | +class ExtractionError(RuntimeError): | |
| 35 | + pass | |
| 36 | + | |
| 37 | + | |
| 38 | +@dataclass(frozen=True, slots=True) | |
| 39 | +class ExtractionResult: | |
| 40 | + """Candidate AIR data + how sure the extractor is about it.""" | |
| 41 | + | |
| 42 | + events: list[dict[str, Any]] # candidate EconomicEvent payloads | |
| 43 | + confidence: Decimal # 0..1, per-document | |
| 44 | + extractor: str # e.g. "mock", "claude:claude-opus-5" | |
| 45 | + notes: str = "" | |
| 46 | + | |
| 47 | + | |
| 48 | +class Extractor(abc.ABC): | |
| 49 | + @abc.abstractmethod | |
| 50 | + def extract(self, text: str) -> ExtractionResult: ... | |
| 51 | + | |
| 52 | + | |
| 53 | +# --- Mock extractor (offline, deterministic) ----------------------------------- | |
| 54 | +class MockExtractor(Extractor): | |
| 55 | + """Parses the AIR fixture format — deterministic, no network. | |
| 56 | + | |
| 57 | + Format (one event per block, blank-line separated): | |
| 58 | + | |
| 59 | + type: Sale | |
| 60 | + id: evt_x | |
| 61 | + date: 2026-07-15 | |
| 62 | + amount: 1000.00 CAD | |
| 63 | + jurisdiction: CA-QC | |
| 64 | + confidence: 0.95 | |
| 65 | + | |
| 66 | + Unknown or malformed blocks lower confidence instead of crashing — | |
| 67 | + mirroring how a real extractor degrades on noisy documents. | |
| 68 | + """ | |
| 69 | + | |
| 70 | + def extract(self, text: str) -> ExtractionResult: | |
| 71 | + events: list[dict[str, Any]] = [] | |
| 72 | + confidences: list[Decimal] = [] | |
| 73 | + blocks = [b for b in text.split("\n\n") if b.strip()] | |
| 74 | + for i, block in enumerate(blocks): | |
| 75 | + fields: dict[str, str] = {} | |
| 76 | + for line in block.splitlines(): | |
| 77 | + line = line.strip() | |
| 78 | + if not line or line.startswith("#") or ":" not in line: | |
| 79 | + continue | |
| 80 | + key, _, value = line.partition(":") | |
| 81 | + fields[key.strip().lower()] = value.strip() | |
| 82 | + if "type" not in fields: | |
| 83 | + continue | |
| 84 | + event: dict[str, Any] = { | |
| 85 | + "id": fields.get("id", f"evt_extracted_{i:03d}"), | |
| 86 | + "type": fields["type"], | |
| 87 | + "date": fields.get("date", ""), | |
| 88 | + } | |
| 89 | + if "description" in fields: | |
| 90 | + event["description"] = fields["description"] | |
| 91 | + if "amount" in fields: | |
| 92 | + parts = fields["amount"].split() | |
| 93 | + event["amount"] = { | |
| 94 | + "amount": parts[0], | |
| 95 | + "currency": parts[1] if len(parts) > 1 else "CAD", | |
| 96 | + } | |
| 97 | + if "jurisdiction" in fields: | |
| 98 | + event["tax"] = {"jurisdiction": fields["jurisdiction"]} | |
| 99 | + if fields.get("exempt", "").lower() == "true": | |
| 100 | + event["tax"]["exempt"] = True | |
| 101 | + if "related_event" in fields: | |
| 102 | + event["related_event"] = fields["related_event"] | |
| 103 | + if "immediate" in fields: | |
| 104 | + event["payment"] = { | |
| 105 | + "immediate": fields["immediate"].lower() == "true" | |
| 106 | + } | |
| 107 | + events.append(event) | |
| 108 | + try: | |
| 109 | + confidences.append(Decimal(fields.get("confidence", "0.9"))) | |
| 110 | + except InvalidOperation: | |
| 111 | + confidences.append(Decimal("0.5")) | |
| 112 | + if not events: | |
| 113 | + raise ExtractionError("mock extractor found no events in the source text") | |
| 114 | + return ExtractionResult( | |
| 115 | + events=events, | |
| 116 | + confidence=min(confidences), | |
| 117 | + extractor="mock", | |
| 118 | + ) | |
| 119 | + | |
| 120 | + | |
| 121 | +# --- Claude extractor (optional — requires an Anthropic API key) -------------------- | |
| 122 | +EXTRACTION_SCHEMA: dict[str, Any] = { | |
| 123 | + "type": "object", | |
| 124 | + "properties": { | |
| 125 | + "events": { | |
| 126 | + "type": "array", | |
| 127 | + "items": { | |
| 128 | + "type": "object", | |
| 129 | + "properties": { | |
| 130 | + "id": {"type": "string"}, | |
| 131 | + "type": {"type": "string", "enum": [ | |
| 132 | + "Sale", "Purchase", "Refund", "PaymentReceived", | |
| 133 | + "PaymentSent", "OwnerContribution", "LoanReceived", | |
| 134 | + ]}, | |
| 135 | + "date": {"type": "string", "description": "ISO 8601 date"}, | |
| 136 | + "description": {"type": "string"}, | |
| 137 | + "amount": { | |
| 138 | + "type": "object", | |
| 139 | + "properties": { | |
| 140 | + "amount": {"type": "string", | |
| 141 | + "description": "decimal as string, never a float"}, | |
| 142 | + "currency": {"type": "string"}, | |
| 143 | + }, | |
| 144 | + "required": ["amount", "currency"], | |
| 145 | + "additionalProperties": False, | |
| 146 | + }, | |
| 147 | + "jurisdiction": {"type": "string"}, | |
| 148 | + "exempt": {"type": "boolean"}, | |
| 149 | + "related_event": {"type": "string"}, | |
| 150 | + "immediate_payment": {"type": "boolean"}, | |
| 151 | + }, | |
| 152 | + "required": ["id", "type", "date", "amount"], | |
| 153 | + "additionalProperties": False, | |
| 154 | + }, | |
| 155 | + }, | |
| 156 | + "confidence": { | |
| 157 | + "type": "string", | |
| 158 | + "description": "overall extraction confidence 0..1 as a decimal string", | |
| 159 | + }, | |
| 160 | + "notes": {"type": "string"}, | |
| 161 | + }, | |
| 162 | + "required": ["events", "confidence"], | |
| 163 | + "additionalProperties": False, | |
| 164 | +} | |
| 165 | + | |
| 166 | +SYSTEM_PROMPT = """You extract economic events from business documents into AIR \ | |
| 167 | +(Accounting Intermediate Representation). | |
| 168 | + | |
| 169 | +Rules: | |
| 170 | +- You describe WHAT HAPPENED economically. You NEVER produce journal entries, \ | |
| 171 | +account codes, or debits/credits — a deterministic compiler applies those rules. | |
| 172 | +- All amounts and rates are decimal STRINGS ("19.99"), never numbers. | |
| 173 | +- Do not compute taxes; report the pre-tax amount and the jurisdiction. | |
| 174 | +- Report an honest overall confidence (0..1). If any field is uncertain or \ | |
| 175 | +illegible, lower it — low-confidence extractions are reviewed by a human.""" | |
| 176 | + | |
| 177 | + | |
| 178 | +class ClaudeExtractor(Extractor): | |
| 179 | + """LLM extraction via the Claude API (structured outputs). | |
| 180 | + | |
| 181 | + OPTIONAL: requires the `anthropic` package and an API key (resolved by the | |
| 182 | + SDK from the environment). Never used in tests — MockExtractor is the | |
| 183 | + offline default. | |
| 184 | + """ | |
| 185 | + | |
| 186 | + def __init__(self, model: str = "claude-opus-5"): | |
| 187 | + try: | |
| 188 | + import anthropic | |
| 189 | + except ImportError as exc: # pragma: no cover | |
| 190 | + raise ExtractionError( | |
| 191 | + "the 'anthropic' package is required for LLM extraction: " | |
| 192 | + "pip install anthropic — or use the mock extractor (--mock)" | |
| 193 | + ) from exc | |
| 194 | + self._client = anthropic.Anthropic() | |
| 195 | + self.model = model | |
| 196 | + | |
| 197 | + def extract(self, text: str) -> ExtractionResult: # pragma: no cover | |
| 198 | + response = self._client.messages.create( | |
| 199 | + model=self.model, | |
| 200 | + max_tokens=16000, | |
| 201 | + system=SYSTEM_PROMPT, | |
| 202 | + output_config={"format": {"type": "json_schema", | |
| 203 | + "schema": EXTRACTION_SCHEMA}}, | |
| 204 | + messages=[{"role": "user", "content": | |
| 205 | + f"Extract the economic events from this document:\n\n{text}"}], | |
| 206 | + ) | |
| 207 | + if response.stop_reason == "refusal": | |
| 208 | + raise ExtractionError("the model declined this document (refusal)") | |
| 209 | + payload = json.loads( | |
| 210 | + next(b.text for b in response.content if b.type == "text") | |
| 211 | + ) | |
| 212 | + events: list[dict[str, Any]] = [] | |
| 213 | + for raw in payload["events"]: | |
| 214 | + event: dict[str, Any] = { | |
| 215 | + "id": raw["id"], "type": raw["type"], "date": raw["date"], | |
| 216 | + "amount": raw["amount"], | |
| 217 | + } | |
| 218 | + if raw.get("description"): | |
| 219 | + event["description"] = raw["description"] | |
| 220 | + if raw.get("jurisdiction"): | |
| 221 | + event["tax"] = {"jurisdiction": raw["jurisdiction"], | |
| 222 | + "exempt": bool(raw.get("exempt"))} | |
| 223 | + if raw.get("related_event"): | |
| 224 | + event["related_event"] = raw["related_event"] | |
| 225 | + if raw.get("immediate_payment"): | |
| 226 | + event["payment"] = {"immediate": True} | |
| 227 | + events.append(event) | |
| 228 | + return ExtractionResult( | |
| 229 | + events=events, | |
| 230 | + confidence=Decimal(payload["confidence"]), | |
| 231 | + extractor=f"claude:{self.model}", | |
| 232 | + notes=payload.get("notes", ""), | |
| 233 | + ) | |
added
ingestion/pipeline.py
+102 −0
@@ -0,0 +1,102 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : pipeline.py | |
| 6 | +# Description : Ingestion pipeline — extract, validate against the AIR schema, route by confidence. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Ingestion pipeline. | |
| 9 | + | |
| 10 | + source text -> Extractor -> schema validation -> routing | |
| 11 | + | | | |
| 12 | + (Pydantic) auto-approve OR human inbox | |
| 13 | + | |
| 14 | +Routing rules (docs/research/llm-structured-extraction.md): | |
| 15 | +- schema-invalid extraction -> ALWAYS to the human inbox (with the errors) | |
| 16 | +- confidence < threshold -> human inbox | |
| 17 | +- confidence >= threshold -> auto-approved, ready to compile | |
| 18 | + | |
| 19 | +The threshold is configurable per home; the default is deliberately | |
| 20 | +conservative. The LLM's output NEVER goes to the ledger directly — even | |
| 21 | +auto-approved documents go through the deterministic compiler. | |
| 22 | +""" | |
| 23 | +from __future__ import annotations | |
| 24 | + | |
| 25 | +import enum | |
| 26 | +from dataclasses import dataclass, field | |
| 27 | +from datetime import datetime | |
| 28 | +from decimal import Decimal | |
| 29 | +from typing import Any | |
| 30 | + | |
| 31 | +from pydantic import ValidationError | |
| 32 | + | |
| 33 | +from core.events import AirDocument | |
| 34 | +from ingestion.extractor import ExtractionResult, Extractor | |
| 35 | + | |
| 36 | +DEFAULT_CONFIDENCE_THRESHOLD = Decimal("0.85") | |
| 37 | + | |
| 38 | + | |
| 39 | +class Route(str, enum.Enum): | |
| 40 | + AUTO_APPROVED = "auto_approved" | |
| 41 | + NEEDS_REVIEW = "needs_review" | |
| 42 | + | |
| 43 | + | |
| 44 | +@dataclass | |
| 45 | +class IngestionOutcome: | |
| 46 | + route: Route | |
| 47 | + extraction: ExtractionResult | |
| 48 | + document: AirDocument | None = None # set when schema-valid | |
| 49 | + validation_errors: list[str] = field(default_factory=list) | |
| 50 | + reasons: list[str] = field(default_factory=list) | |
| 51 | + | |
| 52 | + | |
| 53 | +def _stamp_meta( | |
| 54 | + events: list[dict[str, Any]], | |
| 55 | + extraction: ExtractionResult, | |
| 56 | + ingested_at: datetime | None, | |
| 57 | +) -> list[dict[str, Any]]: | |
| 58 | + """Attach traceability metadata to every extracted event.""" | |
| 59 | + stamped = [] | |
| 60 | + for event in events: | |
| 61 | + meta = dict(event.get("meta") or {}) | |
| 62 | + meta["llm"] = { | |
| 63 | + "model": extraction.extractor, | |
| 64 | + "confidence": str(extraction.confidence), | |
| 65 | + } | |
| 66 | + if ingested_at is not None: | |
| 67 | + meta["timestamps"] = {"ingested": ingested_at.isoformat()} | |
| 68 | + stamped.append({**event, "meta": meta}) | |
| 69 | + return stamped | |
| 70 | + | |
| 71 | + | |
| 72 | +def ingest( | |
| 73 | + text: str, | |
| 74 | + extractor: Extractor, | |
| 75 | + threshold: Decimal = DEFAULT_CONFIDENCE_THRESHOLD, | |
| 76 | + ingested_at: datetime | None = None, | |
| 77 | +) -> IngestionOutcome: | |
| 78 | + """Run one document through extract -> validate -> route.""" | |
| 79 | + extraction = extractor.extract(text) | |
| 80 | + outcome = IngestionOutcome(route=Route.NEEDS_REVIEW, extraction=extraction) | |
| 81 | + | |
| 82 | + events = _stamp_meta(extraction.events, extraction, ingested_at) | |
| 83 | + try: | |
| 84 | + outcome.document = AirDocument.model_validate({"events": events}) | |
| 85 | + except ValidationError as exc: | |
| 86 | + outcome.validation_errors = [ | |
| 87 | + f"{'.'.join(str(p) for p in err['loc'])}: {err['msg']}" | |
| 88 | + for err in exc.errors() | |
| 89 | + ] | |
| 90 | + outcome.reasons.append( | |
| 91 | + "extraction does not validate against the AIR schema" | |
| 92 | + ) | |
| 93 | + return outcome # schema-invalid ALWAYS goes to a human | |
| 94 | + | |
| 95 | + if extraction.confidence < threshold: | |
| 96 | + outcome.reasons.append( | |
| 97 | + f"confidence {extraction.confidence} below threshold {threshold}" | |
| 98 | + ) | |
| 99 | + return outcome | |
| 100 | + | |
| 101 | + outcome.route = Route.AUTO_APPROVED | |
| 102 | + return outcome | |
added
kernel/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
kernel/audit.py
+88 −0
@@ -0,0 +1,88 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : audit.py | |
| 6 | +# Description : Append-only, hash-chained audit log — every agent syscall is journaled here. | |
| 7 | +# ============================================================================= | |
| 8 | +"""The audit log: who did what, in order, tamper-evident. | |
| 9 | + | |
| 10 | +Every syscall an agent makes (successful OR failed) appends one record. | |
| 11 | +Records are hash-chained exactly like the ledger (SHA-256 over the canonical | |
| 12 | +JSON of the record plus the previous hash), so any after-the-fact edit, | |
| 13 | +deletion, or reordering breaks verification. Nothing is ever rewritten. | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +import hashlib | |
| 18 | +import json | |
| 19 | +from dataclasses import dataclass, field | |
| 20 | +from pathlib import Path | |
| 21 | +from typing import Any | |
| 22 | + | |
| 23 | +GENESIS_HASH = "0" * 64 | |
| 24 | + | |
| 25 | +_AUTHOR = "Simon-Pierre Boucher <contact@spboucher.ai>" | |
| 26 | + | |
| 27 | + | |
| 28 | +def _canonical(record: dict[str, Any]) -> str: | |
| 29 | + return json.dumps(record, sort_keys=True, separators=(",", ":"), default=str) | |
| 30 | + | |
| 31 | + | |
| 32 | +@dataclass | |
| 33 | +class AuditLog: | |
| 34 | + path: Path | None = None | |
| 35 | + _records: list[dict[str, Any]] = field(default_factory=list) | |
| 36 | + | |
| 37 | + def __post_init__(self) -> None: | |
| 38 | + if self.path is not None and self.path.exists(): | |
| 39 | + for raw in self.path.read_text(encoding="utf-8").splitlines(): | |
| 40 | + if raw.strip(): | |
| 41 | + self._records.append(json.loads(raw)) | |
| 42 | + | |
| 43 | + def append( | |
| 44 | + self, | |
| 45 | + actor: str, | |
| 46 | + syscall: str, | |
| 47 | + params: dict[str, Any], | |
| 48 | + status: str, # "ok" | "error" | |
| 49 | + detail: str = "", | |
| 50 | + at: str | None = None, | |
| 51 | + ) -> dict[str, Any]: | |
| 52 | + record: dict[str, Any] = { | |
| 53 | + "seq": len(self._records), | |
| 54 | + "actor": actor, | |
| 55 | + "syscall": syscall, | |
| 56 | + "params": params, | |
| 57 | + "status": status, | |
| 58 | + "detail": detail, | |
| 59 | + "at": at, | |
| 60 | + } | |
| 61 | + prev = self._records[-1]["hash"] if self._records else GENESIS_HASH | |
| 62 | + record["prev_hash"] = prev | |
| 63 | + record["hash"] = hashlib.sha256( | |
| 64 | + (prev + _canonical({k: v for k, v in record.items() if k != "hash"})) | |
| 65 | + .encode("utf-8") | |
| 66 | + ).hexdigest() | |
| 67 | + self._records.append(record) | |
| 68 | + if self.path is not None: | |
| 69 | + self.path.parent.mkdir(parents=True, exist_ok=True) | |
| 70 | + with self.path.open("a", encoding="utf-8") as f: | |
| 71 | + f.write(_canonical(record) + "\n") | |
| 72 | + return record | |
| 73 | + | |
| 74 | + def verify_chain(self) -> bool: | |
| 75 | + prev = GENESIS_HASH | |
| 76 | + for record in self._records: | |
| 77 | + expected = hashlib.sha256( | |
| 78 | + (prev + _canonical({k: v for k, v in record.items() if k != "hash"})) | |
| 79 | + .encode("utf-8") | |
| 80 | + ).hexdigest() | |
| 81 | + if record.get("hash") != expected or record.get("prev_hash") != prev: | |
| 82 | + return False | |
| 83 | + prev = str(record["hash"]) | |
| 84 | + return True | |
| 85 | + | |
| 86 | + @property | |
| 87 | + def records(self) -> list[dict[str, Any]]: | |
| 88 | + return list(self._records) | |
added
kernel/bank_formats.py
+222 −0
@@ -0,0 +1,222 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : bank_formats.py | |
| 6 | +# Description : Bank statement parsers — camt.053, MT940, CSV -> BankTransaction. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Bank statement parsers (specs: docs/research/bank-statement-formats.md, | |
| 9 | +verified 2026-08-05 — ISO 20022 camt.053 is the strategic standard, MT940 is | |
| 10 | +deprecated by SWIFT but still the most deployed corporate format; AIR parses | |
| 11 | +both into one model). | |
| 12 | + | |
| 13 | +Field semantics implemented from the research: | |
| 14 | +- camt.053: Ntry/Amt carries NO sign and a Ccy attribute; direction comes | |
| 15 | + exclusively from CdtDbtInd (CRDT = money in, DBIT = money out); date from | |
| 16 | + BookgDt/Dt (ValDt/Dt fallback); reference from NtryDtls/TxDtls/Refs/ | |
| 17 | + EndToEndId, else NtryRef; description from AddtlNtryInf, else RmtInf/Ustrd. | |
| 18 | +- MT940 :61: line — value date YYMMDD, optional entry date MMDD, D/C mark | |
| 19 | + (D, C, RD = reversal of debit -> money in, RC = reversal of credit -> money | |
| 20 | + out), optional funds code, UNSIGNED amount with COMMA decimal separator, | |
| 21 | + 1!a3!c transaction type, customer reference (16x, often NONREF), optional | |
| 22 | + //bank reference. Currency comes from :60F:/:62F:, never from :61:. | |
| 23 | + Integrity: opening balance +/- sum of lines must equal closing balance. | |
| 24 | + | |
| 25 | +Amounts are Decimal from string, never floats. | |
| 26 | +""" | |
| 27 | +from __future__ import annotations | |
| 28 | + | |
| 29 | +import csv | |
| 30 | +import re | |
| 31 | +import xml.etree.ElementTree as ET | |
| 32 | +from datetime import date, datetime | |
| 33 | +from decimal import Decimal, InvalidOperation | |
| 34 | +from pathlib import Path | |
| 35 | + | |
| 36 | +from kernel.reconcile import BankTransaction | |
| 37 | + | |
| 38 | + | |
| 39 | +class BankFormatError(ValueError): | |
| 40 | + pass | |
| 41 | + | |
| 42 | + | |
| 43 | +# --- camt.053 (ISO 20022 BankToCustomerStatement) -------------------------------- | |
| 44 | +def _local(tag: str) -> str: | |
| 45 | + """Strip the XML namespace: '{urn:...}Ntry' -> 'Ntry'.""" | |
| 46 | + return tag.rsplit("}", 1)[-1] | |
| 47 | + | |
| 48 | + | |
| 49 | +def _find(element: ET.Element, *path: str) -> ET.Element | None: | |
| 50 | + """Namespace-agnostic descent (camt versions differ only in namespace).""" | |
| 51 | + current: ET.Element | None = element | |
| 52 | + for name in path: | |
| 53 | + if current is None: | |
| 54 | + return None | |
| 55 | + current = next((c for c in current if _local(c.tag) == name), None) | |
| 56 | + return current | |
| 57 | + | |
| 58 | + | |
| 59 | +def _text(element: ET.Element | None) -> str: | |
| 60 | + return (element.text or "").strip() if element is not None else "" | |
| 61 | + | |
| 62 | + | |
| 63 | +def parse_camt053(path: Path) -> list[BankTransaction]: | |
| 64 | + root = ET.parse(path).getroot() | |
| 65 | + statement = _find(root, "BkToCstmrStmt", "Stmt") | |
| 66 | + if statement is None: | |
| 67 | + raise BankFormatError(f"{path}: no BkToCstmrStmt/Stmt — not a camt.053 file") | |
| 68 | + | |
| 69 | + transactions: list[BankTransaction] = [] | |
| 70 | + for entry in statement: | |
| 71 | + if _local(entry.tag) != "Ntry": | |
| 72 | + continue | |
| 73 | + amt = _find(entry, "Amt") | |
| 74 | + if amt is None: | |
| 75 | + raise BankFormatError(f"{path}: Ntry without Amt") | |
| 76 | + amount = Decimal(_text(amt)) # unsigned by spec | |
| 77 | + currency = amt.get("Ccy", "") | |
| 78 | + direction = _text(_find(entry, "CdtDbtInd")) | |
| 79 | + if direction == "DBIT": | |
| 80 | + amount = -amount | |
| 81 | + elif direction != "CRDT": | |
| 82 | + raise BankFormatError(f"{path}: CdtDbtInd must be CRDT|DBIT, got {direction!r}") | |
| 83 | + when = _text(_find(entry, "BookgDt", "Dt")) or _text(_find(entry, "ValDt", "Dt")) | |
| 84 | + if not when: | |
| 85 | + raise BankFormatError(f"{path}: Ntry without BookgDt/ValDt date") | |
| 86 | + reference = ( | |
| 87 | + _text(_find(entry, "NtryDtls", "TxDtls", "Refs", "EndToEndId")) | |
| 88 | + or _text(_find(entry, "NtryRef")) | |
| 89 | + ) | |
| 90 | + description = ( | |
| 91 | + _text(_find(entry, "AddtlNtryInf")) | |
| 92 | + or _text(_find(entry, "NtryDtls", "TxDtls", "RmtInf", "Ustrd")) | |
| 93 | + ) | |
| 94 | + transactions.append(BankTransaction( | |
| 95 | + date=date.fromisoformat(when[:10]), | |
| 96 | + amount=amount, currency=currency, | |
| 97 | + description=description, reference=reference, | |
| 98 | + )) | |
| 99 | + return transactions | |
| 100 | + | |
| 101 | + | |
| 102 | +# --- MT940 (SWIFT customer statement) ----------------------------------------------- | |
| 103 | +# :61: value-date(6) [entry-date(4)] D/C-mark [funds-code] amount type(4) refs | |
| 104 | +_LINE_61 = re.compile( | |
| 105 | + r"^:61:(?P<valdate>\d{6})(?P<entrydate>\d{4})?(?P<mark>RC|RD|C|D)" | |
| 106 | + r"(?P<funds>[A-Z])?(?P<amount>\d{1,15},\d*)" | |
| 107 | + r"(?P<type>[A-Z][A-Z0-9]{3})(?P<custref>[^/\n]{0,16})(?://(?P<bankref>.{0,16}))?" | |
| 108 | +) | |
| 109 | +_BALANCE = re.compile( | |
| 110 | + r"^:6[02][FM]:(?P<mark>[CD])(?P<date>\d{6})(?P<ccy>[A-Z]{3})(?P<amount>\d{1,15},\d*)" | |
| 111 | +) | |
| 112 | + | |
| 113 | + | |
| 114 | +def _yy_to_date(yymmdd: str) -> date: | |
| 115 | + year = int(yymmdd[:2]) | |
| 116 | + year += 1900 if year >= 70 else 2000 # SWIFT 2-digit pivot | |
| 117 | + return date(year, int(yymmdd[2:4]), int(yymmdd[4:6])) | |
| 118 | + | |
| 119 | + | |
| 120 | +def _comma_decimal(text: str) -> Decimal: | |
| 121 | + try: | |
| 122 | + return Decimal(text.replace(",", ".")) | |
| 123 | + except InvalidOperation as exc: | |
| 124 | + raise BankFormatError(f"bad MT940 amount {text!r}") from exc | |
| 125 | + | |
| 126 | + | |
| 127 | +def parse_mt940(path: Path) -> list[BankTransaction]: | |
| 128 | + lines = path.read_text(encoding="utf-8").splitlines() | |
| 129 | + currency = "" | |
| 130 | + opening: Decimal | None = None | |
| 131 | + closing: Decimal | None = None | |
| 132 | + transactions: list[BankTransaction] = [] | |
| 133 | + | |
| 134 | + i = 0 | |
| 135 | + while i < len(lines): | |
| 136 | + line = lines[i].strip() | |
| 137 | + if line.startswith((":60F:", ":60M:", ":62F:", ":62M:")): | |
| 138 | + match = _BALANCE.match(line) | |
| 139 | + if not match: | |
| 140 | + raise BankFormatError(f"{path}: malformed balance line {line!r}") | |
| 141 | + currency = match["ccy"] | |
| 142 | + balance = _comma_decimal(match["amount"]) | |
| 143 | + if match["mark"] == "D": | |
| 144 | + balance = -balance | |
| 145 | + if line.startswith(":60"): | |
| 146 | + opening = balance | |
| 147 | + else: | |
| 148 | + closing = balance | |
| 149 | + elif line.startswith(":61:"): | |
| 150 | + match = _LINE_61.match(line) | |
| 151 | + if not match: | |
| 152 | + raise BankFormatError(f"{path}: malformed :61: line {line!r}") | |
| 153 | + amount = _comma_decimal(match["amount"]) | |
| 154 | + # D = money out; C = money in; RD reverses a debit (in); RC (out) | |
| 155 | + if match["mark"] in ("D", "RC"): | |
| 156 | + amount = -amount | |
| 157 | + reference = match["custref"].strip() | |
| 158 | + if reference.upper() == "NONREF": | |
| 159 | + reference = match["bankref"] or "" | |
| 160 | + description = "" | |
| 161 | + if i + 1 < len(lines) and lines[i + 1].startswith(":86:"): | |
| 162 | + description = lines[i + 1][4:].strip() | |
| 163 | + i += 1 | |
| 164 | + transactions.append(BankTransaction( | |
| 165 | + date=_yy_to_date(match["valdate"]), | |
| 166 | + amount=amount, currency=currency, | |
| 167 | + description=description, reference=reference, | |
| 168 | + )) | |
| 169 | + i += 1 | |
| 170 | + | |
| 171 | + # research §2.2: :60F: +/- sum(:61:) must equal :62F: | |
| 172 | + if opening is not None and closing is not None: | |
| 173 | + total = opening + sum(t.amount for t in transactions) | |
| 174 | + if total != closing: | |
| 175 | + raise BankFormatError( | |
| 176 | + f"{path}: statement does not balance — opening {opening} + " | |
| 177 | + f"movements = {total}, but closing balance is {closing}" | |
| 178 | + ) | |
| 179 | + if not currency: | |
| 180 | + raise BankFormatError(f"{path}: no :60F:/:62F: balance line — currency unknown") | |
| 181 | + return transactions | |
| 182 | + | |
| 183 | + | |
| 184 | +# --- generic CSV ---------------------------------------------------------------------- | |
| 185 | +def parse_bank_csv(path: Path) -> list[BankTransaction]: | |
| 186 | + """Columns: date, amount (signed, money in positive), currency, | |
| 187 | + description, reference.""" | |
| 188 | + transactions: list[BankTransaction] = [] | |
| 189 | + with path.open(encoding="utf-8") as f: | |
| 190 | + for row in csv.DictReader(f): | |
| 191 | + try: | |
| 192 | + transactions.append(BankTransaction( | |
| 193 | + date=datetime.strptime(row["date"].strip(), "%Y-%m-%d").date(), | |
| 194 | + amount=Decimal(row["amount"].strip()), | |
| 195 | + currency=row["currency"].strip(), | |
| 196 | + description=(row.get("description") or "").strip(), | |
| 197 | + reference=(row.get("reference") or "").strip(), | |
| 198 | + )) | |
| 199 | + except (KeyError, ValueError, InvalidOperation) as exc: | |
| 200 | + raise BankFormatError(f"{path}: bad CSV row {row!r}: {exc}") from exc | |
| 201 | + return transactions | |
| 202 | + | |
| 203 | + | |
| 204 | +# --- dispatcher -------------------------------------------------------------------------- | |
| 205 | +def parse_statement(path: Path) -> list[BankTransaction]: | |
| 206 | + """Detect the format by suffix, then by content sniffing.""" | |
| 207 | + suffix = path.suffix.lower() | |
| 208 | + if suffix == ".xml": | |
| 209 | + return parse_camt053(path) | |
| 210 | + if suffix in (".mt940", ".sta", ".940"): | |
| 211 | + return parse_mt940(path) | |
| 212 | + if suffix == ".csv": | |
| 213 | + return parse_bank_csv(path) | |
| 214 | + head = path.read_text(encoding="utf-8", errors="replace")[:2000] | |
| 215 | + if "<Document" in head or head.lstrip().startswith("<?xml"): | |
| 216 | + return parse_camt053(path) | |
| 217 | + if ":61:" in head: | |
| 218 | + return parse_mt940(path) | |
| 219 | + raise BankFormatError( | |
| 220 | + f"{path}: unrecognized statement format (expected camt.053 XML, " | |
| 221 | + "MT940, or CSV with date,amount,currency,description,reference)" | |
| 222 | + ) | |
added
kernel/ledger.py
+203 −0
@@ -0,0 +1,203 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : ledger.py | |
| 6 | +# Description : Native ledger — append-only, hash-chained event store with balance projections. | |
| 7 | +# ============================================================================= | |
| 8 | +"""AIR's own ledger: the standalone mode. | |
| 9 | + | |
| 10 | +AIR does not require any third-party system. Compiled journals can be posted | |
| 11 | +to this native ledger: an append-only, hash-chained JSONL store (each record | |
| 12 | +carries the SHA-256 of the previous record over canonical JSON — tamper | |
| 13 | +evidence, per docs/research/ledger-engines.md). Balances, trial balance and | |
| 14 | +financial statements are projections computed from the store. | |
| 15 | + | |
| 16 | +Corrections never mutate: reversals append contra entries. | |
| 17 | +""" | |
| 18 | +from __future__ import annotations | |
| 19 | + | |
| 20 | +import hashlib | |
| 21 | +import json | |
| 22 | +from collections import defaultdict | |
| 23 | +from dataclasses import dataclass, field | |
| 24 | +from decimal import Decimal | |
| 25 | +from pathlib import Path | |
| 26 | + | |
| 27 | +from core.journal import ( | |
| 28 | + Account, | |
| 29 | + AccountType, | |
| 30 | + CompiledJournal, | |
| 31 | + JournalEntry, | |
| 32 | + JournalLine, | |
| 33 | + Side, | |
| 34 | +) | |
| 35 | +from core.money import Money | |
| 36 | + | |
| 37 | +GENESIS_HASH = "0" * 64 | |
| 38 | + | |
| 39 | + | |
| 40 | +def _canonical(record: dict[str, object]) -> str: | |
| 41 | + return json.dumps(record, sort_keys=True, separators=(",", ":"), default=str) | |
| 42 | + | |
| 43 | + | |
| 44 | +def _entry_record(entry: JournalEntry, idempotency_key: str) -> dict[str, object]: | |
| 45 | + return { | |
| 46 | + "id": entry.id, | |
| 47 | + "date": entry.date.isoformat(), | |
| 48 | + "description": entry.description, | |
| 49 | + "source_event_id": entry.source_event_id, | |
| 50 | + "policy_set": entry.policy_set, | |
| 51 | + "policy_version": entry.policy_version, | |
| 52 | + "reverses": entry.reverses, | |
| 53 | + "idempotency_key": idempotency_key, | |
| 54 | + "lines": [ | |
| 55 | + { | |
| 56 | + "account_code": line.account.code, | |
| 57 | + "account_name": line.account.name, | |
| 58 | + "account_type": line.account.type.value, | |
| 59 | + "side": line.side.value, | |
| 60 | + "amount": str(line.amount.amount), | |
| 61 | + "currency": line.amount.currency, | |
| 62 | + "memo": line.memo, | |
| 63 | + "provenance_id": line.provenance_id, | |
| 64 | + } | |
| 65 | + for line in entry.lines | |
| 66 | + ], | |
| 67 | + } | |
| 68 | + | |
| 69 | + | |
| 70 | +@dataclass | |
| 71 | +class Ledger: | |
| 72 | + """Append-only ledger with optional JSONL persistence.""" | |
| 73 | + | |
| 74 | + path: Path | None = None | |
| 75 | + _entries: list[JournalEntry] = field(default_factory=list) | |
| 76 | + _records: list[dict[str, object]] = field(default_factory=list) | |
| 77 | + _keys: set[str] = field(default_factory=set) | |
| 78 | + | |
| 79 | + def __post_init__(self) -> None: | |
| 80 | + if self.path is not None and self.path.exists(): | |
| 81 | + for raw in self.path.read_text(encoding="utf-8").splitlines(): | |
| 82 | + if raw.strip(): | |
| 83 | + self._load_record(json.loads(raw)) | |
| 84 | + | |
| 85 | + # -- append ------------------------------------------------------------------ | |
| 86 | + def post_journal(self, journal: CompiledJournal, idempotency_key: str) -> int: | |
| 87 | + """Append all entries of a compiled journal. Idempotent per key. | |
| 88 | + | |
| 89 | + Returns the number of entries actually appended (0 on a replay). | |
| 90 | + """ | |
| 91 | + if idempotency_key in self._keys: | |
| 92 | + return 0 | |
| 93 | + appended = 0 | |
| 94 | + for entry in journal.entries: | |
| 95 | + self._append(entry, idempotency_key) | |
| 96 | + appended += 1 | |
| 97 | + self._keys.add(idempotency_key) | |
| 98 | + return appended | |
| 99 | + | |
| 100 | + def reverse_entry(self, entry_id: str, idempotency_key: str) -> JournalEntry: | |
| 101 | + """Append a contra entry that reverses a posted entry. Never deletes.""" | |
| 102 | + original = next((e for e in self._entries if e.id == entry_id), None) | |
| 103 | + if original is None: | |
| 104 | + raise KeyError(f"ledger has no entry '{entry_id}'") | |
| 105 | + contra = JournalEntry( | |
| 106 | + id=f"rev_{original.id}", | |
| 107 | + date=original.date, | |
| 108 | + description=f"REVERSAL: {original.description}", | |
| 109 | + lines=tuple( | |
| 110 | + JournalLine( | |
| 111 | + account=line.account, | |
| 112 | + side=Side.CREDIT if line.side is Side.DEBIT else Side.DEBIT, | |
| 113 | + amount=line.amount, | |
| 114 | + memo=f"reversal of {original.id}: {line.memo}", | |
| 115 | + provenance_id=line.provenance_id, | |
| 116 | + ) | |
| 117 | + for line in original.lines | |
| 118 | + ), | |
| 119 | + source_event_id=original.source_event_id, | |
| 120 | + policy_set=original.policy_set, | |
| 121 | + policy_version=original.policy_version, | |
| 122 | + reverses=original.id, | |
| 123 | + ) | |
| 124 | + self._append(contra, idempotency_key) | |
| 125 | + return contra | |
| 126 | + | |
| 127 | + def _append(self, entry: JournalEntry, idempotency_key: str) -> None: | |
| 128 | + record = _entry_record(entry, idempotency_key) | |
| 129 | + prev = self._records[-1]["hash"] if self._records else GENESIS_HASH | |
| 130 | + record["prev_hash"] = prev | |
| 131 | + record["hash"] = hashlib.sha256( | |
| 132 | + (str(prev) + _canonical({k: v for k, v in record.items() if k != "hash"})) | |
| 133 | + .encode("utf-8") | |
| 134 | + ).hexdigest() | |
| 135 | + self._records.append(record) | |
| 136 | + self._entries.append(entry) | |
| 137 | + if self.path is not None: | |
| 138 | + self.path.parent.mkdir(parents=True, exist_ok=True) | |
| 139 | + with self.path.open("a", encoding="utf-8") as f: | |
| 140 | + f.write(_canonical(record) + "\n") | |
| 141 | + | |
| 142 | + def _load_record(self, record: dict[str, object]) -> None: | |
| 143 | + lines = tuple( | |
| 144 | + JournalLine( | |
| 145 | + account=Account( | |
| 146 | + code=str(l["account_code"]), | |
| 147 | + name=str(l["account_name"]), | |
| 148 | + type=AccountType(str(l["account_type"])), | |
| 149 | + ), | |
| 150 | + side=Side(str(l["side"])), | |
| 151 | + amount=Money(Decimal(str(l["amount"])), str(l["currency"])), | |
| 152 | + memo=str(l.get("memo", "")), | |
| 153 | + provenance_id=(str(l["provenance_id"]) if l.get("provenance_id") else None), | |
| 154 | + ) | |
| 155 | + for l in record["lines"] # type: ignore[union-attr] | |
| 156 | + ) | |
| 157 | + from datetime import date as _date | |
| 158 | + entry = JournalEntry( | |
| 159 | + id=str(record["id"]), | |
| 160 | + date=_date.fromisoformat(str(record["date"])), | |
| 161 | + description=str(record["description"]), | |
| 162 | + lines=lines, | |
| 163 | + source_event_id=str(record["source_event_id"]), | |
| 164 | + policy_set=str(record.get("policy_set", "")), | |
| 165 | + policy_version=str(record.get("policy_version", "")), | |
| 166 | + reverses=(str(record["reverses"]) if record.get("reverses") else None), | |
| 167 | + ) | |
| 168 | + self._records.append(record) | |
| 169 | + self._entries.append(entry) | |
| 170 | + self._keys.add(str(record.get("idempotency_key", ""))) | |
| 171 | + | |
| 172 | + # -- integrity & projections --------------------------------------------------- | |
| 173 | + def verify_chain(self) -> bool: | |
| 174 | + prev = GENESIS_HASH | |
| 175 | + for record in self._records: | |
| 176 | + expected = hashlib.sha256( | |
| 177 | + (prev + _canonical({k: v for k, v in record.items() if k != "hash"})) | |
| 178 | + .encode("utf-8") | |
| 179 | + ).hexdigest() | |
| 180 | + if record.get("hash") != expected or record.get("prev_hash") != prev: | |
| 181 | + return False | |
| 182 | + prev = str(record["hash"]) | |
| 183 | + return True | |
| 184 | + | |
| 185 | + @property | |
| 186 | + def entries(self) -> list[JournalEntry]: | |
| 187 | + return list(self._entries) | |
| 188 | + | |
| 189 | + def accounts(self) -> dict[str, Account]: | |
| 190 | + out: dict[str, Account] = {} | |
| 191 | + for entry in self._entries: | |
| 192 | + for line in entry.lines: | |
| 193 | + out[line.account.code] = line.account | |
| 194 | + return dict(sorted(out.items())) | |
| 195 | + | |
| 196 | + def balances(self) -> dict[str, dict[str, Decimal]]: | |
| 197 | + """Normal-side balance per account code per currency.""" | |
| 198 | + out: dict[str, dict[str, Decimal]] = defaultdict(lambda: defaultdict(Decimal)) | |
| 199 | + for entry in self._entries: | |
| 200 | + for line in entry.lines: | |
| 201 | + sign = 1 if line.side is line.account.type.normal_side else -1 | |
| 202 | + out[line.account.code][line.amount.currency] += sign * line.amount.amount | |
| 203 | + return {code: dict(per) for code, per in sorted(out.items())} | |
added
kernel/reconcile.py
+139 −0
@@ -0,0 +1,139 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : reconcile.py | |
| 6 | +# Description : Bank reconciliation — match statement lines against ledger cash movements. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Bank reconciliation (Phase 6). | |
| 9 | + | |
| 10 | +A bank statement (camt.053, MT940, or CSV — parsers in kernel/bank_formats.py) | |
| 11 | +is reduced to a list of BankTransaction records, then matched against the | |
| 12 | +ledger's cash-account movements: | |
| 13 | + | |
| 14 | +- signed amounts: positive = money into our account (bank credit = our debit); | |
| 15 | +- a match is exact on (signed amount, currency) within a configurable | |
| 16 | + date-tolerance window (default 3 days); | |
| 17 | +- greedy one-to-one matching, earliest ledger candidate first; | |
| 18 | +- everything unmatched — on either side — is reported, never dropped. | |
| 19 | + | |
| 20 | +Amounts are Decimal end to end; floats are rejected at the parser boundary. | |
| 21 | +""" | |
| 22 | +from __future__ import annotations | |
| 23 | + | |
| 24 | +from dataclasses import dataclass, field | |
| 25 | +from datetime import date | |
| 26 | +from decimal import Decimal | |
| 27 | + | |
| 28 | +from core.journal import Side | |
| 29 | +from kernel.ledger import Ledger | |
| 30 | + | |
| 31 | + | |
| 32 | +@dataclass(frozen=True, slots=True) | |
| 33 | +class BankTransaction: | |
| 34 | + """One statement line, format-agnostic. Positive amount = money in.""" | |
| 35 | + | |
| 36 | + date: date | |
| 37 | + amount: Decimal | |
| 38 | + currency: str | |
| 39 | + description: str = "" | |
| 40 | + reference: str = "" | |
| 41 | + | |
| 42 | + | |
| 43 | +@dataclass(frozen=True, slots=True) | |
| 44 | +class LedgerMovement: | |
| 45 | + """One cash-account line in the books. Positive amount = money in (debit).""" | |
| 46 | + | |
| 47 | + date: date | |
| 48 | + amount: Decimal | |
| 49 | + currency: str | |
| 50 | + entry_id: str | |
| 51 | + memo: str = "" | |
| 52 | + | |
| 53 | + | |
| 54 | +@dataclass | |
| 55 | +class ReconciliationResult: | |
| 56 | + matched: list[tuple[BankTransaction, LedgerMovement]] = field(default_factory=list) | |
| 57 | + unmatched_bank: list[BankTransaction] = field(default_factory=list) | |
| 58 | + unmatched_ledger: list[LedgerMovement] = field(default_factory=list) | |
| 59 | + | |
| 60 | + @property | |
| 61 | + def is_clean(self) -> bool: | |
| 62 | + return not self.unmatched_bank and not self.unmatched_ledger | |
| 63 | + | |
| 64 | + def summary(self) -> dict[str, int]: | |
| 65 | + return { | |
| 66 | + "matched": len(self.matched), | |
| 67 | + "unmatched_bank": len(self.unmatched_bank), | |
| 68 | + "unmatched_ledger": len(self.unmatched_ledger), | |
| 69 | + } | |
| 70 | + | |
| 71 | + def render(self) -> str: | |
| 72 | + lines = ["Bank Reconciliation", "-" * 72] | |
| 73 | + for txn, movement in self.matched: | |
| 74 | + lines.append( | |
| 75 | + f"MATCH {txn.date} {txn.amount:>12} {txn.currency} " | |
| 76 | + f"bank:{txn.reference or txn.description[:24]:<24} " | |
| 77 | + f"ledger:{movement.entry_id}" | |
| 78 | + ) | |
| 79 | + for txn in self.unmatched_bank: | |
| 80 | + lines.append( | |
| 81 | + f"BANK? {txn.date} {txn.amount:>12} {txn.currency} " | |
| 82 | + f"{txn.description[:40]} <- in the bank, not in the books" | |
| 83 | + ) | |
| 84 | + for movement in self.unmatched_ledger: | |
| 85 | + lines.append( | |
| 86 | + f"BOOK? {movement.date} {movement.amount:>12} " | |
| 87 | + f"{movement.currency} {movement.entry_id} " | |
| 88 | + f"<- in the books, not at the bank" | |
| 89 | + ) | |
| 90 | + lines.append("-" * 72) | |
| 91 | + s = self.summary() | |
| 92 | + lines.append( | |
| 93 | + f"{s['matched']} matched, {s['unmatched_bank']} unexplained bank, " | |
| 94 | + f"{s['unmatched_ledger']} outstanding ledger — " | |
| 95 | + + ("CLEAN" if self.is_clean else "DIFFERENCES FOUND") | |
| 96 | + ) | |
| 97 | + return "\n".join(lines) | |
| 98 | + | |
| 99 | + | |
| 100 | +def cash_movements(ledger: Ledger, cash_account: str = "1000") -> list[LedgerMovement]: | |
| 101 | + """Every line touching the cash account, debit-positive (money in).""" | |
| 102 | + movements: list[LedgerMovement] = [] | |
| 103 | + for entry in ledger.entries: | |
| 104 | + for line in entry.lines: | |
| 105 | + if line.account.code != cash_account: | |
| 106 | + continue | |
| 107 | + signed = (line.amount.amount if line.side is Side.DEBIT | |
| 108 | + else -line.amount.amount) | |
| 109 | + movements.append(LedgerMovement( | |
| 110 | + date=entry.date, amount=signed, | |
| 111 | + currency=line.amount.currency, | |
| 112 | + entry_id=entry.id, memo=line.memo, | |
| 113 | + )) | |
| 114 | + return movements | |
| 115 | + | |
| 116 | + | |
| 117 | +def reconcile( | |
| 118 | + transactions: list[BankTransaction], | |
| 119 | + movements: list[LedgerMovement], | |
| 120 | + tolerance_days: int = 3, | |
| 121 | +) -> ReconciliationResult: | |
| 122 | + """Greedy one-to-one matching on (signed amount, currency) within the | |
| 123 | + date window; among candidates, the closest date wins.""" | |
| 124 | + result = ReconciliationResult() | |
| 125 | + remaining = list(movements) | |
| 126 | + for txn in sorted(transactions, key=lambda t: t.date): | |
| 127 | + candidates = [ | |
| 128 | + m for m in remaining | |
| 129 | + if m.amount == txn.amount and m.currency == txn.currency | |
| 130 | + and abs((m.date - txn.date).days) <= tolerance_days | |
| 131 | + ] | |
| 132 | + if candidates: | |
| 133 | + best = min(candidates, key=lambda m: abs((m.date - txn.date).days)) | |
| 134 | + remaining.remove(best) | |
| 135 | + result.matched.append((txn, best)) | |
| 136 | + else: | |
| 137 | + result.unmatched_bank.append(txn) | |
| 138 | + result.unmatched_ledger = remaining | |
| 139 | + return result | |
added
kernel/reporting.py
+187 −0
@@ -0,0 +1,187 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : reporting.py | |
| 6 | +# Description : Financial statements from the native ledger — multi-format (text, markdown, csv, json). | |
| 7 | +# ============================================================================= | |
| 8 | +"""Reporting: AIR standalone mode needs no third-party system. | |
| 9 | + | |
| 10 | +All statements are pure projections over the ledger: | |
| 11 | +- general ledger (every line, by account) | |
| 12 | +- trial balance (debit/credit totals per account) | |
| 13 | +- income statement (revenue - expenses = net income) | |
| 14 | +- balance sheet (assets = liabilities + equity + net income) | |
| 15 | + | |
| 16 | +Each report renders to: "text", "markdown", "csv", "json". | |
| 17 | +Amounts stay Decimal end-to-end and serialize as strings. | |
| 18 | +""" | |
| 19 | +from __future__ import annotations | |
| 20 | + | |
| 21 | +import csv | |
| 22 | +import io | |
| 23 | +import json | |
| 24 | +from collections import defaultdict | |
| 25 | +from decimal import Decimal | |
| 26 | + | |
| 27 | +from core.journal import AccountType | |
| 28 | +from kernel.ledger import Ledger | |
| 29 | + | |
| 30 | +FORMATS = ("text", "markdown", "csv", "json") | |
| 31 | + | |
| 32 | +Row = list[str] | |
| 33 | + | |
| 34 | + | |
| 35 | +def _render(title: str, headers: list[str], rows: list[Row], fmt: str) -> str: | |
| 36 | + if fmt == "json": | |
| 37 | + return json.dumps( | |
| 38 | + {"_author": "Simon-Pierre Boucher <contact@spboucher.ai>", | |
| 39 | + "report": title, | |
| 40 | + "rows": [dict(zip(headers, r)) for r in rows]}, | |
| 41 | + indent=2, | |
| 42 | + ) | |
| 43 | + if fmt == "csv": | |
| 44 | + buf = io.StringIO() | |
| 45 | + writer = csv.writer(buf) | |
| 46 | + writer.writerow(headers) | |
| 47 | + writer.writerows(rows) | |
| 48 | + return buf.getvalue() | |
| 49 | + if fmt == "markdown": | |
| 50 | + out = [f"## {title}", "", "| " + " | ".join(headers) + " |", | |
| 51 | + "|" + "|".join("---" for _ in headers) + "|"] | |
| 52 | + out += ["| " + " | ".join(r) + " |" for r in rows] | |
| 53 | + return "\n".join(out) + "\n" | |
| 54 | + # text | |
| 55 | + widths = [max(len(h), *(len(r[i]) for r in rows)) if rows else len(h) | |
| 56 | + for i, h in enumerate(headers)] | |
| 57 | + line = " ".join(h.ljust(widths[i]) for i, h in enumerate(headers)) | |
| 58 | + sep = "-" * len(line) | |
| 59 | + body = [ | |
| 60 | + " ".join(r[i].ljust(widths[i]) for i in range(len(headers))) for r in rows | |
| 61 | + ] | |
| 62 | + return "\n".join([title, sep, line, sep, *body, sep]) + "\n" | |
| 63 | + | |
| 64 | + | |
| 65 | +def general_ledger(ledger: Ledger, fmt: str = "text") -> str: | |
| 66 | + headers = ["account", "name", "date", "entry", "side", "amount", "currency", "memo"] | |
| 67 | + rows: list[Row] = [] | |
| 68 | + accounts = ledger.accounts() | |
| 69 | + for code, account in accounts.items(): | |
| 70 | + for entry in ledger.entries: | |
| 71 | + for line in entry.lines: | |
| 72 | + if line.account.code != code: | |
| 73 | + continue | |
| 74 | + rows.append([ | |
| 75 | + code, account.name, entry.date.isoformat(), entry.id, | |
| 76 | + line.side.value, str(line.amount.amount), | |
| 77 | + line.amount.currency, line.memo, | |
| 78 | + ]) | |
| 79 | + return _render("General Ledger", headers, rows, fmt) | |
| 80 | + | |
| 81 | + | |
| 82 | +def trial_balance(ledger: Ledger, fmt: str = "text") -> str: | |
| 83 | + headers = ["account", "name", "type", "currency", "debit", "credit"] | |
| 84 | + debit: dict[tuple[str, str], Decimal] = defaultdict(Decimal) | |
| 85 | + credit: dict[tuple[str, str], Decimal] = defaultdict(Decimal) | |
| 86 | + for entry in ledger.entries: | |
| 87 | + for line in entry.lines: | |
| 88 | + key = (line.account.code, line.amount.currency) | |
| 89 | + if line.side.value == "debit": | |
| 90 | + debit[key] += line.amount.amount | |
| 91 | + else: | |
| 92 | + credit[key] += line.amount.amount | |
| 93 | + accounts = ledger.accounts() | |
| 94 | + rows: list[Row] = [] | |
| 95 | + total_d = defaultdict(Decimal) | |
| 96 | + total_c = defaultdict(Decimal) | |
| 97 | + for key in sorted(set(debit) | set(credit)): | |
| 98 | + code, ccy = key | |
| 99 | + account = accounts[code] | |
| 100 | + d, c = debit[key], credit[key] | |
| 101 | + # present net movement on the account's normal side | |
| 102 | + net = d - c | |
| 103 | + d_show = net if net > 0 else Decimal(0) | |
| 104 | + c_show = -net if net < 0 else Decimal(0) | |
| 105 | + total_d[ccy] += d_show | |
| 106 | + total_c[ccy] += c_show | |
| 107 | + rows.append([code, account.name, account.type.value, ccy, | |
| 108 | + str(d_show), str(c_show)]) | |
| 109 | + for ccy in sorted(total_d): | |
| 110 | + rows.append(["TOTAL", "", "", ccy, str(total_d[ccy]), str(total_c[ccy])]) | |
| 111 | + return _render("Trial Balance", headers, rows, fmt) | |
| 112 | + | |
| 113 | + | |
| 114 | +def _type_totals(ledger: Ledger) -> dict[AccountType, dict[str, Decimal]]: | |
| 115 | + totals: dict[AccountType, dict[str, Decimal]] = { | |
| 116 | + t: defaultdict(Decimal) for t in AccountType | |
| 117 | + } | |
| 118 | + balances = ledger.balances() | |
| 119 | + accounts = ledger.accounts() | |
| 120 | + for code, per_ccy in balances.items(): | |
| 121 | + for ccy, bal in per_ccy.items(): | |
| 122 | + totals[accounts[code].type][ccy] += bal | |
| 123 | + return totals | |
| 124 | + | |
| 125 | + | |
| 126 | +def income_statement(ledger: Ledger, fmt: str = "text") -> str: | |
| 127 | + headers = ["section", "account", "currency", "amount"] | |
| 128 | + accounts = ledger.accounts() | |
| 129 | + balances = ledger.balances() | |
| 130 | + rows: list[Row] = [] | |
| 131 | + net = defaultdict(Decimal) | |
| 132 | + for wanted, section, sign in ( | |
| 133 | + (AccountType.REVENUE, "Revenue", 1), | |
| 134 | + (AccountType.EXPENSE, "Expenses", -1), | |
| 135 | + ): | |
| 136 | + for code, per_ccy in balances.items(): | |
| 137 | + if accounts[code].type is not wanted: | |
| 138 | + continue | |
| 139 | + for ccy, bal in per_ccy.items(): | |
| 140 | + if bal == 0: | |
| 141 | + continue | |
| 142 | + rows.append([section, f"{code} {accounts[code].name}", ccy, str(bal)]) | |
| 143 | + net[ccy] += sign * bal | |
| 144 | + for ccy in sorted(net): | |
| 145 | + rows.append(["NET INCOME", "", ccy, str(net[ccy])]) | |
| 146 | + return _render("Income Statement", headers, rows, fmt) | |
| 147 | + | |
| 148 | + | |
| 149 | +def balance_sheet(ledger: Ledger, fmt: str = "text") -> str: | |
| 150 | + headers = ["section", "account", "currency", "amount"] | |
| 151 | + accounts = ledger.accounts() | |
| 152 | + balances = ledger.balances() | |
| 153 | + totals = _type_totals(ledger) | |
| 154 | + rows: list[Row] = [] | |
| 155 | + for wanted, section in ( | |
| 156 | + (AccountType.ASSET, "Assets"), | |
| 157 | + (AccountType.LIABILITY, "Liabilities"), | |
| 158 | + (AccountType.EQUITY, "Equity"), | |
| 159 | + ): | |
| 160 | + for code, per_ccy in balances.items(): | |
| 161 | + if accounts[code].type is not wanted: | |
| 162 | + continue | |
| 163 | + for ccy, bal in per_ccy.items(): | |
| 164 | + if bal == 0: | |
| 165 | + continue | |
| 166 | + rows.append([section, f"{code} {accounts[code].name}", ccy, str(bal)]) | |
| 167 | + currencies = sorted({ | |
| 168 | + ccy for per in totals.values() for ccy in per | |
| 169 | + }) | |
| 170 | + for ccy in currencies: | |
| 171 | + net_income = totals[AccountType.REVENUE][ccy] - totals[AccountType.EXPENSE][ccy] | |
| 172 | + rows.append(["Equity", "Net income (current period)", ccy, str(net_income)]) | |
| 173 | + rows.append(["TOTAL ASSETS", "", ccy, str(totals[AccountType.ASSET][ccy])]) | |
| 174 | + rows.append([ | |
| 175 | + "TOTAL LIAB.+EQUITY", "", ccy, | |
| 176 | + str(totals[AccountType.LIABILITY][ccy] | |
| 177 | + + totals[AccountType.EQUITY][ccy] + net_income), | |
| 178 | + ]) | |
| 179 | + return _render("Balance Sheet", headers, rows, fmt) | |
| 180 | + | |
| 181 | + | |
| 182 | +REPORTS = { | |
| 183 | + "general-ledger": general_ledger, | |
| 184 | + "trial-balance": trial_balance, | |
| 185 | + "income-statement": income_statement, | |
| 186 | + "balance-sheet": balance_sheet, | |
| 187 | +} | |
added
kernel/workspace.py
+126 −0
@@ -0,0 +1,126 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : workspace.py | |
| 6 | +# Description : AIR home — a managed local data directory: ledger, archived documents, exports. | |
| 7 | +# ============================================================================= | |
| 8 | +"""The AIR home: managed local data for standalone users. | |
| 9 | + | |
| 10 | +Users without any third-party system keep everything in one directory: | |
| 11 | + | |
| 12 | + <home>/ | |
| 13 | + ├── meta.json # company name, default policy set, created date | |
| 14 | + ├── ledger.jsonl # the hash-chained book of record | |
| 15 | + ├── documents/ # every AIR document ever posted (content-addressed) | |
| 16 | + └── exports/ # generated CSV/JSON exports and reports | |
| 17 | + | |
| 18 | +Documents are archived content-addressed (sha256 prefix + original name), so | |
| 19 | +the exact input of every posting can be replayed later — pairing with the | |
| 20 | +ledger's idempotency keys and the compiler's determinism, the entire book of | |
| 21 | +record is reproducible from this directory alone. | |
| 22 | +""" | |
| 23 | +from __future__ import annotations | |
| 24 | + | |
| 25 | +import hashlib | |
| 26 | +import json | |
| 27 | +from dataclasses import dataclass | |
| 28 | +from pathlib import Path | |
| 29 | + | |
| 30 | +from kernel.ledger import Ledger | |
| 31 | + | |
| 32 | +META_FILE = "meta.json" | |
| 33 | + | |
| 34 | + | |
| 35 | +class WorkspaceError(RuntimeError): | |
| 36 | + pass | |
| 37 | + | |
| 38 | + | |
| 39 | +@dataclass | |
| 40 | +class Workspace: | |
| 41 | + root: Path | |
| 42 | + | |
| 43 | + # -- lifecycle ----------------------------------------------------------- | |
| 44 | + @classmethod | |
| 45 | + def init( | |
| 46 | + cls, | |
| 47 | + root: str | Path, | |
| 48 | + *, | |
| 49 | + name: str = "my-books", | |
| 50 | + policies: str | None = None, | |
| 51 | + created: str | None = None, | |
| 52 | + ) -> "Workspace": | |
| 53 | + root = Path(root) | |
| 54 | + if (root / META_FILE).exists(): | |
| 55 | + raise WorkspaceError(f"workspace already initialized: {root}") | |
| 56 | + (root / "documents").mkdir(parents=True, exist_ok=True) | |
| 57 | + (root / "exports").mkdir(parents=True, exist_ok=True) | |
| 58 | + meta = { | |
| 59 | + "_author": "Simon-Pierre Boucher <contact@spboucher.ai>", | |
| 60 | + "air_home_version": "1", | |
| 61 | + "name": name, | |
| 62 | + "policies": policies, | |
| 63 | + "created": created, | |
| 64 | + } | |
| 65 | + (root / META_FILE).write_text(json.dumps(meta, indent=2), encoding="utf-8") | |
| 66 | + (root / "ledger.jsonl").touch() | |
| 67 | + return cls(root=root) | |
| 68 | + | |
| 69 | + @classmethod | |
| 70 | + def open(cls, root: str | Path) -> "Workspace": | |
| 71 | + root = Path(root) | |
| 72 | + if not (root / META_FILE).exists(): | |
| 73 | + raise WorkspaceError( | |
| 74 | + f"no AIR home at {root} — run: air init --home {root} " | |
| 75 | + "--policies <policy-set.yaml>" | |
| 76 | + ) | |
| 77 | + return cls(root=root) | |
| 78 | + | |
| 79 | + # -- accessors ------------------------------------------------------------- | |
| 80 | + @property | |
| 81 | + def meta(self) -> dict: | |
| 82 | + return json.loads((self.root / META_FILE).read_text(encoding="utf-8")) | |
| 83 | + | |
| 84 | + @property | |
| 85 | + def ledger_path(self) -> Path: | |
| 86 | + return self.root / "ledger.jsonl" | |
| 87 | + | |
| 88 | + @property | |
| 89 | + def exports_dir(self) -> Path: | |
| 90 | + return self.root / "exports" | |
| 91 | + | |
| 92 | + def ledger(self) -> Ledger: | |
| 93 | + return Ledger(path=self.ledger_path) | |
| 94 | + | |
| 95 | + def default_policies(self) -> str | None: | |
| 96 | + return self.meta.get("policies") | |
| 97 | + | |
| 98 | + # -- data management ----------------------------------------------------------- | |
| 99 | + def archive_document(self, path: str | Path) -> Path: | |
| 100 | + """Store a posted AIR document content-addressed; idempotent.""" | |
| 101 | + path = Path(path) | |
| 102 | + content = path.read_bytes() | |
| 103 | + digest = hashlib.sha256(content).hexdigest()[:12] | |
| 104 | + target = self.root / "documents" / f"{digest}_{path.name}" | |
| 105 | + if not target.exists(): | |
| 106 | + target.write_bytes(content) | |
| 107 | + return target | |
| 108 | + | |
| 109 | + def save_export(self, filename: str, content: str) -> Path: | |
| 110 | + target = self.exports_dir / filename | |
| 111 | + target.write_text(content, encoding="utf-8") | |
| 112 | + return target | |
| 113 | + | |
| 114 | + def status(self) -> dict: | |
| 115 | + ledger = self.ledger() | |
| 116 | + documents = sorted((self.root / "documents").glob("*")) | |
| 117 | + exports = sorted(self.exports_dir.glob("*")) | |
| 118 | + return { | |
| 119 | + "home": str(self.root), | |
| 120 | + "name": self.meta.get("name"), | |
| 121 | + "policies": self.meta.get("policies"), | |
| 122 | + "entries": len(ledger.entries), | |
| 123 | + "chain_valid": ledger.verify_chain(), | |
| 124 | + "documents_archived": len(documents), | |
| 125 | + "exports": len(exports), | |
| 126 | + } | |
added
pyproject.toml
+66 −0
@@ -0,0 +1,66 @@ | ||
| 1 | +# Project : AIR — Accounting Intermediate Representation | |
| 2 | +# Author : Simon-Pierre Boucher | |
| 3 | +# Contact : contact@spboucher.ai | |
| 4 | + | |
| 5 | +[build-system] | |
| 6 | +requires = ["setuptools>=68"] | |
| 7 | +build-backend = "setuptools.build_meta" | |
| 8 | + | |
| 9 | +[project] | |
| 10 | +name = "air-accounting" | |
| 11 | +version = "0.1.0" | |
| 12 | +description = "AIR — Accounting Intermediate Representation. LLVM-style compiler infrastructure for accounting: LLMs produce economic events (AIR), a deterministic compiler (AIC) produces journal entries." | |
| 13 | +authors = [{ name = "Simon-Pierre Boucher", email = "contact@spboucher.ai" }] | |
| 14 | +readme = "README.md" | |
| 15 | +license = { text = "MIT" } | |
| 16 | +requires-python = ">=3.11" | |
| 17 | +keywords = ["accounting", "compiler", "double-entry", "ledger", "llm", "ir", "gst", "qst", "erp"] | |
| 18 | +classifiers = [ | |
| 19 | + "Development Status :: 4 - Beta", | |
| 20 | + "Intended Audience :: Developers", | |
| 21 | + "Intended Audience :: Financial and Insurance Industry", | |
| 22 | + "License :: OSI Approved :: MIT License", | |
| 23 | + "Programming Language :: Python :: 3 :: Only", | |
| 24 | + "Programming Language :: Python :: 3.11", | |
| 25 | + "Programming Language :: Python :: 3.12", | |
| 26 | + "Programming Language :: Python :: 3.13", | |
| 27 | + "Topic :: Office/Business :: Financial :: Accounting", | |
| 28 | + "Topic :: Software Development :: Compilers", | |
| 29 | +] | |
| 30 | +dependencies = [ | |
| 31 | + "pydantic>=2.7", | |
| 32 | + "pyyaml>=6.0", | |
| 33 | +] | |
| 34 | + | |
| 35 | +[project.urls] | |
| 36 | +Homepage = "https://github.com/spboucher-ai/air" | |
| 37 | +Documentation = "https://github.com/spboucher-ai/air/tree/main/docs" | |
| 38 | +Issues = "https://github.com/spboucher-ai/air/issues" | |
| 39 | + | |
| 40 | +[project.scripts] | |
| 41 | +air = "sdk.cli:main" | |
| 42 | + | |
| 43 | +[project.optional-dependencies] | |
| 44 | +dev = [ | |
| 45 | + "pytest>=8.0", | |
| 46 | + "hypothesis>=6.100", | |
| 47 | + "mypy>=1.10", | |
| 48 | +] | |
| 49 | +# Optional: real LLM extraction (ingestion works offline without it via the mock extractor) | |
| 50 | +llm = [ | |
| 51 | + "anthropic>=0.40", | |
| 52 | +] | |
| 53 | + | |
| 54 | +[tool.setuptools.packages.find] | |
| 55 | +include = ["core*", "aic*", "alsl*", "backends*", "kernel*", "sdk*", "ingestion*"] | |
| 56 | + | |
| 57 | +[tool.setuptools.package-data] | |
| 58 | +alsl = ["policies/*.yaml"] | |
| 59 | + | |
| 60 | +[tool.pytest.ini_options] | |
| 61 | +testpaths = ["tests"] | |
| 62 | +pythonpath = ["."] | |
| 63 | + | |
| 64 | +[tool.mypy] | |
| 65 | +python_version = "3.11" | |
| 66 | +strict = true | |
added
schemas/air-0.1.schema.json
+565 −0
@@ -0,0 +1,565 @@ | ||
| 1 | +{ | |
| 2 | + "_author": "Simon-Pierre Boucher <contact@spboucher.ai>", | |
| 3 | + "$schema": "https://json-schema.org/draft/2020-12/schema", | |
| 4 | + "$id": "https://spboucher.ai/air/schemas/air-0.1.schema.json", | |
| 5 | + "title": "AirDocument", | |
| 6 | + "$defs": { | |
| 7 | + "AmountSpec": { | |
| 8 | + "additionalProperties": false, | |
| 9 | + "description": "An amount as it appears in AIR documents (exact decimal + currency).", | |
| 10 | + "properties": { | |
| 11 | + "amount": { | |
| 12 | + "anyOf": [ | |
| 13 | + { | |
| 14 | + "type": "number" | |
| 15 | + }, | |
| 16 | + { | |
| 17 | + "pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", | |
| 18 | + "type": "string" | |
| 19 | + } | |
| 20 | + ], | |
| 21 | + "title": "Amount" | |
| 22 | + }, | |
| 23 | + "currency": { | |
| 24 | + "title": "Currency", | |
| 25 | + "type": "string" | |
| 26 | + } | |
| 27 | + }, | |
| 28 | + "required": [ | |
| 29 | + "amount", | |
| 30 | + "currency" | |
| 31 | + ], | |
| 32 | + "title": "AmountSpec", | |
| 33 | + "type": "object" | |
| 34 | + }, | |
| 35 | + "DeliveryInfo": { | |
| 36 | + "additionalProperties": false, | |
| 37 | + "properties": { | |
| 38 | + "status": { | |
| 39 | + "default": "pending", | |
| 40 | + "title": "Status", | |
| 41 | + "type": "string" | |
| 42 | + }, | |
| 43 | + "expected": { | |
| 44 | + "anyOf": [ | |
| 45 | + { | |
| 46 | + "format": "date", | |
| 47 | + "type": "string" | |
| 48 | + }, | |
| 49 | + { | |
| 50 | + "type": "null" | |
| 51 | + } | |
| 52 | + ], | |
| 53 | + "default": null, | |
| 54 | + "title": "Expected" | |
| 55 | + } | |
| 56 | + }, | |
| 57 | + "title": "DeliveryInfo", | |
| 58 | + "type": "object" | |
| 59 | + }, | |
| 60 | + "EconomicEvent": { | |
| 61 | + "additionalProperties": false, | |
| 62 | + "description": "A single economic event \u2014 the atom of the AIR language.", | |
| 63 | + "properties": { | |
| 64 | + "air_version": { | |
| 65 | + "default": "0.1", | |
| 66 | + "title": "Air Version", | |
| 67 | + "type": "string" | |
| 68 | + }, | |
| 69 | + "id": { | |
| 70 | + "title": "Id", | |
| 71 | + "type": "string" | |
| 72 | + }, | |
| 73 | + "type": { | |
| 74 | + "$ref": "#/$defs/EventType" | |
| 75 | + }, | |
| 76 | + "date": { | |
| 77 | + "format": "date", | |
| 78 | + "title": "Date", | |
| 79 | + "type": "string" | |
| 80 | + }, | |
| 81 | + "description": { | |
| 82 | + "anyOf": [ | |
| 83 | + { | |
| 84 | + "type": "string" | |
| 85 | + }, | |
| 86 | + { | |
| 87 | + "type": "null" | |
| 88 | + } | |
| 89 | + ], | |
| 90 | + "default": null, | |
| 91 | + "title": "Description" | |
| 92 | + }, | |
| 93 | + "parties": { | |
| 94 | + "additionalProperties": { | |
| 95 | + "type": "string" | |
| 96 | + }, | |
| 97 | + "title": "Parties", | |
| 98 | + "type": "object" | |
| 99 | + }, | |
| 100 | + "items": { | |
| 101 | + "default": [], | |
| 102 | + "items": { | |
| 103 | + "$ref": "#/$defs/LineItem" | |
| 104 | + }, | |
| 105 | + "title": "Items", | |
| 106 | + "type": "array" | |
| 107 | + }, | |
| 108 | + "amount": { | |
| 109 | + "anyOf": [ | |
| 110 | + { | |
| 111 | + "$ref": "#/$defs/AmountSpec" | |
| 112 | + }, | |
| 113 | + { | |
| 114 | + "type": "null" | |
| 115 | + } | |
| 116 | + ], | |
| 117 | + "default": null | |
| 118 | + }, | |
| 119 | + "payment": { | |
| 120 | + "anyOf": [ | |
| 121 | + { | |
| 122 | + "$ref": "#/$defs/PaymentInfo" | |
| 123 | + }, | |
| 124 | + { | |
| 125 | + "type": "null" | |
| 126 | + } | |
| 127 | + ], | |
| 128 | + "default": null | |
| 129 | + }, | |
| 130 | + "delivery": { | |
| 131 | + "anyOf": [ | |
| 132 | + { | |
| 133 | + "$ref": "#/$defs/DeliveryInfo" | |
| 134 | + }, | |
| 135 | + { | |
| 136 | + "type": "null" | |
| 137 | + } | |
| 138 | + ], | |
| 139 | + "default": null | |
| 140 | + }, | |
| 141 | + "tax": { | |
| 142 | + "anyOf": [ | |
| 143 | + { | |
| 144 | + "$ref": "#/$defs/TaxContext" | |
| 145 | + }, | |
| 146 | + { | |
| 147 | + "type": "null" | |
| 148 | + } | |
| 149 | + ], | |
| 150 | + "default": null | |
| 151 | + }, | |
| 152 | + "fx": { | |
| 153 | + "anyOf": [ | |
| 154 | + { | |
| 155 | + "$ref": "#/$defs/FxInfo" | |
| 156 | + }, | |
| 157 | + { | |
| 158 | + "type": "null" | |
| 159 | + } | |
| 160 | + ], | |
| 161 | + "default": null | |
| 162 | + }, | |
| 163 | + "related_event": { | |
| 164 | + "anyOf": [ | |
| 165 | + { | |
| 166 | + "type": "string" | |
| 167 | + }, | |
| 168 | + { | |
| 169 | + "type": "null" | |
| 170 | + } | |
| 171 | + ], | |
| 172 | + "default": null, | |
| 173 | + "title": "Related Event" | |
| 174 | + }, | |
| 175 | + "meta": { | |
| 176 | + "anyOf": [ | |
| 177 | + { | |
| 178 | + "$ref": "#/$defs/Meta" | |
| 179 | + }, | |
| 180 | + { | |
| 181 | + "type": "null" | |
| 182 | + } | |
| 183 | + ], | |
| 184 | + "default": null | |
| 185 | + } | |
| 186 | + }, | |
| 187 | + "required": [ | |
| 188 | + "id", | |
| 189 | + "type", | |
| 190 | + "date" | |
| 191 | + ], | |
| 192 | + "title": "EconomicEvent", | |
| 193 | + "type": "object" | |
| 194 | + }, | |
| 195 | + "EventType": { | |
| 196 | + "enum": [ | |
| 197 | + "Sale", | |
| 198 | + "Purchase", | |
| 199 | + "Refund", | |
| 200 | + "PaymentReceived", | |
| 201 | + "PaymentSent", | |
| 202 | + "OwnerContribution", | |
| 203 | + "LoanReceived" | |
| 204 | + ], | |
| 205 | + "title": "EventType", | |
| 206 | + "type": "string" | |
| 207 | + }, | |
| 208 | + "FxInfo": { | |
| 209 | + "additionalProperties": false, | |
| 210 | + "description": "Observed FX data attached to the event (input data, not a rule).\n\nThe rate is a fact about the world (e.g. Bank of Canada daily rate on the\ntransaction date); it is provided by ingestion, never hardcoded in AIR/AIC.", | |
| 211 | + "properties": { | |
| 212 | + "rate": { | |
| 213 | + "anyOf": [ | |
| 214 | + { | |
| 215 | + "type": "number" | |
| 216 | + }, | |
| 217 | + { | |
| 218 | + "pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", | |
| 219 | + "type": "string" | |
| 220 | + } | |
| 221 | + ], | |
| 222 | + "title": "Rate" | |
| 223 | + }, | |
| 224 | + "source": { | |
| 225 | + "title": "Source", | |
| 226 | + "type": "string" | |
| 227 | + }, | |
| 228 | + "rate_date": { | |
| 229 | + "format": "date", | |
| 230 | + "title": "Rate Date", | |
| 231 | + "type": "string" | |
| 232 | + } | |
| 233 | + }, | |
| 234 | + "required": [ | |
| 235 | + "rate", | |
| 236 | + "source", | |
| 237 | + "rate_date" | |
| 238 | + ], | |
| 239 | + "title": "FxInfo", | |
| 240 | + "type": "object" | |
| 241 | + }, | |
| 242 | + "LineItem": { | |
| 243 | + "additionalProperties": false, | |
| 244 | + "properties": { | |
| 245 | + "sku": { | |
| 246 | + "anyOf": [ | |
| 247 | + { | |
| 248 | + "type": "string" | |
| 249 | + }, | |
| 250 | + { | |
| 251 | + "type": "null" | |
| 252 | + } | |
| 253 | + ], | |
| 254 | + "default": null, | |
| 255 | + "title": "Sku" | |
| 256 | + }, | |
| 257 | + "description": { | |
| 258 | + "anyOf": [ | |
| 259 | + { | |
| 260 | + "type": "string" | |
| 261 | + }, | |
| 262 | + { | |
| 263 | + "type": "null" | |
| 264 | + } | |
| 265 | + ], | |
| 266 | + "default": null, | |
| 267 | + "title": "Description" | |
| 268 | + }, | |
| 269 | + "qty": { | |
| 270 | + "anyOf": [ | |
| 271 | + { | |
| 272 | + "type": "number" | |
| 273 | + }, | |
| 274 | + { | |
| 275 | + "pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", | |
| 276 | + "type": "string" | |
| 277 | + } | |
| 278 | + ], | |
| 279 | + "default": "1", | |
| 280 | + "title": "Qty" | |
| 281 | + }, | |
| 282 | + "unit_price": { | |
| 283 | + "$ref": "#/$defs/AmountSpec" | |
| 284 | + } | |
| 285 | + }, | |
| 286 | + "required": [ | |
| 287 | + "unit_price" | |
| 288 | + ], | |
| 289 | + "title": "LineItem", | |
| 290 | + "type": "object" | |
| 291 | + }, | |
| 292 | + "LlmInfo": { | |
| 293 | + "additionalProperties": false, | |
| 294 | + "properties": { | |
| 295 | + "model": { | |
| 296 | + "anyOf": [ | |
| 297 | + { | |
| 298 | + "type": "string" | |
| 299 | + }, | |
| 300 | + { | |
| 301 | + "type": "null" | |
| 302 | + } | |
| 303 | + ], | |
| 304 | + "default": null, | |
| 305 | + "title": "Model" | |
| 306 | + }, | |
| 307 | + "confidence": { | |
| 308 | + "anyOf": [ | |
| 309 | + { | |
| 310 | + "type": "number" | |
| 311 | + }, | |
| 312 | + { | |
| 313 | + "pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", | |
| 314 | + "type": "string" | |
| 315 | + }, | |
| 316 | + { | |
| 317 | + "type": "null" | |
| 318 | + } | |
| 319 | + ], | |
| 320 | + "default": null, | |
| 321 | + "title": "Confidence" | |
| 322 | + }, | |
| 323 | + "reasoning_hash": { | |
| 324 | + "anyOf": [ | |
| 325 | + { | |
| 326 | + "type": "string" | |
| 327 | + }, | |
| 328 | + { | |
| 329 | + "type": "null" | |
| 330 | + } | |
| 331 | + ], | |
| 332 | + "default": null, | |
| 333 | + "title": "Reasoning Hash" | |
| 334 | + } | |
| 335 | + }, | |
| 336 | + "title": "LlmInfo", | |
| 337 | + "type": "object" | |
| 338 | + }, | |
| 339 | + "Meta": { | |
| 340 | + "additionalProperties": false, | |
| 341 | + "properties": { | |
| 342 | + "source": { | |
| 343 | + "anyOf": [ | |
| 344 | + { | |
| 345 | + "$ref": "#/$defs/SourceInfo" | |
| 346 | + }, | |
| 347 | + { | |
| 348 | + "type": "null" | |
| 349 | + } | |
| 350 | + ], | |
| 351 | + "default": null | |
| 352 | + }, | |
| 353 | + "llm": { | |
| 354 | + "anyOf": [ | |
| 355 | + { | |
| 356 | + "$ref": "#/$defs/LlmInfo" | |
| 357 | + }, | |
| 358 | + { | |
| 359 | + "type": "null" | |
| 360 | + } | |
| 361 | + ], | |
| 362 | + "default": null | |
| 363 | + }, | |
| 364 | + "policy_version": { | |
| 365 | + "anyOf": [ | |
| 366 | + { | |
| 367 | + "type": "string" | |
| 368 | + }, | |
| 369 | + { | |
| 370 | + "type": "null" | |
| 371 | + } | |
| 372 | + ], | |
| 373 | + "default": null, | |
| 374 | + "title": "Policy Version" | |
| 375 | + }, | |
| 376 | + "timestamps": { | |
| 377 | + "anyOf": [ | |
| 378 | + { | |
| 379 | + "$ref": "#/$defs/Timestamps" | |
| 380 | + }, | |
| 381 | + { | |
| 382 | + "type": "null" | |
| 383 | + } | |
| 384 | + ], | |
| 385 | + "default": null | |
| 386 | + }, | |
| 387 | + "approver": { | |
| 388 | + "anyOf": [ | |
| 389 | + { | |
| 390 | + "type": "string" | |
| 391 | + }, | |
| 392 | + { | |
| 393 | + "type": "null" | |
| 394 | + } | |
| 395 | + ], | |
| 396 | + "default": null, | |
| 397 | + "title": "Approver" | |
| 398 | + } | |
| 399 | + }, | |
| 400 | + "title": "Meta", | |
| 401 | + "type": "object" | |
| 402 | + }, | |
| 403 | + "PaymentInfo": { | |
| 404 | + "additionalProperties": false, | |
| 405 | + "properties": { | |
| 406 | + "method": { | |
| 407 | + "anyOf": [ | |
| 408 | + { | |
| 409 | + "type": "string" | |
| 410 | + }, | |
| 411 | + { | |
| 412 | + "type": "null" | |
| 413 | + } | |
| 414 | + ], | |
| 415 | + "default": null, | |
| 416 | + "title": "Method" | |
| 417 | + }, | |
| 418 | + "gross": { | |
| 419 | + "anyOf": [ | |
| 420 | + { | |
| 421 | + "$ref": "#/$defs/AmountSpec" | |
| 422 | + }, | |
| 423 | + { | |
| 424 | + "type": "null" | |
| 425 | + } | |
| 426 | + ], | |
| 427 | + "default": null | |
| 428 | + }, | |
| 429 | + "immediate": { | |
| 430 | + "default": false, | |
| 431 | + "title": "Immediate", | |
| 432 | + "type": "boolean" | |
| 433 | + } | |
| 434 | + }, | |
| 435 | + "title": "PaymentInfo", | |
| 436 | + "type": "object" | |
| 437 | + }, | |
| 438 | + "SourceInfo": { | |
| 439 | + "additionalProperties": false, | |
| 440 | + "properties": { | |
| 441 | + "kind": { | |
| 442 | + "anyOf": [ | |
| 443 | + { | |
| 444 | + "type": "string" | |
| 445 | + }, | |
| 446 | + { | |
| 447 | + "type": "null" | |
| 448 | + } | |
| 449 | + ], | |
| 450 | + "default": null, | |
| 451 | + "title": "Kind" | |
| 452 | + }, | |
| 453 | + "uri": { | |
| 454 | + "anyOf": [ | |
| 455 | + { | |
| 456 | + "type": "string" | |
| 457 | + }, | |
| 458 | + { | |
| 459 | + "type": "null" | |
| 460 | + } | |
| 461 | + ], | |
| 462 | + "default": null, | |
| 463 | + "title": "Uri" | |
| 464 | + }, | |
| 465 | + "ocr_score": { | |
| 466 | + "anyOf": [ | |
| 467 | + { | |
| 468 | + "type": "number" | |
| 469 | + }, | |
| 470 | + { | |
| 471 | + "pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", | |
| 472 | + "type": "string" | |
| 473 | + }, | |
| 474 | + { | |
| 475 | + "type": "null" | |
| 476 | + } | |
| 477 | + ], | |
| 478 | + "default": null, | |
| 479 | + "title": "Ocr Score" | |
| 480 | + } | |
| 481 | + }, | |
| 482 | + "title": "SourceInfo", | |
| 483 | + "type": "object" | |
| 484 | + }, | |
| 485 | + "TaxContext": { | |
| 486 | + "additionalProperties": false, | |
| 487 | + "description": "Where the supply takes place. Rates NEVER live here \u2014 they live in\nversioned ALSL policies; the compiler resolves jurisdiction \u2192 policy.", | |
| 488 | + "properties": { | |
| 489 | + "jurisdiction": { | |
| 490 | + "title": "Jurisdiction", | |
| 491 | + "type": "string" | |
| 492 | + }, | |
| 493 | + "codes": { | |
| 494 | + "default": [], | |
| 495 | + "items": { | |
| 496 | + "type": "string" | |
| 497 | + }, | |
| 498 | + "title": "Codes", | |
| 499 | + "type": "array" | |
| 500 | + }, | |
| 501 | + "exempt": { | |
| 502 | + "default": false, | |
| 503 | + "title": "Exempt", | |
| 504 | + "type": "boolean" | |
| 505 | + } | |
| 506 | + }, | |
| 507 | + "required": [ | |
| 508 | + "jurisdiction" | |
| 509 | + ], | |
| 510 | + "title": "TaxContext", | |
| 511 | + "type": "object" | |
| 512 | + }, | |
| 513 | + "Timestamps": { | |
| 514 | + "additionalProperties": false, | |
| 515 | + "properties": { | |
| 516 | + "ingested": { | |
| 517 | + "anyOf": [ | |
| 518 | + { | |
| 519 | + "format": "date-time", | |
| 520 | + "type": "string" | |
| 521 | + }, | |
| 522 | + { | |
| 523 | + "type": "null" | |
| 524 | + } | |
| 525 | + ], | |
| 526 | + "default": null, | |
| 527 | + "title": "Ingested" | |
| 528 | + }, | |
| 529 | + "approved": { | |
| 530 | + "anyOf": [ | |
| 531 | + { | |
| 532 | + "format": "date-time", | |
| 533 | + "type": "string" | |
| 534 | + }, | |
| 535 | + { | |
| 536 | + "type": "null" | |
| 537 | + } | |
| 538 | + ], | |
| 539 | + "default": null, | |
| 540 | + "title": "Approved" | |
| 541 | + } | |
| 542 | + }, | |
| 543 | + "title": "Timestamps", | |
| 544 | + "type": "object" | |
| 545 | + } | |
| 546 | + }, | |
| 547 | + "additionalProperties": false, | |
| 548 | + "description": "A batch of economic events (the compilation unit's input).", | |
| 549 | + "properties": { | |
| 550 | + "air_version": { | |
| 551 | + "default": "0.1", | |
| 552 | + "title": "Air Version", | |
| 553 | + "type": "string" | |
| 554 | + }, | |
| 555 | + "events": { | |
| 556 | + "default": [], | |
| 557 | + "items": { | |
| 558 | + "$ref": "#/$defs/EconomicEvent" | |
| 559 | + }, | |
| 560 | + "title": "Events", | |
| 561 | + "type": "array" | |
| 562 | + } | |
| 563 | + }, | |
| 564 | + "type": "object" | |
| 565 | +} | |
| \ No newline at end of file | ||
added
scripts/check_headers.py
+74 −0
@@ -0,0 +1,74 @@ | ||
| 1 | +#!/usr/bin/env python3 | |
| 2 | +# ============================================================================= | |
| 3 | +# Projet : AIR — Accounting Intermediate Representation | |
| 4 | +# Auteur : Simon-Pierre Boucher | |
| 5 | +# Contact : contact@spboucher.ai | |
| 6 | +# Fichier : check_headers.py | |
| 7 | +# Description : CI gate — fails if any project file is missing the author header. | |
| 8 | +# ============================================================================= | |
| 9 | +"""Verify that every project file carries the mandatory author header. | |
| 10 | + | |
| 11 | +Usage: python3 scripts/check_headers.py [root] | |
| 12 | +Exit code 0 if all files pass, 1 otherwise (with a list of offenders). | |
| 13 | +""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +import json | |
| 17 | +import sys | |
| 18 | +from pathlib import Path | |
| 19 | + | |
| 20 | +AUTHOR = "Simon-Pierre Boucher" | |
| 21 | +CONTACT = "contact@spboucher.ai" | |
| 22 | + | |
| 23 | +# Extensions checked for a comment header containing author + contact. | |
| 24 | +COMMENT_EXTS = { | |
| 25 | + ".py", ".ts", ".js", ".rs", ".go", ".c", ".h", ".zig", | |
| 26 | + ".md", ".yaml", ".yml", ".toml", ".sql", ".sh", | |
| 27 | +} | |
| 28 | +SKIP_DIRS = {".git", ".venv", "venv", "node_modules", "__pycache__", | |
| 29 | + ".pytest_cache", ".mypy_cache", ".hypothesis", ".claude", "dist", "build"} | |
| 30 | +SKIP_FILES = {".gitignore", "LICENSE", "py.typed"} | |
| 31 | +HEADER_WINDOW = 15 # header must appear within the first N lines | |
| 32 | + | |
| 33 | + | |
| 34 | +def has_header(path: Path) -> bool: | |
| 35 | + if path.suffix == ".json": | |
| 36 | + try: | |
| 37 | + data = json.loads(path.read_text(encoding="utf-8")) | |
| 38 | + except (json.JSONDecodeError, UnicodeDecodeError): | |
| 39 | + return False | |
| 40 | + return isinstance(data, dict) and CONTACT in str(data.get("_author", "")) | |
| 41 | + try: | |
| 42 | + head = "".join( | |
| 43 | + path.read_text(encoding="utf-8").splitlines(keepends=True)[:HEADER_WINDOW] | |
| 44 | + ) | |
| 45 | + except UnicodeDecodeError: | |
| 46 | + return True # binary-ish file, not subject to header rule | |
| 47 | + return AUTHOR in head and CONTACT in head | |
| 48 | + | |
| 49 | + | |
| 50 | +def main() -> int: | |
| 51 | + root = Path(sys.argv[1]) if len(sys.argv) > 1 else Path(__file__).resolve().parent.parent | |
| 52 | + offenders: list[Path] = [] | |
| 53 | + for path in sorted(root.rglob("*")): | |
| 54 | + if not path.is_file(): | |
| 55 | + continue | |
| 56 | + if any(part in SKIP_DIRS for part in path.parts): | |
| 57 | + continue | |
| 58 | + if path.name in SKIP_FILES: | |
| 59 | + continue | |
| 60 | + if path.suffix not in COMMENT_EXTS and path.suffix != ".json": | |
| 61 | + continue | |
| 62 | + if not has_header(path): | |
| 63 | + offenders.append(path.relative_to(root)) | |
| 64 | + if offenders: | |
| 65 | + print("Missing author header (Simon-Pierre Boucher / contact@spboucher.ai):") | |
| 66 | + for p in offenders: | |
| 67 | + print(f" - {p}") | |
| 68 | + return 1 | |
| 69 | + print("check_headers: OK — all files carry the author header.") | |
| 70 | + return 0 | |
| 71 | + | |
| 72 | + | |
| 73 | +if __name__ == "__main__": | |
| 74 | + raise SystemExit(main()) | |
added
sdk/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
sdk/cli.py
+453 −0
@@ -0,0 +1,453 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : cli.py | |
| 6 | +# Description : Standalone CLI — AIR home, compile, incremental recompile, statements, exports. | |
| 7 | +# ============================================================================= | |
| 8 | +"""AIR standalone command line. | |
| 9 | + | |
| 10 | +AIR needs no third-party system: keep your books in a managed local AIR home | |
| 11 | +(hash-chained ledger + archived documents + exports) and produce every | |
| 12 | +statement in several formats. QuickBooks/CSV are optional export targets. | |
| 13 | + | |
| 14 | + # one-time: create your books directory | |
| 15 | + python -m sdk.cli init --home books --policies alsl/policies/ca-qc-2026.yaml | |
| 16 | + | |
| 17 | + # day to day: compile events into your books (document is archived) | |
| 18 | + python -m sdk.cli compile doc.yaml --home books --report trial-balance | |
| 19 | + | |
| 20 | + # a source document changed? post only the delta (reversal + replacement) | |
| 21 | + python -m sdk.cli recompile old.yaml new.yaml --home books | |
| 22 | + | |
| 23 | + # statements any time, any format | |
| 24 | + python -m sdk.cli report balance-sheet --home books --format markdown | |
| 25 | + python -m sdk.cli status --home books | |
| 26 | + | |
| 27 | + # optional exports (no QuickBooks account required for qbo-export) | |
| 28 | + python -m sdk.cli compile doc.yaml --home books --backend csv | |
| 29 | + python -m sdk.cli compile doc.yaml --home books --backend qbo-export | |
| 30 | +""" | |
| 31 | +from __future__ import annotations | |
| 32 | + | |
| 33 | +import argparse | |
| 34 | +import hashlib | |
| 35 | +import json | |
| 36 | +import sys | |
| 37 | +from pathlib import Path | |
| 38 | + | |
| 39 | +from aic.compiler import compile_document | |
| 40 | +from aic.diagnostics import CompilationError, Diagnostic | |
| 41 | +from aic.incremental import recompile | |
| 42 | +from alsl.loader import load_policy_set | |
| 43 | +from backends.generic_csv.backend import GenericCsvBackend | |
| 44 | +from backends.native.backend import NativeLedgerBackend | |
| 45 | +from backends.quickbooks.backend import QuickBooksBackend | |
| 46 | +from core.document_io import load_air_document | |
| 47 | +from core.events import AirDocument | |
| 48 | +from core.journal import CompiledJournal | |
| 49 | +from ingestion.approval import ApprovalQueue | |
| 50 | +from ingestion.extractor import MockExtractor | |
| 51 | +from ingestion.pipeline import DEFAULT_CONFIDENCE_THRESHOLD, Route, ingest | |
| 52 | +from kernel.ledger import Ledger | |
| 53 | +from kernel.reporting import FORMATS, REPORTS | |
| 54 | +from kernel.workspace import Workspace, WorkspaceError | |
| 55 | + | |
| 56 | + | |
| 57 | +def _print_diags(diags: list[Diagnostic]) -> None: | |
| 58 | + for d in diags: | |
| 59 | + print(d.render(), file=sys.stderr) | |
| 60 | + | |
| 61 | + | |
| 62 | +def _resolve(args: argparse.Namespace) -> tuple[Workspace | None, str, str]: | |
| 63 | + """Resolve (workspace, policies_path, ledger_path) from --home/--policies.""" | |
| 64 | + workspace: Workspace | None = None | |
| 65 | + if getattr(args, "home", None): | |
| 66 | + workspace = Workspace.open(args.home) | |
| 67 | + policies = getattr(args, "policies", None) or ( | |
| 68 | + workspace.default_policies() if workspace else None | |
| 69 | + ) | |
| 70 | + if not policies: | |
| 71 | + raise WorkspaceError( | |
| 72 | + "no policy set: pass --policies, or init the home with a default " | |
| 73 | + "(air init --home <dir> --policies <set.yaml>)" | |
| 74 | + ) | |
| 75 | + ledger = str(workspace.ledger_path) if workspace else getattr( | |
| 76 | + args, "ledger", "books/ledger.jsonl" | |
| 77 | + ) | |
| 78 | + return workspace, policies, ledger | |
| 79 | + | |
| 80 | + | |
| 81 | +def _idempotency_key(*parts: str) -> str: | |
| 82 | + return hashlib.sha256("".join(parts).encode("utf-8")).hexdigest()[:16] | |
| 83 | + | |
| 84 | + | |
| 85 | +def _dispatch_backend( | |
| 86 | + args: argparse.Namespace, | |
| 87 | + workspace: Workspace | None, | |
| 88 | + ledger_path: str, | |
| 89 | + journal: CompiledJournal, | |
| 90 | + key: str, | |
| 91 | +) -> str: | |
| 92 | + """Post the journal to the chosen backend; returns a human summary.""" | |
| 93 | + if args.backend == "csv": | |
| 94 | + out = (workspace.exports_dir if workspace else Path(args.out)) | |
| 95 | + backend = GenericCsvBackend(out) | |
| 96 | + receipt = backend.post(backend.compile(journal), key) | |
| 97 | + return f"{len(journal.entries)} entries -> CSV {receipt.reference}" | |
| 98 | + if args.backend == "qbo-export": | |
| 99 | + # Offline QuickBooks-shaped JSON export: NO QuickBooks account needed. | |
| 100 | + payload = QuickBooksBackend().compile(journal) | |
| 101 | + body = json.dumps( | |
| 102 | + {"_author": "Simon-Pierre Boucher <contact@spboucher.ai>", | |
| 103 | + "journal_entries": payload.body}, | |
| 104 | + indent=2, default=str, | |
| 105 | + ) | |
| 106 | + out_dir = workspace.exports_dir if workspace else Path(args.out) | |
| 107 | + out_dir.mkdir(parents=True, exist_ok=True) | |
| 108 | + target = out_dir / f"qbo_journal_{key}.json" | |
| 109 | + target.write_text(body, encoding="utf-8") | |
| 110 | + return f"{len(journal.entries)} entries -> QBO JSON export {target}" | |
| 111 | + backend = NativeLedgerBackend(ledger_path) | |
| 112 | + receipt = backend.post(backend.compile(journal), key) | |
| 113 | + for name in getattr(args, "report", None) or []: | |
| 114 | + print(REPORTS[name](backend.ledger, args.format)) | |
| 115 | + return ( | |
| 116 | + f"{len(journal.entries)} entries -> native ledger {receipt.reference} " | |
| 117 | + f"(+{receipt.details['entries_appended']} appended)" | |
| 118 | + ) | |
| 119 | + | |
| 120 | + | |
| 121 | +def _init(args: argparse.Namespace) -> int: | |
| 122 | + workspace = Workspace.init(args.home, name=args.name, policies=args.policies) | |
| 123 | + print(f"initialized AIR home at {workspace.root} " | |
| 124 | + f"(default policies: {args.policies or 'none'})") | |
| 125 | + return 0 | |
| 126 | + | |
| 127 | + | |
| 128 | +def _status(args: argparse.Namespace) -> int: | |
| 129 | + status = Workspace.open(args.home).status() | |
| 130 | + for k, v in status.items(): | |
| 131 | + print(f"{k:20} {v}") | |
| 132 | + return 0 | |
| 133 | + | |
| 134 | + | |
| 135 | +def _compile(args: argparse.Namespace) -> int: | |
| 136 | + workspace, policies_path, ledger_path = _resolve(args) | |
| 137 | + document = load_air_document(args.document) | |
| 138 | + policies = load_policy_set(policies_path) | |
| 139 | + try: | |
| 140 | + journal, diagnostics = compile_document( | |
| 141 | + document, policies, optimize=getattr(args, "optimize", False)) | |
| 142 | + except CompilationError as exc: | |
| 143 | + _print_diags(exc.diagnostics) | |
| 144 | + return 1 | |
| 145 | + _print_diags(diagnostics) | |
| 146 | + | |
| 147 | + key = _idempotency_key( | |
| 148 | + Path(args.document).read_text(encoding="utf-8"), | |
| 149 | + policies.name, policies.version, | |
| 150 | + ) | |
| 151 | + summary = _dispatch_backend(args, workspace, ledger_path, journal, key) | |
| 152 | + if summary: | |
| 153 | + print(f"compiled {summary}") | |
| 154 | + if workspace: | |
| 155 | + archived = workspace.archive_document(args.document) | |
| 156 | + print(f"document archived: {archived}") | |
| 157 | + return 0 | |
| 158 | + | |
| 159 | + | |
| 160 | +def _recompile(args: argparse.Namespace) -> int: | |
| 161 | + workspace, policies_path, ledger_path = _resolve(args) | |
| 162 | + old_doc = load_air_document(args.old_document) | |
| 163 | + new_doc = load_air_document(args.new_document) | |
| 164 | + policies = load_policy_set(policies_path) | |
| 165 | + try: | |
| 166 | + result = recompile(old_doc, new_doc, policies) | |
| 167 | + except CompilationError as exc: | |
| 168 | + _print_diags(exc.diagnostics) | |
| 169 | + return 1 | |
| 170 | + _print_diags(result.diagnostics) | |
| 171 | + | |
| 172 | + diff = result.diff | |
| 173 | + print(f"diff: +{len(diff.added)} added, -{len(diff.removed)} removed, " | |
| 174 | + f"~{len(diff.changed)} changed, ={len(diff.unchanged)} unchanged") | |
| 175 | + if diff.is_empty(): | |
| 176 | + print("nothing to post: documents compile identically") | |
| 177 | + return 0 | |
| 178 | + | |
| 179 | + key = _idempotency_key( | |
| 180 | + Path(args.old_document).read_text(encoding="utf-8"), | |
| 181 | + Path(args.new_document).read_text(encoding="utf-8"), | |
| 182 | + policies.name, policies.version, | |
| 183 | + ) | |
| 184 | + summary = _dispatch_backend(args, workspace, ledger_path, result.journal, key) | |
| 185 | + if summary: | |
| 186 | + print(f"posted {len(result.reversals)} reversal(s) + " | |
| 187 | + f"{len(result.new_entries)} new -> {summary}") | |
| 188 | + if workspace: | |
| 189 | + workspace.archive_document(args.new_document) | |
| 190 | + return 0 | |
| 191 | + | |
| 192 | + | |
| 193 | +def _verify(args: argparse.Namespace) -> int: | |
| 194 | + _, policies_path, _ = _resolve(args) | |
| 195 | + document = load_air_document(args.document) | |
| 196 | + policies = load_policy_set(policies_path) | |
| 197 | + try: | |
| 198 | + journal, diagnostics = compile_document(document, policies) | |
| 199 | + except CompilationError as exc: | |
| 200 | + _print_diags(exc.diagnostics) | |
| 201 | + return 1 | |
| 202 | + _print_diags(diagnostics) | |
| 203 | + print(f"OK: {len(document.events)} events -> {len(journal.entries)} balanced entries") | |
| 204 | + return 0 | |
| 205 | + | |
| 206 | + | |
| 207 | +def _compile_and_post(workspace: Workspace, policies_path: str, | |
| 208 | + document: AirDocument, key: str) -> str: | |
| 209 | + policies = load_policy_set(policies_path) | |
| 210 | + journal, diagnostics = compile_document(document, policies) | |
| 211 | + _print_diags(diagnostics) | |
| 212 | + backend = NativeLedgerBackend(workspace.ledger_path) | |
| 213 | + receipt = backend.post(backend.compile(journal), key) | |
| 214 | + return (f"{len(journal.entries)} entries posted to {receipt.reference} " | |
| 215 | + f"(+{receipt.details['entries_appended']} appended)") | |
| 216 | + | |
| 217 | + | |
| 218 | +def _ingest(args: argparse.Namespace) -> int: | |
| 219 | + workspace, policies_path, _ = _resolve(args) | |
| 220 | + assert workspace is not None | |
| 221 | + text = Path(args.source).read_text(encoding="utf-8") | |
| 222 | + | |
| 223 | + if args.llm: | |
| 224 | + from ingestion.extractor import ClaudeExtractor | |
| 225 | + extractor = ClaudeExtractor() | |
| 226 | + else: | |
| 227 | + extractor = MockExtractor() | |
| 228 | + | |
| 229 | + from decimal import Decimal | |
| 230 | + outcome = ingest(text, extractor, threshold=Decimal(args.threshold)) | |
| 231 | + | |
| 232 | + if outcome.route is Route.AUTO_APPROVED: | |
| 233 | + assert outcome.document is not None | |
| 234 | + key = _idempotency_key(text, "ingest") | |
| 235 | + try: | |
| 236 | + summary = _compile_and_post(workspace, policies_path, | |
| 237 | + outcome.document, key) | |
| 238 | + except CompilationError as exc: | |
| 239 | + _print_diags(exc.diagnostics) | |
| 240 | + return 1 | |
| 241 | + workspace.archive_document(args.source) | |
| 242 | + print(f"auto-approved (confidence {outcome.extraction.confidence}): {summary}") | |
| 243 | + return 0 | |
| 244 | + | |
| 245 | + queue = ApprovalQueue(workspace.root) | |
| 246 | + item_id = queue.submit(outcome, text) | |
| 247 | + print(f"routed to human review: {item_id}") | |
| 248 | + for reason in outcome.reasons: | |
| 249 | + print(f" reason: {reason}") | |
| 250 | + for error in outcome.validation_errors: | |
| 251 | + print(f" schema: {error}") | |
| 252 | + print(f"review with: air inbox --home {workspace.root} | " | |
| 253 | + f"air approve {item_id} --home {workspace.root} --approver <name>") | |
| 254 | + return 0 | |
| 255 | + | |
| 256 | + | |
| 257 | +def _inbox(args: argparse.Namespace) -> int: | |
| 258 | + workspace = Workspace.open(args.home) | |
| 259 | + items = ApprovalQueue(workspace.root).pending() | |
| 260 | + if not items: | |
| 261 | + print("inbox empty: nothing awaiting review") | |
| 262 | + return 0 | |
| 263 | + for item in items: | |
| 264 | + print(f"{item.id} confidence={item.confidence} " | |
| 265 | + f"events={len(item.events)}") | |
| 266 | + for reason in item.reasons: | |
| 267 | + print(f" reason: {reason}") | |
| 268 | + for error in item.validation_errors: | |
| 269 | + print(f" schema: {error}") | |
| 270 | + return 0 | |
| 271 | + | |
| 272 | + | |
| 273 | +def _approve(args: argparse.Namespace) -> int: | |
| 274 | + workspace, policies_path, _ = _resolve(args) | |
| 275 | + assert workspace is not None | |
| 276 | + queue = ApprovalQueue(workspace.root) | |
| 277 | + document = queue.approve(args.item_id, approver=args.approver) | |
| 278 | + key = _idempotency_key(args.item_id, "approve") | |
| 279 | + try: | |
| 280 | + summary = _compile_and_post(workspace, policies_path, document, key) | |
| 281 | + except CompilationError as exc: | |
| 282 | + _print_diags(exc.diagnostics) | |
| 283 | + return 1 | |
| 284 | + print(f"approved by {args.approver}: {summary}") | |
| 285 | + return 0 | |
| 286 | + | |
| 287 | + | |
| 288 | +def _reject(args: argparse.Namespace) -> int: | |
| 289 | + workspace = Workspace.open(args.home) | |
| 290 | + ApprovalQueue(workspace.root).reject(args.item_id, reason=args.reason) | |
| 291 | + print(f"rejected {args.item_id}") | |
| 292 | + return 0 | |
| 293 | + | |
| 294 | + | |
| 295 | +def _reconcile(args: argparse.Namespace) -> int: | |
| 296 | + from kernel.bank_formats import parse_statement | |
| 297 | + from kernel.reconcile import cash_movements, reconcile | |
| 298 | + | |
| 299 | + workspace = Workspace.open(args.home) | |
| 300 | + ledger = Ledger(path=workspace.ledger_path) | |
| 301 | + transactions = parse_statement(Path(args.statement)) | |
| 302 | + result = reconcile(transactions, | |
| 303 | + cash_movements(ledger, args.cash_account), | |
| 304 | + tolerance_days=args.tolerance_days) | |
| 305 | + print(result.render()) | |
| 306 | + return 0 if result.is_clean else 2 # 2: differences found (not an error) | |
| 307 | + | |
| 308 | + | |
| 309 | +def _audit(args: argparse.Namespace) -> int: | |
| 310 | + from kernel.audit import AuditLog | |
| 311 | + workspace = Workspace.open(args.home) | |
| 312 | + log = AuditLog(path=workspace.root / "audit.jsonl") | |
| 313 | + if not log.verify_chain(): | |
| 314 | + print("error: audit log hash chain verification FAILED", file=sys.stderr) | |
| 315 | + return 1 | |
| 316 | + if not log.records: | |
| 317 | + print("audit log empty: no syscalls recorded yet") | |
| 318 | + return 0 | |
| 319 | + for record in log.records: | |
| 320 | + status = "OK " if record["status"] == "ok" else "ERR" | |
| 321 | + detail = f" <- {record['detail']}" if record.get("detail") else "" | |
| 322 | + print(f"#{record['seq']:04d} {status} {record['actor']:<16} " | |
| 323 | + f"{record['syscall']:<20} {json.dumps(record['params'])}{detail}") | |
| 324 | + print(f"\n{len(log.records)} syscalls, hash chain VALID") | |
| 325 | + return 0 | |
| 326 | + | |
| 327 | + | |
| 328 | +def _report(args: argparse.Namespace) -> int: | |
| 329 | + ledger_path = ( | |
| 330 | + Workspace.open(args.home).ledger_path if args.home else Path(args.ledger) | |
| 331 | + ) | |
| 332 | + ledger = Ledger(path=Path(ledger_path)) | |
| 333 | + if not ledger.verify_chain(): | |
| 334 | + print("error: ledger hash chain verification FAILED", file=sys.stderr) | |
| 335 | + return 1 | |
| 336 | + print(REPORTS[args.name](ledger, args.format)) | |
| 337 | + return 0 | |
| 338 | + | |
| 339 | + | |
| 340 | +def _add_common(p: argparse.ArgumentParser, *, backend: bool = True) -> None: | |
| 341 | + p.add_argument("--home", help="AIR home directory (managed books)") | |
| 342 | + p.add_argument("--policies", help="ALSL policy set (default: the home's)") | |
| 343 | + p.add_argument("--ledger", default="books/ledger.jsonl", | |
| 344 | + help="ledger path when no --home is used") | |
| 345 | + if backend: | |
| 346 | + p.add_argument("--backend", choices=["native", "csv", "qbo-export"], | |
| 347 | + default="native") | |
| 348 | + p.add_argument("--out", default="out", | |
| 349 | + help="export directory when no --home is used") | |
| 350 | + | |
| 351 | + | |
| 352 | +def main(argv: list[str] | None = None) -> int: | |
| 353 | + parser = argparse.ArgumentParser( | |
| 354 | + prog="air", description="AIR — the language of accounting (standalone mode)" | |
| 355 | + ) | |
| 356 | + sub = parser.add_subparsers(dest="command", required=True) | |
| 357 | + | |
| 358 | + p = sub.add_parser("init", help="create a managed AIR home (your books directory)") | |
| 359 | + p.add_argument("--home", required=True) | |
| 360 | + p.add_argument("--name", default="my-books") | |
| 361 | + p.add_argument("--policies", help="default ALSL policy set for this home") | |
| 362 | + p.set_defaults(func=_init) | |
| 363 | + | |
| 364 | + p = sub.add_parser("status", help="show AIR home health: entries, chain, archives") | |
| 365 | + p.add_argument("--home", required=True) | |
| 366 | + p.set_defaults(func=_status) | |
| 367 | + | |
| 368 | + p = sub.add_parser("compile", help="compile AIR events and post to a backend") | |
| 369 | + p.add_argument("document") | |
| 370 | + _add_common(p) | |
| 371 | + p.add_argument("--optimize", action="store_true", | |
| 372 | + help="enable optimization passes: duplicate detection, " | |
| 373 | + "netting, payment fusion") | |
| 374 | + p.add_argument("--report", action="append", choices=sorted(REPORTS)) | |
| 375 | + p.add_argument("--format", choices=FORMATS, default="text") | |
| 376 | + p.set_defaults(func=_compile) | |
| 377 | + | |
| 378 | + p = sub.add_parser( | |
| 379 | + "recompile", | |
| 380 | + help="incremental compile: diff two AIR documents, post reversal + replacement", | |
| 381 | + ) | |
| 382 | + p.add_argument("old_document") | |
| 383 | + p.add_argument("new_document") | |
| 384 | + _add_common(p) | |
| 385 | + p.set_defaults(func=_recompile) | |
| 386 | + | |
| 387 | + p = sub.add_parser("verify", help="compile without posting; print diagnostics") | |
| 388 | + p.add_argument("document") | |
| 389 | + _add_common(p, backend=False) | |
| 390 | + p.set_defaults(func=_verify) | |
| 391 | + | |
| 392 | + p = sub.add_parser( | |
| 393 | + "ingest", | |
| 394 | + help="extract AIR events from a source document; route by confidence", | |
| 395 | + ) | |
| 396 | + p.add_argument("source", help="text file (invoice text, email, OCR output)") | |
| 397 | + p.add_argument("--home", required=True) | |
| 398 | + p.add_argument("--policies") | |
| 399 | + p.add_argument("--threshold", default=str(DEFAULT_CONFIDENCE_THRESHOLD), | |
| 400 | + help="auto-approve confidence threshold (default 0.85)") | |
| 401 | + p.add_argument("--llm", action="store_true", | |
| 402 | + help="use the Claude extractor (needs an Anthropic API key); " | |
| 403 | + "default is the offline mock extractor") | |
| 404 | + p.set_defaults(func=_ingest) | |
| 405 | + | |
| 406 | + p = sub.add_parser("inbox", help="list extractions awaiting human review") | |
| 407 | + p.add_argument("--home", required=True) | |
| 408 | + p.set_defaults(func=_inbox) | |
| 409 | + | |
| 410 | + p = sub.add_parser("approve", help="approve a pending extraction and post it") | |
| 411 | + p.add_argument("item_id") | |
| 412 | + p.add_argument("--home", required=True) | |
| 413 | + p.add_argument("--policies") | |
| 414 | + p.add_argument("--approver", required=True) | |
| 415 | + p.set_defaults(func=_approve) | |
| 416 | + | |
| 417 | + p = sub.add_parser("reject", help="reject a pending extraction") | |
| 418 | + p.add_argument("item_id") | |
| 419 | + p.add_argument("--home", required=True) | |
| 420 | + p.add_argument("--reason", default="") | |
| 421 | + p.set_defaults(func=_reject) | |
| 422 | + | |
| 423 | + p = sub.add_parser( | |
| 424 | + "reconcile", | |
| 425 | + help="match a bank statement (camt.053 / MT940 / CSV) against the books", | |
| 426 | + ) | |
| 427 | + p.add_argument("statement") | |
| 428 | + p.add_argument("--home", required=True) | |
| 429 | + p.add_argument("--cash-account", default="1000") | |
| 430 | + p.add_argument("--tolerance-days", type=int, default=3) | |
| 431 | + p.set_defaults(func=_reconcile) | |
| 432 | + | |
| 433 | + p = sub.add_parser("audit", help="show and verify the agent syscall audit log") | |
| 434 | + p.add_argument("--home", required=True) | |
| 435 | + p.set_defaults(func=_audit) | |
| 436 | + | |
| 437 | + p = sub.add_parser("report", help="generate a statement from the books") | |
| 438 | + p.add_argument("name", choices=sorted(REPORTS)) | |
| 439 | + p.add_argument("--home") | |
| 440 | + p.add_argument("--ledger", default="books/ledger.jsonl") | |
| 441 | + p.add_argument("--format", choices=FORMATS, default="text") | |
| 442 | + p.set_defaults(func=_report) | |
| 443 | + | |
| 444 | + args = parser.parse_args(argv) | |
| 445 | + try: | |
| 446 | + return int(args.func(args)) | |
| 447 | + except WorkspaceError as exc: | |
| 448 | + print(f"error: {exc}", file=sys.stderr) | |
| 449 | + return 1 | |
| 450 | + | |
| 451 | + | |
| 452 | +if __name__ == "__main__": | |
| 453 | + raise SystemExit(main()) | |
added
sdk/demo_agent.py
+92 −0
@@ -0,0 +1,92 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : demo_agent.py | |
| 6 | +# Description : Demonstration agent — books a month of activity using ONLY syscalls. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Demo agent (Phase 5). | |
| 9 | + | |
| 10 | +A scripted stand-in for an AI agent: it records a small month of business | |
| 11 | +using exclusively the syscall interface — it never sees the ledger file, the | |
| 12 | +compiler internals, or the account codes. Every call it makes lands in the | |
| 13 | +hash-chained audit log. | |
| 14 | + | |
| 15 | +Run: python -m sdk.demo_agent --home books | |
| 16 | +""" | |
| 17 | +from __future__ import annotations | |
| 18 | + | |
| 19 | +import argparse | |
| 20 | + | |
| 21 | +from sdk.syscalls import AirKernel, SyscallError | |
| 22 | + | |
| 23 | + | |
| 24 | +def run(home: str, actor: str = "agent:demo") -> int: | |
| 25 | + kernel = AirKernel(home, actor=actor) | |
| 26 | + | |
| 27 | + print(f"[{actor}] staging January's events via CreateEconomicEvent...") | |
| 28 | + kernel.create_economic_event({ | |
| 29 | + "id": "evt_agent_owner", "type": "OwnerContribution", | |
| 30 | + "date": "2026-01-02", "description": "Initial owner investment", | |
| 31 | + "amount": {"amount": "15000.00", "currency": "CAD"}, | |
| 32 | + }) | |
| 33 | + kernel.create_economic_event({ | |
| 34 | + "id": "evt_agent_sale", "type": "Sale", "date": "2026-01-15", | |
| 35 | + "description": "Consulting engagement", | |
| 36 | + "amount": {"amount": "2500.00", "currency": "CAD"}, | |
| 37 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 38 | + }) | |
| 39 | + kernel.create_economic_event({ | |
| 40 | + "id": "evt_agent_purchase", "type": "Purchase", "date": "2026-01-18", | |
| 41 | + "description": "Office supplies (cash)", | |
| 42 | + "amount": {"amount": "180.00", "currency": "CAD"}, | |
| 43 | + "payment": {"immediate": True}, | |
| 44 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 45 | + }) | |
| 46 | + | |
| 47 | + print(f"[{actor}] Validate...") | |
| 48 | + diagnostics = kernel.validate() | |
| 49 | + for d in diagnostics: | |
| 50 | + print(" " + d.render().splitlines()[0]) | |
| 51 | + | |
| 52 | + print(f"[{actor}] Post...") | |
| 53 | + receipt = kernel.post() | |
| 54 | + print(f" posted {receipt['entries']} entries " | |
| 55 | + f"(+{receipt['appended']} appended, key {receipt['idempotency_key']})") | |
| 56 | + | |
| 57 | + print(f"[{actor}] oops — the purchase was a duplicate. Reverse...") | |
| 58 | + contra = kernel.reverse("je_evt_agent_purchase") | |
| 59 | + print(f" reversal entry: {contra}") | |
| 60 | + | |
| 61 | + print(f"[{actor}] ClosePeriod 2026-01...") | |
| 62 | + kernel.close_period("2026-01") | |
| 63 | + try: | |
| 64 | + kernel.create_economic_event({ | |
| 65 | + "id": "evt_agent_late", "type": "Sale", "date": "2026-01-31", | |
| 66 | + "amount": {"amount": "10.00", "currency": "CAD"}, | |
| 67 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 68 | + }) | |
| 69 | + kernel.post() | |
| 70 | + except SyscallError as exc: | |
| 71 | + print(f" refused as expected: {str(exc).splitlines()[0]}") | |
| 72 | + | |
| 73 | + print(f"[{actor}] GenerateReport trial-balance...") | |
| 74 | + print(kernel.generate_report("trial-balance")) | |
| 75 | + | |
| 76 | + ok = kernel.audit.verify_chain() | |
| 77 | + print(f"audit log: {len(kernel.audit.records)} syscalls recorded, " | |
| 78 | + f"chain {'VALID' if ok else 'BROKEN'}") | |
| 79 | + return 0 if ok else 1 | |
| 80 | + | |
| 81 | + | |
| 82 | +def main() -> int: | |
| 83 | + parser = argparse.ArgumentParser( | |
| 84 | + description="AIR demo agent — keeps books through syscalls only") | |
| 85 | + parser.add_argument("--home", required=True) | |
| 86 | + parser.add_argument("--actor", default="agent:demo") | |
| 87 | + args = parser.parse_args() | |
| 88 | + return run(args.home, args.actor) | |
| 89 | + | |
| 90 | + | |
| 91 | +if __name__ == "__main__": | |
| 92 | + raise SystemExit(main()) | |
added
sdk/syscalls.py
+230 −0
@@ -0,0 +1,230 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : syscalls.py | |
| 6 | +# Description : The agent SDK — accounting syscalls; agents NEVER touch the ledger directly. | |
| 7 | +# ============================================================================= | |
| 8 | +"""AIR syscalls (CLAUDE.md §3.5). | |
| 9 | + | |
| 10 | +AI agents do not get the ledger, the compiler internals, or the files — | |
| 11 | +they get THIS interface and nothing else: | |
| 12 | + | |
| 13 | + CreateEconomicEvent | Validate | Compile | Post | Reverse | |
| 14 | + ClosePeriod | GenerateReport (Merge/Reconcile arrive in Phase 6) | |
| 15 | + | |
| 16 | +Guarantees: | |
| 17 | +- every call — successful or failed — appends one record to the hash-chained | |
| 18 | + audit log (who, what, params digest, outcome); | |
| 19 | +- drafts are staged in the AIR home and only reach the books through the | |
| 20 | + deterministic compiler (Post); | |
| 21 | +- posting into a closed period is refused with a compiler-grade diagnostic; | |
| 22 | +- corrections are reversals (Reverse), never edits. | |
| 23 | +""" | |
| 24 | +from __future__ import annotations | |
| 25 | + | |
| 26 | +import hashlib | |
| 27 | +import json | |
| 28 | +from datetime import datetime, timezone | |
| 29 | +from pathlib import Path | |
| 30 | +from typing import Any, Callable | |
| 31 | + | |
| 32 | +from aic.compiler import compile_document | |
| 33 | +from aic.diagnostics import Diagnostic | |
| 34 | +from alsl.loader import load_policy_set | |
| 35 | +from core.events import AirDocument, EconomicEvent | |
| 36 | +from core.journal import CompiledJournal | |
| 37 | +from kernel.audit import AuditLog | |
| 38 | +from kernel.ledger import Ledger | |
| 39 | +from kernel.reporting import REPORTS | |
| 40 | +from kernel.workspace import Workspace | |
| 41 | + | |
| 42 | +_AUTHOR = "Simon-Pierre Boucher <contact@spboucher.ai>" | |
| 43 | + | |
| 44 | + | |
| 45 | +class SyscallError(RuntimeError): | |
| 46 | + """A syscall was refused; the refusal is in the audit log.""" | |
| 47 | + | |
| 48 | + | |
| 49 | +class AirKernel: | |
| 50 | + """One agent's handle on the books. All access is syscalls.""" | |
| 51 | + | |
| 52 | + def __init__( | |
| 53 | + self, | |
| 54 | + home: str | Path | Workspace, | |
| 55 | + actor: str, | |
| 56 | + policies: str | None = None, | |
| 57 | + clock: Callable[[], datetime] | None = None, | |
| 58 | + ): | |
| 59 | + self.workspace = home if isinstance(home, Workspace) else Workspace.open(home) | |
| 60 | + self.actor = actor | |
| 61 | + policies_path = policies or self.workspace.default_policies() | |
| 62 | + if not policies_path: | |
| 63 | + raise SyscallError("no policy set configured for this AIR home") | |
| 64 | + self.policies = load_policy_set(policies_path) | |
| 65 | + self.audit = AuditLog(path=self.workspace.root / "audit.jsonl") | |
| 66 | + self._ledger = Ledger(path=self.workspace.ledger_path) # private: no syscall exposes it raw | |
| 67 | + self._clock = clock or (lambda: datetime.now(timezone.utc)) | |
| 68 | + self._drafts_dir = self.workspace.root / "drafts" | |
| 69 | + self._drafts_dir.mkdir(parents=True, exist_ok=True) | |
| 70 | + | |
| 71 | + # -- audit plumbing ------------------------------------------------------------ | |
| 72 | + def _audited(self, syscall: str, params: dict[str, Any], | |
| 73 | + fn: Callable[[], Any]) -> Any: | |
| 74 | + try: | |
| 75 | + result = fn() | |
| 76 | + except Exception as exc: | |
| 77 | + # failures are audited too — an agent's denied action is evidence | |
| 78 | + self.audit.append(self.actor, syscall, params, "error", | |
| 79 | + detail=str(exc).splitlines()[0], | |
| 80 | + at=self._clock().isoformat()) | |
| 81 | + raise | |
| 82 | + self.audit.append(self.actor, syscall, params, "ok", | |
| 83 | + at=self._clock().isoformat()) | |
| 84 | + return result | |
| 85 | + | |
| 86 | + # -- drafts --------------------------------------------------------------------- | |
| 87 | + def _draft_paths(self) -> list[Path]: | |
| 88 | + return sorted(self._drafts_dir.glob("*.json")) | |
| 89 | + | |
| 90 | + def _load_drafts(self) -> AirDocument: | |
| 91 | + events = [json.loads(p.read_text(encoding="utf-8"))["event"] | |
| 92 | + for p in self._draft_paths()] | |
| 93 | + if not events: | |
| 94 | + raise SyscallError("no draft events staged; call create_economic_event first") | |
| 95 | + return AirDocument.model_validate({"events": events}) | |
| 96 | + | |
| 97 | + # -- syscalls --------------------------------------------------------------------- | |
| 98 | + def create_economic_event(self, event_data: dict[str, Any]) -> str: | |
| 99 | + """Stage a draft economic event. Validated against the AIR schema now; | |
| 100 | + it reaches the books only via post().""" | |
| 101 | + def run() -> str: | |
| 102 | + event = EconomicEvent.model_validate(event_data) | |
| 103 | + payload = {"_author": _AUTHOR, "staged_by": self.actor, | |
| 104 | + "event": json.loads(event.model_dump_json(exclude_none=True))} | |
| 105 | + (self._drafts_dir / f"{event.id}.json").write_text( | |
| 106 | + json.dumps(payload, indent=2), encoding="utf-8") | |
| 107 | + return event.id | |
| 108 | + return self._audited( | |
| 109 | + "CreateEconomicEvent", | |
| 110 | + {"event_id": event_data.get("id"), "type": event_data.get("type")}, | |
| 111 | + run, | |
| 112 | + ) | |
| 113 | + | |
| 114 | + def validate(self) -> list[Diagnostic]: | |
| 115 | + """Compile the staged drafts without posting; returns diagnostics.""" | |
| 116 | + def run() -> list[Diagnostic]: | |
| 117 | + document = self._load_drafts() | |
| 118 | + _, diagnostics = compile_document(document, self.policies) | |
| 119 | + return diagnostics | |
| 120 | + return self._audited("Validate", {"drafts": len(self._draft_paths())}, run) | |
| 121 | + | |
| 122 | + def compile(self) -> CompiledJournal: | |
| 123 | + """Deterministically compile the staged drafts (no posting).""" | |
| 124 | + def run() -> CompiledJournal: | |
| 125 | + document = self._load_drafts() | |
| 126 | + journal, _ = compile_document(document, self.policies) | |
| 127 | + return journal | |
| 128 | + return self._audited("Compile", {"drafts": len(self._draft_paths())}, run) | |
| 129 | + | |
| 130 | + def post(self, optimize: bool = False) -> dict[str, Any]: | |
| 131 | + """Compile the drafts and append the entries to the books. | |
| 132 | + | |
| 133 | + Refuses events dated in a closed period. Idempotent: the key is the | |
| 134 | + content hash of the staged drafts. optimize=True applies the Phase 6 | |
| 135 | + passes (duplicate detection, netting, fusion) before posting. | |
| 136 | + """ | |
| 137 | + def run() -> dict[str, Any]: | |
| 138 | + document = self._load_drafts() | |
| 139 | + self._refuse_closed_periods(document) | |
| 140 | + journal, _ = compile_document(document, self.policies, | |
| 141 | + optimize=optimize) | |
| 142 | + key = hashlib.sha256( | |
| 143 | + document.model_dump_json().encode("utf-8")).hexdigest()[:16] | |
| 144 | + appended = self._ledger.post_journal(journal, key) | |
| 145 | + # archive the posted drafts; the staging area empties | |
| 146 | + archive = self.workspace.root / "documents" | |
| 147 | + archive.mkdir(exist_ok=True) | |
| 148 | + for path in self._draft_paths(): | |
| 149 | + path.rename(archive / f"posted_{key}_{path.name}") | |
| 150 | + return {"entries": len(journal.entries), "appended": appended, | |
| 151 | + "idempotency_key": key, | |
| 152 | + "entry_ids": [e.id for e in journal.entries]} | |
| 153 | + return self._audited( | |
| 154 | + "Merge" if optimize else "Post", | |
| 155 | + {"drafts": len(self._draft_paths()), "optimize": optimize}, | |
| 156 | + run, | |
| 157 | + ) | |
| 158 | + | |
| 159 | + def merge(self) -> dict[str, Any]: | |
| 160 | + """Post the drafts with optimization: duplicates flagged, refunds | |
| 161 | + netted against their sales, identical payments fused into batches.""" | |
| 162 | + return self.post(optimize=True) | |
| 163 | + | |
| 164 | + def reverse(self, entry_id: str) -> str: | |
| 165 | + """Append the exact contra of a posted entry. Never deletes.""" | |
| 166 | + def run() -> str: | |
| 167 | + try: | |
| 168 | + contra = self._ledger.reverse_entry( | |
| 169 | + entry_id, f"rev:{self.actor}:{entry_id}") | |
| 170 | + except KeyError as exc: | |
| 171 | + raise SyscallError(str(exc)) from None | |
| 172 | + return contra.id | |
| 173 | + return self._audited("Reverse", {"entry_id": entry_id}, run) | |
| 174 | + | |
| 175 | + def close_period(self, period: str) -> None: | |
| 176 | + """Close an accounting period ("YYYY-MM"); later posts into it are refused.""" | |
| 177 | + def run() -> None: | |
| 178 | + if len(period) != 7 or period[4] != "-": | |
| 179 | + raise SyscallError(f"invalid period '{period}': use YYYY-MM") | |
| 180 | + meta = self.workspace.meta | |
| 181 | + closed = sorted(set(meta.get("closed_periods", []) or []) | {period}) | |
| 182 | + meta["closed_periods"] = closed | |
| 183 | + (self.workspace.root / "meta.json").write_text( | |
| 184 | + json.dumps(meta, indent=2), encoding="utf-8") | |
| 185 | + return self._audited("ClosePeriod", {"period": period}, run) | |
| 186 | + | |
| 187 | + def reconcile(self, statement_path: str | Path, | |
| 188 | + cash_account: str = "1000", | |
| 189 | + tolerance_days: int = 3) -> dict[str, Any]: | |
| 190 | + """Match a bank statement (camt.053 / MT940 / CSV) against the books' | |
| 191 | + cash movements. Returns the summary; the full report is in `render`.""" | |
| 192 | + def run() -> dict[str, Any]: | |
| 193 | + from kernel.bank_formats import parse_statement | |
| 194 | + from kernel.reconcile import cash_movements, reconcile | |
| 195 | + | |
| 196 | + transactions = parse_statement(Path(statement_path)) | |
| 197 | + movements = cash_movements(self._ledger, cash_account) | |
| 198 | + result = reconcile(transactions, movements, | |
| 199 | + tolerance_days=tolerance_days) | |
| 200 | + summary: dict[str, Any] = dict(result.summary()) | |
| 201 | + summary["clean"] = result.is_clean | |
| 202 | + summary["render"] = result.render() | |
| 203 | + return summary | |
| 204 | + return self._audited( | |
| 205 | + "Reconcile", | |
| 206 | + {"statement": Path(statement_path).name, | |
| 207 | + "cash_account": cash_account}, | |
| 208 | + run, | |
| 209 | + ) | |
| 210 | + | |
| 211 | + def generate_report(self, name: str, fmt: str = "text") -> str: | |
| 212 | + def run() -> str: | |
| 213 | + if name not in REPORTS: | |
| 214 | + raise SyscallError( | |
| 215 | + f"unknown report '{name}'; available: {sorted(REPORTS)}") | |
| 216 | + return REPORTS[name](self._ledger, fmt) | |
| 217 | + return self._audited("GenerateReport", {"report": name, "format": fmt}, run) | |
| 218 | + | |
| 219 | + # -- helpers -------------------------------------------------------------------- | |
| 220 | + def _refuse_closed_periods(self, document: AirDocument) -> None: | |
| 221 | + closed = set(self.workspace.meta.get("closed_periods", []) or []) | |
| 222 | + for event in document.events: | |
| 223 | + period = event.date.strftime("%Y-%m") | |
| 224 | + if period in closed: | |
| 225 | + raise SyscallError( | |
| 226 | + f"error[AIR-E700]: event '{event.id}' is dated {event.date} " | |
| 227 | + f"but period {period} is closed\n" | |
| 228 | + f" help: date the correction in an open period and reference " | |
| 229 | + f"the original via related_event" | |
| 230 | + ) | |
added
tests/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
tests/fixtures/bank_statement.csv
+4 −0
@@ -0,0 +1,4 @@ | ||
| 1 | +date,amount,currency,description,reference | |
| 2 | +2026-08-01,-1150.00,CAD,PAYMENT ACME INC INVOICE INV-2026-0042,INV-2026-0042 | |
| 3 | +2026-08-04,3449.93,CAD,CUSTOMER CUST 123 SALE 7781,E2E-SALE-7781 | |
| 4 | +2026-08-04,-42.00,CAD,BANK SERVICE FEE,FEE-0804 | |
added
tests/fixtures/bank_statement.mt940
+9 −0
@@ -0,0 +1,9 @@ | ||
| 1 | +:20:AIR-STMT-0805 | |
| 2 | +:25:BOFCCAM2/00112233445 | |
| 3 | +:28C:151/1 | |
| 4 | +:60F:C260731CAD25000,00 | |
| 5 | +:61:2608010801D1150,00NTRFINV-2026-0042//BKREF001 | |
| 6 | +:86:PAYMENT ACME INC INVOICE INV-2026-0042 | |
| 7 | +:61:2608040804C3449,93NTRFE2E-SALE-7781//BKREF002 | |
| 8 | +:86:CUSTOMER CUST 123 SALE 7781 | |
| 9 | +:62F:C260804CAD27299,93 | |
added
tests/fixtures/bank_statement.xml
+64 −0
@@ -0,0 +1,64 @@ | ||
| 1 | +<?xml version="1.0" encoding="UTF-8"?> | |
| 2 | +<!-- Projet : AIR | Auteur : Simon-Pierre Boucher | contact@spboucher.ai | |
| 3 | + Fixture: camt.053.001.02 statement (structure per docs/research/bank-statement-formats.md) --> | |
| 4 | +<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02"> | |
| 5 | + <BkToCstmrStmt> | |
| 6 | + <GrpHdr> | |
| 7 | + <MsgId>AIR-STMT-20260805-001</MsgId> | |
| 8 | + <CreDtTm>2026-08-05T06:00:00</CreDtTm> | |
| 9 | + </GrpHdr> | |
| 10 | + <Stmt> | |
| 11 | + <Id>STMT-2026-0151</Id> | |
| 12 | + <ElctrncSeqNb>151</ElctrncSeqNb> | |
| 13 | + <CreDtTm>2026-08-05T06:00:00</CreDtTm> | |
| 14 | + <Acct> | |
| 15 | + <Id><Othr><Id>00112233445</Id></Othr></Id> | |
| 16 | + <Ccy>CAD</Ccy> | |
| 17 | + </Acct> | |
| 18 | + <Bal> | |
| 19 | + <Tp><CdOrPrtry><Cd>OPBD</Cd></CdOrPrtry></Tp> | |
| 20 | + <Amt Ccy="CAD">25000.00</Amt> | |
| 21 | + <CdtDbtInd>CRDT</CdtDbtInd> | |
| 22 | + <Dt><Dt>2026-07-31</Dt></Dt> | |
| 23 | + </Bal> | |
| 24 | + <Bal> | |
| 25 | + <Tp><CdOrPrtry><Cd>CLBD</Cd></CdOrPrtry></Tp> | |
| 26 | + <Amt Ccy="CAD">27299.93</Amt> | |
| 27 | + <CdtDbtInd>CRDT</CdtDbtInd> | |
| 28 | + <Dt><Dt>2026-08-04</Dt></Dt> | |
| 29 | + </Bal> | |
| 30 | + <Ntry> | |
| 31 | + <NtryRef>BKREF001</NtryRef> | |
| 32 | + <Amt Ccy="CAD">1150.00</Amt> | |
| 33 | + <CdtDbtInd>DBIT</CdtDbtInd> | |
| 34 | + <Sts>BOOK</Sts> | |
| 35 | + <BookgDt><Dt>2026-08-01</Dt></BookgDt> | |
| 36 | + <ValDt><Dt>2026-08-01</Dt></ValDt> | |
| 37 | + <BkTxCd><Prtry><Cd>NTRF</Cd></Prtry></BkTxCd> | |
| 38 | + <NtryDtls> | |
| 39 | + <TxDtls> | |
| 40 | + <Refs><EndToEndId>INV-2026-0042</EndToEndId></Refs> | |
| 41 | + <RmtInf><Ustrd>PAYMENT ACME INC INVOICE INV-2026-0042</Ustrd></RmtInf> | |
| 42 | + </TxDtls> | |
| 43 | + </NtryDtls> | |
| 44 | + <AddtlNtryInf>Supplier payment Acme Inc.</AddtlNtryInf> | |
| 45 | + </Ntry> | |
| 46 | + <Ntry> | |
| 47 | + <NtryRef>BKREF002</NtryRef> | |
| 48 | + <Amt Ccy="CAD">3449.93</Amt> | |
| 49 | + <CdtDbtInd>CRDT</CdtDbtInd> | |
| 50 | + <Sts>BOOK</Sts> | |
| 51 | + <BookgDt><Dt>2026-08-04</Dt></BookgDt> | |
| 52 | + <ValDt><Dt>2026-08-04</Dt></ValDt> | |
| 53 | + <BkTxCd><Prtry><Cd>NTRF</Cd></Prtry></BkTxCd> | |
| 54 | + <NtryDtls> | |
| 55 | + <TxDtls> | |
| 56 | + <Refs><EndToEndId>E2E-SALE-7781</EndToEndId></Refs> | |
| 57 | + <RmtInf><Ustrd>CUSTOMER CUST 123 SALE 7781</Ustrd></RmtInf> | |
| 58 | + </TxDtls> | |
| 59 | + </NtryDtls> | |
| 60 | + <AddtlNtryInf>Customer payment, sale 7781</AddtlNtryInf> | |
| 61 | + </Ntry> | |
| 62 | + </Stmt> | |
| 63 | + </BkToCstmrStmt> | |
| 64 | +</Document> | |
added
tests/fixtures/demo_document.yaml
+72 −0
@@ -0,0 +1,72 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | |
| 3 | +# Contact : contact@spboucher.ai | |
| 4 | +# | |
| 5 | +# Demo AIR document: a small first month of business, standalone mode. | |
| 6 | +# All amounts are strings — bare YAML floats are rejected by the schema. | |
| 7 | + | |
| 8 | +air_version: "0.1" | |
| 9 | +events: | |
| 10 | + - id: evt_demo_owner | |
| 11 | + type: OwnerContribution | |
| 12 | + date: 2026-01-02 | |
| 13 | + description: "Initial owner investment" | |
| 14 | + amount: {amount: "20000.00", currency: CAD} | |
| 15 | + | |
| 16 | + - id: evt_demo_loan | |
| 17 | + type: LoanReceived | |
| 18 | + date: 2026-01-05 | |
| 19 | + description: "Working-capital bank loan" | |
| 20 | + amount: {amount: "15000.00", currency: CAD} | |
| 21 | + | |
| 22 | + - id: evt_demo_equipment | |
| 23 | + type: Purchase | |
| 24 | + date: 2026-01-08 | |
| 25 | + description: "Workshop CNC machine" | |
| 26 | + parties: {buyer: "company:acme", vendor: "vendor:machinco"} | |
| 27 | + amount: {amount: "8000.00", currency: CAD} | |
| 28 | + tax: {jurisdiction: CA-QC} | |
| 29 | + | |
| 30 | + - id: evt_demo_supplies | |
| 31 | + type: Purchase | |
| 32 | + date: 2026-01-10 | |
| 33 | + description: "Office supplies (cash)" | |
| 34 | + amount: {amount: "250.00", currency: CAD} | |
| 35 | + payment: {method: cash, immediate: true} | |
| 36 | + tax: {jurisdiction: CA-QC} | |
| 37 | + | |
| 38 | + - id: evt_demo_sale1 | |
| 39 | + type: Sale | |
| 40 | + date: 2026-01-15 | |
| 41 | + description: "3 standard chairs, Visa" | |
| 42 | + parties: {seller: "company:acme", buyer: "customer:cust_123"} | |
| 43 | + items: | |
| 44 | + - {sku: chair-std, qty: "3", unit_price: {amount: "333.33", currency: CAD}} | |
| 45 | + payment: {method: card.visa, immediate: true} | |
| 46 | + tax: {jurisdiction: CA-QC, codes: [GST, QST]} | |
| 47 | + meta: | |
| 48 | + source: {kind: invoice_pdf, uri: "s3://demo/inv-001.pdf", ocr_score: "0.97"} | |
| 49 | + llm: {model: "claude-fable-5", confidence: "0.93"} | |
| 50 | + policy_version: "2026.08" | |
| 51 | + | |
| 52 | + - id: evt_demo_sale2 | |
| 53 | + type: Sale | |
| 54 | + date: 2026-01-20 | |
| 55 | + description: "Consulting engagement (on account)" | |
| 56 | + amount: {amount: "5000.00", currency: CAD} | |
| 57 | + tax: {jurisdiction: CA-QC} | |
| 58 | + | |
| 59 | + - id: evt_demo_export | |
| 60 | + type: Sale | |
| 61 | + date: 2026-01-22 | |
| 62 | + description: "Export sale to US customer (zero-rated)" | |
| 63 | + amount: {amount: "2000.00", currency: USD} | |
| 64 | + tax: {jurisdiction: CA-QC, exempt: true} | |
| 65 | + fx: {rate: "1.3450", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-01-22} | |
| 66 | + | |
| 67 | + - id: evt_demo_collect | |
| 68 | + type: PaymentReceived | |
| 69 | + date: 2026-01-28 | |
| 70 | + description: "Customer pays consulting invoice" | |
| 71 | + related_event: evt_demo_sale2 | |
| 72 | + amount: {amount: "5748.75", currency: CAD} | |
added
tests/fixtures/invoice_high_confidence.txt
+12 −0
@@ -0,0 +1,12 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | contact@spboucher.ai | |
| 3 | +# Fixture: clean invoice text -> high-confidence extraction (mock format) | |
| 4 | + | |
| 5 | +type: Sale | |
| 6 | +id: evt_ing_sale_001 | |
| 7 | +date: 2026-07-15 | |
| 8 | +description: 3 standard chairs, invoice INV-001 | |
| 9 | +amount: 999.99 CAD | |
| 10 | +jurisdiction: CA-QC | |
| 11 | +immediate: true | |
| 12 | +confidence: 0.97 | |
added
tests/fixtures/invoice_low_confidence.txt
+11 −0
@@ -0,0 +1,11 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | contact@spboucher.ai | |
| 3 | +# Fixture: blurry scan -> low-confidence extraction (mock format) | |
| 4 | + | |
| 5 | +type: Purchase | |
| 6 | +id: evt_ing_purch_001 | |
| 7 | +date: 2026-07-16 | |
| 8 | +description: supplier bill, amount partially illegible | |
| 9 | +amount: 412.50 CAD | |
| 10 | +jurisdiction: CA-QC | |
| 11 | +confidence: 0.55 | |
added
tests/fixtures/invoice_realistic.txt
+24 −0
@@ -0,0 +1,24 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | contact@spboucher.ai | |
| 3 | +# Fixture: realistic free-form invoice text for the LIVE Claude extractor test. | |
| 4 | + | |
| 5 | +ACME FURNITURE INC. | |
| 6 | +1234 Rue Saint-Denis, Montréal, QC H2X 3K8 | |
| 7 | +GST No: 123456789 RT0001 | QST No: 1234567890 TQ0001 | |
| 8 | + | |
| 9 | +INVOICE INV-2026-0042 Date: July 28, 2026 | |
| 10 | + | |
| 11 | +Bill to: | |
| 12 | + Lakeside Cafe Ltd. | |
| 13 | + 88 Main Street, Longueuil, QC | |
| 14 | + | |
| 15 | +Description Qty Unit Price Amount | |
| 16 | +---------------------------------------------------------------------- | |
| 17 | +Standard oak chair (model CH-STD) 3 $333.33 $999.99 | |
| 18 | +---------------------------------------------------------------------- | |
| 19 | + Subtotal: $999.99 | |
| 20 | + GST (5%): $50.00 | |
| 21 | + QST (9.975%): $99.75 | |
| 22 | + TOTAL: $1149.74 | |
| 23 | + | |
| 24 | +Paid in full by Visa on July 28, 2026. Thank you for your business! | |
added
tests/golden/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
tests/golden/cases/fx_payments_equity.yaml
+181 −0
@@ -0,0 +1,181 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | |
| 3 | +# Contact : contact@spboucher.ai | |
| 4 | +# | |
| 5 | +# Golden cases — FX conversion (IAS 21 transaction-date rate), realized FX | |
| 6 | +# gains/losses on settlement, settlements, equity and loan events. | |
| 7 | +# FX semantics per docs/research/fx-handling.md and accounting-standards.md. | |
| 8 | +# Rates in events are observed input data (e.g. Bank of Canada Valet), never | |
| 9 | +# constants of the engine. | |
| 10 | + | |
| 11 | +cases: | |
| 12 | + - name: sale_usd_export | |
| 13 | + description: "1000.00 USD export sale (zero-rated), booked at the 1.3500 transaction-date rate" | |
| 14 | + air: | |
| 15 | + events: | |
| 16 | + - id: evt_sale_usd | |
| 17 | + type: Sale | |
| 18 | + date: 2026-07-02 | |
| 19 | + amount: {amount: "1000.00", currency: USD} | |
| 20 | + tax: {jurisdiction: CA-QC, exempt: true} | |
| 21 | + fx: {rate: "1.3500", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-07-02} | |
| 22 | + expected_entries: | |
| 23 | + - id: je_evt_sale_usd | |
| 24 | + lines: | |
| 25 | + - ["1100", debit, "1350.00", CAD] | |
| 26 | + - ["4000", credit, "1350.00", CAD] | |
| 27 | + | |
| 28 | + - name: purchase_eur_fx | |
| 29 | + description: "500.00 EUR foreign purchase at 1.4800: booked at 740.00 CAD" | |
| 30 | + air: | |
| 31 | + events: | |
| 32 | + - id: evt_purch_eur | |
| 33 | + type: Purchase | |
| 34 | + date: 2026-07-03 | |
| 35 | + amount: {amount: "500.00", currency: EUR} | |
| 36 | + tax: {jurisdiction: CA-QC, exempt: true} | |
| 37 | + fx: {rate: "1.4800", source: "bankofcanada.valet:FXEURCAD", rate_date: 2026-07-03} | |
| 38 | + expected_entries: | |
| 39 | + - id: je_evt_purch_eur | |
| 40 | + lines: | |
| 41 | + - ["5000", debit, "740.00", CAD] | |
| 42 | + - ["2000", credit, "740.00", CAD] | |
| 43 | + | |
| 44 | + - name: payment_received_cad | |
| 45 | + description: "Customer settles the 1149.75 receivable, same currency: no FX" | |
| 46 | + air: | |
| 47 | + events: | |
| 48 | + - id: evt_pay_in | |
| 49 | + type: PaymentReceived | |
| 50 | + date: 2026-07-30 | |
| 51 | + related_event: evt_sale_simple | |
| 52 | + amount: {amount: "1149.75", currency: CAD} | |
| 53 | + expected_entries: | |
| 54 | + - id: je_evt_pay_in | |
| 55 | + lines: | |
| 56 | + - ["1000", debit, "1149.75", CAD] | |
| 57 | + - ["1100", credit, "1149.75", CAD] | |
| 58 | + | |
| 59 | + - name: payment_sent_cad | |
| 60 | + description: "We settle the 459.90 payable, same currency: no FX" | |
| 61 | + air: | |
| 62 | + events: | |
| 63 | + - id: evt_pay_out | |
| 64 | + type: PaymentSent | |
| 65 | + date: 2026-07-31 | |
| 66 | + related_event: evt_purch_supplies | |
| 67 | + amount: {amount: "459.90", currency: CAD} | |
| 68 | + expected_entries: | |
| 69 | + - id: je_evt_pay_out | |
| 70 | + lines: | |
| 71 | + - ["2000", debit, "459.90", CAD] | |
| 72 | + - ["1000", credit, "459.90", CAD] | |
| 73 | + | |
| 74 | + - name: fx_gain_on_receipt | |
| 75 | + description: "USD sale booked at 1.3000; collected at 1.3500 -> realized FX gain 50.00" | |
| 76 | + air: | |
| 77 | + events: | |
| 78 | + - id: evt_sale_usd_g | |
| 79 | + type: Sale | |
| 80 | + date: 2026-06-01 | |
| 81 | + amount: {amount: "1000.00", currency: USD} | |
| 82 | + tax: {jurisdiction: CA-QC, exempt: true} | |
| 83 | + fx: {rate: "1.3000", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-06-01} | |
| 84 | + - id: evt_pay_usd_g | |
| 85 | + type: PaymentReceived | |
| 86 | + date: 2026-06-30 | |
| 87 | + related_event: evt_sale_usd_g | |
| 88 | + amount: {amount: "1000.00", currency: USD} | |
| 89 | + fx: {rate: "1.3500", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-06-30} | |
| 90 | + expected_entries: | |
| 91 | + - id: je_evt_sale_usd_g | |
| 92 | + lines: | |
| 93 | + - ["1100", debit, "1300.00", CAD] | |
| 94 | + - ["4000", credit, "1300.00", CAD] | |
| 95 | + - id: je_evt_pay_usd_g | |
| 96 | + lines: | |
| 97 | + - ["1000", debit, "1350.00", CAD] | |
| 98 | + - ["1100", credit, "1300.00", CAD] | |
| 99 | + - ["4500", credit, "50.00", CAD] | |
| 100 | + | |
| 101 | + - name: fx_loss_on_receipt | |
| 102 | + description: "USD sale booked at 1.3500; collected at 1.3000 -> realized FX loss 50.00" | |
| 103 | + air: | |
| 104 | + events: | |
| 105 | + - id: evt_sale_usd_l | |
| 106 | + type: Sale | |
| 107 | + date: 2026-06-05 | |
| 108 | + amount: {amount: "1000.00", currency: USD} | |
| 109 | + tax: {jurisdiction: CA-QC, exempt: true} | |
| 110 | + fx: {rate: "1.3500", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-06-05} | |
| 111 | + - id: evt_pay_usd_l | |
| 112 | + type: PaymentReceived | |
| 113 | + date: 2026-07-05 | |
| 114 | + related_event: evt_sale_usd_l | |
| 115 | + amount: {amount: "1000.00", currency: USD} | |
| 116 | + fx: {rate: "1.3000", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-07-05} | |
| 117 | + expected_entries: | |
| 118 | + - id: je_evt_sale_usd_l | |
| 119 | + lines: | |
| 120 | + - ["1100", debit, "1350.00", CAD] | |
| 121 | + - ["4000", credit, "1350.00", CAD] | |
| 122 | + - id: je_evt_pay_usd_l | |
| 123 | + lines: | |
| 124 | + - ["1000", debit, "1300.00", CAD] | |
| 125 | + - ["5500", debit, "50.00", CAD] | |
| 126 | + - ["1100", credit, "1350.00", CAD] | |
| 127 | + | |
| 128 | + - name: fx_loss_on_payment_sent | |
| 129 | + description: "2000 USD purchase booked at 1.3000 (AP 2600.00); paid at 1.3400 (2680.00) -> FX loss 80.00" | |
| 130 | + air: | |
| 131 | + events: | |
| 132 | + - id: evt_purch_usd | |
| 133 | + type: Purchase | |
| 134 | + date: 2026-06-10 | |
| 135 | + amount: {amount: "2000.00", currency: USD} | |
| 136 | + tax: {jurisdiction: CA-QC, exempt: true} | |
| 137 | + fx: {rate: "1.3000", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-06-10} | |
| 138 | + - id: evt_pay_usd_out | |
| 139 | + type: PaymentSent | |
| 140 | + date: 2026-07-10 | |
| 141 | + related_event: evt_purch_usd | |
| 142 | + amount: {amount: "2000.00", currency: USD} | |
| 143 | + fx: {rate: "1.3400", source: "bankofcanada.valet:FXUSDCAD", rate_date: 2026-07-10} | |
| 144 | + expected_entries: | |
| 145 | + - id: je_evt_purch_usd | |
| 146 | + lines: | |
| 147 | + - ["5000", debit, "2600.00", CAD] | |
| 148 | + - ["2000", credit, "2600.00", CAD] | |
| 149 | + - id: je_evt_pay_usd_out | |
| 150 | + lines: | |
| 151 | + - ["2000", debit, "2600.00", CAD] | |
| 152 | + - ["5500", debit, "80.00", CAD] | |
| 153 | + - ["1000", credit, "2680.00", CAD] | |
| 154 | + | |
| 155 | + - name: owner_contribution | |
| 156 | + description: "Owner injects 10000.00: cash up, equity up" | |
| 157 | + air: | |
| 158 | + events: | |
| 159 | + - id: evt_owner | |
| 160 | + type: OwnerContribution | |
| 161 | + date: 2026-01-01 | |
| 162 | + amount: {amount: "10000.00", currency: CAD} | |
| 163 | + expected_entries: | |
| 164 | + - id: je_evt_owner | |
| 165 | + lines: | |
| 166 | + - ["1000", debit, "10000.00", CAD] | |
| 167 | + - ["3000", credit, "10000.00", CAD] | |
| 168 | + | |
| 169 | + - name: loan_received | |
| 170 | + description: "Bank loan 25000.00: cash up, liability up" | |
| 171 | + air: | |
| 172 | + events: | |
| 173 | + - id: evt_loan | |
| 174 | + type: LoanReceived | |
| 175 | + date: 2026-01-02 | |
| 176 | + amount: {amount: "25000.00", currency: CAD} | |
| 177 | + expected_entries: | |
| 178 | + - id: je_evt_loan | |
| 179 | + lines: | |
| 180 | + - ["1000", debit, "25000.00", CAD] | |
| 181 | + - ["2500", credit, "25000.00", CAD] | |
added
tests/golden/cases/purchases.yaml
+78 −0
@@ -0,0 +1,78 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | |
| 3 | +# Contact : contact@spboucher.ai | |
| 4 | +# | |
| 5 | +# Golden cases — purchases: recoverable taxes (ITC/ITR) and the ALSL | |
| 6 | +# capitalization policy (>= 5000 CAD -> equipment asset). | |
| 7 | +# Rates & rounding per docs/research/canada-gst-qst.md (verified 2026-08-05). | |
| 8 | + | |
| 9 | +cases: | |
| 10 | + - name: purchase_expense_qc | |
| 11 | + description: "400.00 supplies on account: ITC 20.00, ITR 39.90 both recoverable (debited as receivables)" | |
| 12 | + air: | |
| 13 | + events: | |
| 14 | + - id: evt_purch_supplies | |
| 15 | + type: Purchase | |
| 16 | + date: 2026-07-10 | |
| 17 | + parties: {buyer: "company:acme", vendor: "vendor:papco"} | |
| 18 | + amount: {amount: "400.00", currency: CAD} | |
| 19 | + tax: {jurisdiction: CA-QC} | |
| 20 | + expected_entries: | |
| 21 | + - id: je_evt_purch_supplies | |
| 22 | + lines: | |
| 23 | + - ["5000", debit, "400.00", CAD] | |
| 24 | + - ["1210", debit, "20.00", CAD] | |
| 25 | + - ["1220", debit, "39.90", CAD] | |
| 26 | + - ["2000", credit, "459.90", CAD] | |
| 27 | + | |
| 28 | + - name: purchase_capitalized_qc | |
| 29 | + description: "6000.00 machine: >= 5000 threshold -> capitalized to equipment (ALSL classification policy)" | |
| 30 | + air: | |
| 31 | + events: | |
| 32 | + - id: evt_purch_machine | |
| 33 | + type: Purchase | |
| 34 | + date: 2026-07-11 | |
| 35 | + amount: {amount: "6000.00", currency: CAD} | |
| 36 | + tax: {jurisdiction: CA-QC} | |
| 37 | + expected_entries: | |
| 38 | + - id: je_evt_purch_machine | |
| 39 | + lines: | |
| 40 | + - ["1500", debit, "6000.00", CAD] | |
| 41 | + - ["1210", debit, "300.00", CAD] | |
| 42 | + - ["1220", debit, "598.50", CAD] | |
| 43 | + - ["2000", credit, "6898.50", CAD] | |
| 44 | + | |
| 45 | + - name: purchase_below_threshold_qc | |
| 46 | + description: "4999.99: one cent below the threshold -> stays an expense; QST 498.749 rounds to 498.75" | |
| 47 | + air: | |
| 48 | + events: | |
| 49 | + - id: evt_purch_below | |
| 50 | + type: Purchase | |
| 51 | + date: 2026-07-12 | |
| 52 | + amount: {amount: "4999.99", currency: CAD} | |
| 53 | + tax: {jurisdiction: CA-QC} | |
| 54 | + expected_entries: | |
| 55 | + - id: je_evt_purch_below | |
| 56 | + lines: | |
| 57 | + - ["5000", debit, "4999.99", CAD] | |
| 58 | + - ["1210", debit, "250.00", CAD] | |
| 59 | + - ["1220", debit, "498.75", CAD] | |
| 60 | + - ["2000", credit, "5748.74", CAD] | |
| 61 | + | |
| 62 | + - name: purchase_cash_qc | |
| 63 | + description: "100.00 paid cash immediately: QST 9.975 rounds to 9.98; credit cash not AP" | |
| 64 | + air: | |
| 65 | + events: | |
| 66 | + - id: evt_purch_cash | |
| 67 | + type: Purchase | |
| 68 | + date: 2026-07-13 | |
| 69 | + amount: {amount: "100.00", currency: CAD} | |
| 70 | + payment: {method: cash, immediate: true} | |
| 71 | + tax: {jurisdiction: CA-QC} | |
| 72 | + expected_entries: | |
| 73 | + - id: je_evt_purch_cash | |
| 74 | + lines: | |
| 75 | + - ["5000", debit, "100.00", CAD] | |
| 76 | + - ["1210", debit, "5.00", CAD] | |
| 77 | + - ["1220", debit, "9.98", CAD] | |
| 78 | + - ["1000", credit, "114.98", CAD] | |
added
tests/golden/cases/sales_and_refunds.yaml
+148 −0
@@ -0,0 +1,148 @@ | ||
| 1 | +# Projet : AIR — Accounting Intermediate Representation | |
| 2 | +# Auteur : Simon-Pierre Boucher | |
| 3 | +# Contact : contact@spboucher.ai | |
| 4 | +# | |
| 5 | +# Golden cases — sales and refunds under CA-QC / CA-ON / CA-AB tax policies. | |
| 6 | +# Expected figures follow docs/research/canada-gst-qst.md: | |
| 7 | +# GST 5%, QST 9.975% (both on the pre-GST price), half-up rounding to the | |
| 8 | +# cent per Excise Tax Act s. 165.2(2) / RQ IN-203-V. Verified 2026-08-05. | |
| 9 | + | |
| 10 | +cases: | |
| 11 | + - name: sale_simple_qc | |
| 12 | + description: "1000.00 CAD sale in Quebec on account: GST 50.00, QST 99.75 (research worked example)" | |
| 13 | + air: | |
| 14 | + events: | |
| 15 | + - id: evt_sale_simple | |
| 16 | + type: Sale | |
| 17 | + date: 2026-07-15 | |
| 18 | + parties: {seller: "company:acme", buyer: "customer:cust_001"} | |
| 19 | + amount: {amount: "1000.00", currency: CAD} | |
| 20 | + tax: {jurisdiction: CA-QC} | |
| 21 | + expected_entries: | |
| 22 | + - id: je_evt_sale_simple | |
| 23 | + lines: | |
| 24 | + - ["1100", debit, "1149.75", CAD] | |
| 25 | + - ["4000", credit, "1000.00", CAD] | |
| 26 | + - ["2310", credit, "50.00", CAD] | |
| 27 | + - ["2320", credit, "99.75", CAD] | |
| 28 | + | |
| 29 | + - name: sale_multi_item_qc | |
| 30 | + description: "CLAUDE.md flagship: 3 chairs at 333.33, paid by Visa immediately" | |
| 31 | + air: | |
| 32 | + events: | |
| 33 | + - id: evt_sale_chairs | |
| 34 | + type: Sale | |
| 35 | + date: 2026-07-20 | |
| 36 | + description: "3 standard chairs" | |
| 37 | + parties: {seller: "company:acme", buyer: "customer:cust_123"} | |
| 38 | + items: | |
| 39 | + - {sku: chair-std, qty: "3", unit_price: {amount: "333.33", currency: CAD}} | |
| 40 | + payment: {method: card.visa, immediate: true} | |
| 41 | + tax: {jurisdiction: CA-QC, codes: [GST, QST]} | |
| 42 | + expected_entries: | |
| 43 | + - id: je_evt_sale_chairs | |
| 44 | + lines: | |
| 45 | + - ["1000", debit, "1149.74", CAD] | |
| 46 | + - ["4000", credit, "999.99", CAD] | |
| 47 | + - ["2310", credit, "50.00", CAD] | |
| 48 | + - ["2320", credit, "99.75", CAD] | |
| 49 | + | |
| 50 | + - name: sale_rounding_qc | |
| 51 | + description: "19.99 sale: GST 0.9995 rounds UP to 1.00; QST 1.9940 rounds DOWN to 1.99 (statute worked example)" | |
| 52 | + air: | |
| 53 | + events: | |
| 54 | + - id: evt_sale_rounding | |
| 55 | + type: Sale | |
| 56 | + date: 2026-07-21 | |
| 57 | + amount: {amount: "19.99", currency: CAD} | |
| 58 | + tax: {jurisdiction: CA-QC} | |
| 59 | + expected_entries: | |
| 60 | + - id: je_evt_sale_rounding | |
| 61 | + lines: | |
| 62 | + - ["1100", debit, "22.98", CAD] | |
| 63 | + - ["4000", credit, "19.99", CAD] | |
| 64 | + - ["2310", credit, "1.00", CAD] | |
| 65 | + - ["2320", credit, "1.99", CAD] | |
| 66 | + | |
| 67 | + - name: sale_exempt_qc | |
| 68 | + description: "Exempt/zero-rated supply: no tax lines at all" | |
| 69 | + air: | |
| 70 | + events: | |
| 71 | + - id: evt_sale_exempt | |
| 72 | + type: Sale | |
| 73 | + date: 2026-07-22 | |
| 74 | + amount: {amount: "500.00", currency: CAD} | |
| 75 | + tax: {jurisdiction: CA-QC, exempt: true} | |
| 76 | + expected_entries: | |
| 77 | + - id: je_evt_sale_exempt | |
| 78 | + lines: | |
| 79 | + - ["1100", debit, "500.00", CAD] | |
| 80 | + - ["4000", credit, "500.00", CAD] | |
| 81 | + | |
| 82 | + - name: sale_ontario_hst | |
| 83 | + description: "Ontario sale: single HST 13%" | |
| 84 | + air: | |
| 85 | + events: | |
| 86 | + - id: evt_sale_on | |
| 87 | + type: Sale | |
| 88 | + date: 2026-07-23 | |
| 89 | + amount: {amount: "200.00", currency: CAD} | |
| 90 | + tax: {jurisdiction: CA-ON} | |
| 91 | + expected_entries: | |
| 92 | + - id: je_evt_sale_on | |
| 93 | + lines: | |
| 94 | + - ["1100", debit, "226.00", CAD] | |
| 95 | + - ["4000", credit, "200.00", CAD] | |
| 96 | + - ["2330", credit, "26.00", CAD] | |
| 97 | + | |
| 98 | + - name: sale_alberta_gst_only | |
| 99 | + description: "Alberta sale: GST 5% only, no provincial tax" | |
| 100 | + air: | |
| 101 | + events: | |
| 102 | + - id: evt_sale_ab | |
| 103 | + type: Sale | |
| 104 | + date: 2026-07-24 | |
| 105 | + amount: {amount: "300.00", currency: CAD} | |
| 106 | + tax: {jurisdiction: CA-AB} | |
| 107 | + expected_entries: | |
| 108 | + - id: je_evt_sale_ab | |
| 109 | + lines: | |
| 110 | + - ["1100", debit, "315.00", CAD] | |
| 111 | + - ["4000", credit, "300.00", CAD] | |
| 112 | + - ["2310", credit, "15.00", CAD] | |
| 113 | + | |
| 114 | + - name: refund_full_qc | |
| 115 | + description: "Full refund of the 1000.00 QC sale: mirror entry, taxes reversed" | |
| 116 | + air: | |
| 117 | + events: | |
| 118 | + - id: evt_refund_full | |
| 119 | + type: Refund | |
| 120 | + date: 2026-07-25 | |
| 121 | + related_event: evt_sale_simple | |
| 122 | + amount: {amount: "1000.00", currency: CAD} | |
| 123 | + tax: {jurisdiction: CA-QC} | |
| 124 | + expected_entries: | |
| 125 | + - id: je_evt_refund_full | |
| 126 | + lines: | |
| 127 | + - ["4000", debit, "1000.00", CAD] | |
| 128 | + - ["2310", debit, "50.00", CAD] | |
| 129 | + - ["2320", debit, "99.75", CAD] | |
| 130 | + - ["1100", credit, "1149.75", CAD] | |
| 131 | + | |
| 132 | + - name: refund_partial_qc | |
| 133 | + description: "Partial refund 250.00: GST 12.50; QST 24.9375 rounds to 24.94" | |
| 134 | + air: | |
| 135 | + events: | |
| 136 | + - id: evt_refund_partial | |
| 137 | + type: Refund | |
| 138 | + date: 2026-07-26 | |
| 139 | + related_event: evt_sale_simple | |
| 140 | + amount: {amount: "250.00", currency: CAD} | |
| 141 | + tax: {jurisdiction: CA-QC} | |
| 142 | + expected_entries: | |
| 143 | + - id: je_evt_refund_partial | |
| 144 | + lines: | |
| 145 | + - ["4000", debit, "250.00", CAD] | |
| 146 | + - ["2310", debit, "12.50", CAD] | |
| 147 | + - ["2320", debit, "24.94", CAD] | |
| 148 | + - ["1100", credit, "287.44", CAD] | |
added
tests/golden/test_golden.py
+93 −0
@@ -0,0 +1,93 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_golden.py | |
| 6 | +# Description : Golden tests — AIR documents compiled against expected journal entries. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Golden tests. | |
| 9 | + | |
| 10 | +Each case in tests/golden/cases/*.yaml holds an AIR document and the exact | |
| 11 | +journal entries the compiler must produce under the ca-qc-2026 policy set. | |
| 12 | +Expected figures are derived from cited official sources | |
| 13 | +(docs/research/canada-gst-qst.md, fx-handling.md) — never from memory. | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from decimal import Decimal | |
| 18 | +from pathlib import Path | |
| 19 | + | |
| 20 | +import pytest | |
| 21 | +import yaml | |
| 22 | + | |
| 23 | +from aic.compiler import compile_document | |
| 24 | +from alsl.loader import load_policy_set | |
| 25 | +from core.events import AirDocument | |
| 26 | + | |
| 27 | +ROOT = Path(__file__).resolve().parents[2] | |
| 28 | +CASES_DIR = Path(__file__).parent / "cases" | |
| 29 | +POLICY_FILE = ROOT / "alsl" / "policies" / "ca-qc-2026.yaml" | |
| 30 | + | |
| 31 | + | |
| 32 | +def _load_cases() -> list[dict]: | |
| 33 | + cases: list[dict] = [] | |
| 34 | + for path in sorted(CASES_DIR.glob("*.yaml")): | |
| 35 | + data = yaml.safe_load(path.read_text(encoding="utf-8")) | |
| 36 | + cases.extend(data["cases"]) | |
| 37 | + return cases | |
| 38 | + | |
| 39 | +CASES = _load_cases() | |
| 40 | + | |
| 41 | + | |
| 42 | +@pytest.fixture(scope="module") | |
| 43 | +def policies(): | |
| 44 | + return load_policy_set(POLICY_FILE) | |
| 45 | + | |
| 46 | + | |
| 47 | +def _lines_as_tuples(lines) -> list[tuple[str, str, Decimal, str]]: | |
| 48 | + return sorted( | |
| 49 | + (l.account.code, l.side.value, l.amount.amount, l.amount.currency) | |
| 50 | + for l in lines | |
| 51 | + ) | |
| 52 | + | |
| 53 | + | |
| 54 | +def _expected_as_tuples(raw) -> list[tuple[str, str, Decimal, str]]: | |
| 55 | + return sorted( | |
| 56 | + (str(acct), str(side), Decimal(str(amount)), str(ccy)) | |
| 57 | + for acct, side, amount, ccy in raw | |
| 58 | + ) | |
| 59 | + | |
| 60 | + | |
| 61 | +@pytest.mark.parametrize("case", CASES, ids=[c["name"] for c in CASES]) | |
| 62 | +def test_golden(case: dict, policies) -> None: | |
| 63 | + document = AirDocument.model_validate(case["air"]) | |
| 64 | + journal, _diags = compile_document(document, policies) | |
| 65 | + | |
| 66 | + entries = {e.id: e for e in journal.entries} | |
| 67 | + expected = {e["id"]: e for e in case["expected_entries"]} | |
| 68 | + | |
| 69 | + assert sorted(entries) == sorted(expected), ( | |
| 70 | + f"{case['name']}: produced entries {sorted(entries)} " | |
| 71 | + f"!= expected {sorted(expected)}" | |
| 72 | + ) | |
| 73 | + for entry_id, exp in expected.items(): | |
| 74 | + got = _lines_as_tuples(entries[entry_id].lines) | |
| 75 | + want = _expected_as_tuples(exp["lines"]) | |
| 76 | + assert got == want, ( | |
| 77 | + f"{case['name']} / {entry_id}:\n got: {got}\n want: {want}" | |
| 78 | + ) | |
| 79 | + | |
| 80 | + | |
| 81 | +def test_all_golden_lines_have_provenance(policies) -> None: | |
| 82 | + """Traceability: every posted line points into the provenance graph and | |
| 83 | + traces back to a subtotal rooted in the source event.""" | |
| 84 | + for case in CASES: | |
| 85 | + document = AirDocument.model_validate(case["air"]) | |
| 86 | + journal, _ = compile_document(document, policies) | |
| 87 | + for entry in journal.entries: | |
| 88 | + for line in entry.lines: | |
| 89 | + assert line.provenance_id is not None | |
| 90 | + chain = journal.provenance.trace(line.provenance_id) | |
| 91 | + roots = [n for n in chain if n.kind == "event_subtotal"] | |
| 92 | + assert roots, f"{case['name']}: line has no event_subtotal root" | |
| 93 | + assert roots[-1].source_ref == entry.source_event_id | |
added
tests/property/__init__.py
+7 −0
@@ -0,0 +1,7 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : __init__.py | |
| 6 | +# Description : Package marker. | |
| 7 | +# ============================================================================= | |
added
tests/property/test_balance_invariant.py
+118 −0
@@ -0,0 +1,118 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_balance_invariant.py | |
| 6 | +# Description : Property-based tests — the double-entry invariant holds for ALL inputs. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Property-based tests (hypothesis). | |
| 9 | + | |
| 10 | +For arbitrary well-formed AIR documents: | |
| 11 | +1. every compiled entry balances (sum debits == sum credits per currency); | |
| 12 | +2. the accounting equation residual is zero; | |
| 13 | +3. compilation is deterministic (same input => identical journal). | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from datetime import date | |
| 18 | +from decimal import Decimal | |
| 19 | +from pathlib import Path | |
| 20 | + | |
| 21 | +from hypothesis import given, settings | |
| 22 | +from hypothesis import strategies as st | |
| 23 | + | |
| 24 | +from aic.compiler import compile_document | |
| 25 | +from alsl.loader import load_policy_set | |
| 26 | +from core.events import AirDocument | |
| 27 | +from core.invariant import accounting_equation_residual, entry_imbalances | |
| 28 | + | |
| 29 | +ROOT = Path(__file__).resolve().parents[2] | |
| 30 | +POLICIES = load_policy_set(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 31 | + | |
| 32 | +# amounts: positive, exact 2-decimal values up to 10 million | |
| 33 | +amounts = st.integers(min_value=1, max_value=1_000_000_000).map( | |
| 34 | + lambda cents: str(Decimal(cents) / 100) | |
| 35 | +) | |
| 36 | +jurisdictions = st.sampled_from(["CA-QC", "CA-ON", "CA-AB"]) | |
| 37 | + | |
| 38 | + | |
| 39 | +@st.composite | |
| 40 | +def events(draw, index: int = 0): | |
| 41 | + etype = draw(st.sampled_from( | |
| 42 | + ["Sale", "Purchase", "Refund", "OwnerContribution", "LoanReceived", | |
| 43 | + "PaymentReceived", "PaymentSent"] | |
| 44 | + )) | |
| 45 | + event: dict = { | |
| 46 | + "id": f"evt_{draw(st.integers(min_value=0, max_value=10**9))}_{index}", | |
| 47 | + "type": etype, | |
| 48 | + "date": date(2026, 7, 15), | |
| 49 | + "amount": {"amount": draw(amounts), "currency": "CAD"}, | |
| 50 | + } | |
| 51 | + if etype in ("Sale", "Purchase", "Refund"): | |
| 52 | + event["tax"] = { | |
| 53 | + "jurisdiction": draw(jurisdictions), | |
| 54 | + "exempt": draw(st.booleans()), | |
| 55 | + } | |
| 56 | + event["payment"] = {"immediate": draw(st.booleans())} | |
| 57 | + return event | |
| 58 | + | |
| 59 | + | |
| 60 | +documents = st.lists(events(), min_size=1, max_size=8).map( | |
| 61 | + lambda evs: { | |
| 62 | + "events": [ | |
| 63 | + {**e, "id": f"{e['id']}_{i}"} for i, e in enumerate(evs) | |
| 64 | + ] | |
| 65 | + } | |
| 66 | +) | |
| 67 | + | |
| 68 | + | |
| 69 | +@given(documents) | |
| 70 | +@settings(max_examples=200, deadline=None) | |
| 71 | +def test_every_entry_balances_and_equation_holds(doc_raw) -> None: | |
| 72 | + document = AirDocument.model_validate(doc_raw) | |
| 73 | + journal, _ = compile_document(document, POLICIES) | |
| 74 | + assert len(journal.entries) == len(document.events) | |
| 75 | + for entry in journal.entries: | |
| 76 | + assert entry_imbalances(entry) == {}, f"unbalanced entry {entry.id}" | |
| 77 | + assert accounting_equation_residual(journal.entries) == {} | |
| 78 | + | |
| 79 | + | |
| 80 | +@given(documents) | |
| 81 | +@settings(max_examples=50, deadline=None) | |
| 82 | +def test_compilation_is_deterministic(doc_raw) -> None: | |
| 83 | + document = AirDocument.model_validate(doc_raw) | |
| 84 | + journal_a, _ = compile_document(document, POLICIES) | |
| 85 | + journal_b, _ = compile_document(document, POLICIES) | |
| 86 | + | |
| 87 | + def snapshot(journal): | |
| 88 | + return [ | |
| 89 | + (e.id, e.date, e.description, | |
| 90 | + [(l.account.code, l.side.value, str(l.amount.amount), | |
| 91 | + l.amount.currency, l.memo) for l in e.lines]) | |
| 92 | + for e in journal.entries | |
| 93 | + ] | |
| 94 | + | |
| 95 | + assert snapshot(journal_a) == snapshot(journal_b) | |
| 96 | + | |
| 97 | + | |
| 98 | +@given(amounts, jurisdictions) | |
| 99 | +@settings(max_examples=200, deadline=None) | |
| 100 | +def test_sale_total_equals_subtotal_plus_taxes(amount, jurisdiction) -> None: | |
| 101 | + """For any taxed sale: the receivable equals revenue + all tax lines.""" | |
| 102 | + document = AirDocument.model_validate({ | |
| 103 | + "events": [{ | |
| 104 | + "id": "evt_prop_sale", | |
| 105 | + "type": "Sale", | |
| 106 | + "date": date(2026, 7, 15), | |
| 107 | + "amount": {"amount": amount, "currency": "CAD"}, | |
| 108 | + "tax": {"jurisdiction": jurisdiction}, | |
| 109 | + }] | |
| 110 | + }) | |
| 111 | + journal, _ = compile_document(document, POLICIES) | |
| 112 | + (entry,) = journal.entries | |
| 113 | + debits = [l for l in entry.lines if l.side.value == "debit"] | |
| 114 | + credits = [l for l in entry.lines if l.side.value == "credit"] | |
| 115 | + assert len(debits) == 1 | |
| 116 | + assert debits[0].amount.amount == sum(l.amount.amount for l in credits) | |
| 117 | + for line in entry.lines: | |
| 118 | + assert line.amount.amount >= 0 | |
added
tests/test_core.py
+221 −0
@@ -0,0 +1,221 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_core.py | |
| 6 | +# Description : Unit tests — money, provenance, ALSL strictness, diagnostics, ledger, backends. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Unit tests for the non-golden guarantees: float rejection, SSA provenance, | |
| 9 | +ALSL loader strictness, clang-style diagnostics, the hash-chained ledger, | |
| 10 | +reversal semantics, and the CSV/native backends.""" | |
| 11 | +from __future__ import annotations | |
| 12 | + | |
| 13 | +import json | |
| 14 | +from datetime import date | |
| 15 | +from decimal import Decimal | |
| 16 | +from pathlib import Path | |
| 17 | + | |
| 18 | +import pytest | |
| 19 | + | |
| 20 | +from aic.compiler import compile_document | |
| 21 | +from aic.diagnostics import CompilationError | |
| 22 | +from alsl.loader import AlslLoadError, load_policy_set | |
| 23 | +from backends.generic_csv.backend import GenericCsvBackend | |
| 24 | +from backends.native.backend import NativeLedgerBackend | |
| 25 | +from core.events import AirDocument | |
| 26 | +from core.money import CurrencyMismatchError, Money, RoundingMode, money | |
| 27 | +from kernel.ledger import Ledger | |
| 28 | +from kernel.reporting import balance_sheet, income_statement, trial_balance | |
| 29 | + | |
| 30 | +ROOT = Path(__file__).resolve().parents[1] | |
| 31 | +POLICY_FILE = ROOT / "alsl" / "policies" / "ca-qc-2026.yaml" | |
| 32 | + | |
| 33 | + | |
| 34 | +# --- Money ------------------------------------------------------------------- | |
| 35 | +def test_money_rejects_floats() -> None: | |
| 36 | + with pytest.raises(TypeError): | |
| 37 | + Money(19.99, "CAD") # type: ignore[arg-type] | |
| 38 | + with pytest.raises(TypeError): | |
| 39 | + money("10.00", "CAD").multiply(1.05) # type: ignore[arg-type] | |
| 40 | + | |
| 41 | + | |
| 42 | +def test_money_currency_mismatch() -> None: | |
| 43 | + with pytest.raises(CurrencyMismatchError): | |
| 44 | + money("1", "CAD") + money("1", "USD") | |
| 45 | + | |
| 46 | + | |
| 47 | +def test_rounding_modes_differ_on_ties() -> None: | |
| 48 | + half = money("0.125", "CAD") | |
| 49 | + assert half.quantized(RoundingMode.HALF_UP).amount == Decimal("0.13") | |
| 50 | + assert half.quantized(RoundingMode.HALF_EVEN).amount == Decimal("0.12") | |
| 51 | + | |
| 52 | + | |
| 53 | +def test_air_schema_rejects_float_amounts() -> None: | |
| 54 | + with pytest.raises(Exception): | |
| 55 | + AirDocument.model_validate({ | |
| 56 | + "events": [{ | |
| 57 | + "id": "evt_f", "type": "Sale", "date": date(2026, 1, 1), | |
| 58 | + "amount": {"amount": 19.99, "currency": "CAD"}, | |
| 59 | + }] | |
| 60 | + }) | |
| 61 | + | |
| 62 | + | |
| 63 | +# --- ALSL strictness ----------------------------------------------------------- | |
| 64 | +def test_alsl_rejects_uncited_tax_policy(tmp_path: Path) -> None: | |
| 65 | + bad = tmp_path / "bad.yaml" | |
| 66 | + bad.write_text( | |
| 67 | + "alsl_version: \"0.1\"\npolicy_set: bad\nversion: \"1\"\n" | |
| 68 | + "policies:\n - name: mystery_tax\n kind: tax\n" | |
| 69 | + " when: {jurisdiction: CA-QC}\n" | |
| 70 | + " apply:\n - {code: GST, rate: \"0.05\"}\n", | |
| 71 | + encoding="utf-8", | |
| 72 | + ) | |
| 73 | + with pytest.raises(AlslLoadError, match="cite a source"): | |
| 74 | + load_policy_set(bad) | |
| 75 | + | |
| 76 | + | |
| 77 | +def test_alsl_rejects_float_rates(tmp_path: Path) -> None: | |
| 78 | + bad = tmp_path / "bad.yaml" | |
| 79 | + bad.write_text( | |
| 80 | + "alsl_version: \"0.1\"\npolicy_set: bad\nversion: \"1\"\n" | |
| 81 | + "policies:\n - name: float_tax\n kind: tax\n" | |
| 82 | + " source: docs/research/canada-gst-qst.md\n" | |
| 83 | + " when: {jurisdiction: CA-QC}\n" | |
| 84 | + " apply:\n - {code: GST, rate: 0.05}\n", | |
| 85 | + encoding="utf-8", | |
| 86 | + ) | |
| 87 | + with pytest.raises(AlslLoadError, match="float"): | |
| 88 | + load_policy_set(bad) | |
| 89 | + | |
| 90 | + | |
| 91 | +# --- Diagnostics ----------------------------------------------------------------- | |
| 92 | +def test_unknown_jurisdiction_is_a_precise_error() -> None: | |
| 93 | + policies = load_policy_set(POLICY_FILE) | |
| 94 | + document = AirDocument.model_validate({ | |
| 95 | + "events": [{ | |
| 96 | + "id": "evt_bc", "type": "Sale", "date": date(2026, 1, 1), | |
| 97 | + "amount": {"amount": "100.00", "currency": "CAD"}, | |
| 98 | + "tax": {"jurisdiction": "CA-BC"}, | |
| 99 | + }] | |
| 100 | + }) | |
| 101 | + with pytest.raises(CompilationError) as exc: | |
| 102 | + compile_document(document, policies) | |
| 103 | + codes = [d.code for d in exc.value.diagnostics] | |
| 104 | + assert "AIR-E400" in codes | |
| 105 | + rendered = "\n".join(d.render() for d in exc.value.diagnostics) | |
| 106 | + assert "CA-BC" in rendered and "help:" in rendered | |
| 107 | + | |
| 108 | + | |
| 109 | +def test_missing_fx_rate_is_a_precise_error() -> None: | |
| 110 | + policies = load_policy_set(POLICY_FILE) | |
| 111 | + document = AirDocument.model_validate({ | |
| 112 | + "events": [{ | |
| 113 | + "id": "evt_usd", "type": "Sale", "date": date(2026, 1, 1), | |
| 114 | + "amount": {"amount": "100.00", "currency": "USD"}, | |
| 115 | + "tax": {"jurisdiction": "CA-QC", "exempt": True}, | |
| 116 | + }] | |
| 117 | + }) | |
| 118 | + with pytest.raises(CompilationError) as exc: | |
| 119 | + compile_document(document, policies) | |
| 120 | + assert any(d.code == "AIR-E500" for d in exc.value.diagnostics) | |
| 121 | + | |
| 122 | + | |
| 123 | +# --- helpers --------------------------------------------------------------------- | |
| 124 | +def _demo_journal(): | |
| 125 | + policies = load_policy_set(POLICY_FILE) | |
| 126 | + document = AirDocument.model_validate({ | |
| 127 | + "events": [ | |
| 128 | + {"id": "evt_own", "type": "OwnerContribution", | |
| 129 | + "date": date(2026, 1, 1), | |
| 130 | + "amount": {"amount": "10000.00", "currency": "CAD"}}, | |
| 131 | + {"id": "evt_sale", "type": "Sale", "date": date(2026, 1, 10), | |
| 132 | + "amount": {"amount": "1000.00", "currency": "CAD"}, | |
| 133 | + "tax": {"jurisdiction": "CA-QC"}}, | |
| 134 | + {"id": "evt_buy", "type": "Purchase", "date": date(2026, 1, 12), | |
| 135 | + "amount": {"amount": "400.00", "currency": "CAD"}, | |
| 136 | + "tax": {"jurisdiction": "CA-QC"}}, | |
| 137 | + ] | |
| 138 | + }) | |
| 139 | + journal, _ = compile_document(document, policies) | |
| 140 | + return journal | |
| 141 | + | |
| 142 | + | |
| 143 | +# --- Ledger (native, hash-chained) -------------------------------------------------- | |
| 144 | +def test_ledger_hash_chain_and_idempotency(tmp_path: Path) -> None: | |
| 145 | + journal = _demo_journal() | |
| 146 | + ledger = Ledger(path=tmp_path / "ledger.jsonl") | |
| 147 | + assert ledger.post_journal(journal, "key-1") == 3 | |
| 148 | + assert ledger.post_journal(journal, "key-1") == 0 # idempotent replay | |
| 149 | + assert ledger.verify_chain() | |
| 150 | + | |
| 151 | + # reload from disk: same entries, chain still valid | |
| 152 | + reloaded = Ledger(path=tmp_path / "ledger.jsonl") | |
| 153 | + assert len(reloaded.entries) == 3 | |
| 154 | + assert reloaded.verify_chain() | |
| 155 | + | |
| 156 | + # tampering is detected | |
| 157 | + lines = (tmp_path / "ledger.jsonl").read_text().splitlines() | |
| 158 | + record = json.loads(lines[0]) | |
| 159 | + record["lines"][0]["amount"] = "9999.99" | |
| 160 | + lines[0] = json.dumps(record, sort_keys=True, separators=(",", ":")) | |
| 161 | + (tmp_path / "ledger.jsonl").write_text("\n".join(lines) + "\n") | |
| 162 | + assert not Ledger(path=tmp_path / "ledger.jsonl").verify_chain() | |
| 163 | + | |
| 164 | + | |
| 165 | +def test_ledger_reversal_nets_to_zero() -> None: | |
| 166 | + journal = _demo_journal() | |
| 167 | + ledger = Ledger() | |
| 168 | + ledger.post_journal(journal, "key-1") | |
| 169 | + for entry in list(ledger.entries): | |
| 170 | + ledger.reverse_entry(entry.id, f"rev:{entry.id}") | |
| 171 | + for per_ccy in ledger.balances().values(): | |
| 172 | + for balance in per_ccy.values(): | |
| 173 | + assert balance == 0 | |
| 174 | + | |
| 175 | + | |
| 176 | +# --- Reporting ------------------------------------------------------------------- | |
| 177 | +def test_reports_balance_and_render() -> None: | |
| 178 | + journal = _demo_journal() | |
| 179 | + ledger = Ledger() | |
| 180 | + ledger.post_journal(journal, "key-1") | |
| 181 | + | |
| 182 | + tb = trial_balance(ledger, "csv") | |
| 183 | + total_row = [r for r in tb.splitlines() if r.startswith("TOTAL")][0] | |
| 184 | + cells = total_row.split(",") | |
| 185 | + assert cells[-2] == cells[-1] # debits == credits | |
| 186 | + | |
| 187 | + inc = income_statement(ledger, "json") | |
| 188 | + data = json.loads(inc) | |
| 189 | + net = [r for r in data["rows"] if r["section"] == "NET INCOME"][0] | |
| 190 | + assert Decimal(net["amount"]) == Decimal("600.00") # 1000 revenue - 400 expense | |
| 191 | + | |
| 192 | + bs = balance_sheet(ledger, "markdown") | |
| 193 | + lines = [l for l in bs.splitlines() if "TOTAL" in l] | |
| 194 | + assets = [l for l in lines if "TOTAL ASSETS" in l][0].split("|")[-2].strip() | |
| 195 | + liabeq = [l for l in lines if "TOTAL LIAB." in l][0].split("|")[-2].strip() | |
| 196 | + assert assets == liabeq # Assets = Liabilities + Equity (+ net income) | |
| 197 | + | |
| 198 | + | |
| 199 | +# --- Backends ---------------------------------------------------------------------- | |
| 200 | +def test_csv_backend_post_and_reverse(tmp_path: Path) -> None: | |
| 201 | + journal = _demo_journal() | |
| 202 | + backend = GenericCsvBackend(tmp_path) | |
| 203 | + receipt = backend.post(backend.compile(journal), "abc123") | |
| 204 | + content = Path(receipt.reference).read_text() | |
| 205 | + assert "je_evt_sale" in content and "1149.75" in content | |
| 206 | + | |
| 207 | + reversal = backend.reverse(receipt) | |
| 208 | + rev = Path(reversal.reference).read_text() | |
| 209 | + assert "rev_je_evt_sale" in rev and "REVERSAL:" in rev | |
| 210 | + # sides swapped: original AR debit becomes credit | |
| 211 | + orig_line = [l for l in content.splitlines() if "1149.75" in l][0] | |
| 212 | + rev_line = [l for l in rev.splitlines() if "1149.75" in l][0] | |
| 213 | + assert "debit" in orig_line and "credit" in rev_line | |
| 214 | + | |
| 215 | + | |
| 216 | +def test_native_backend_roundtrip(tmp_path: Path) -> None: | |
| 217 | + journal = _demo_journal() | |
| 218 | + backend = NativeLedgerBackend(tmp_path / "books.jsonl") | |
| 219 | + receipt = backend.post(backend.compile(journal), "batch-1") | |
| 220 | + assert receipt.details["entries_appended"] == "3" | |
| 221 | + assert backend.ledger.verify_chain() | |
added
tests/test_incremental.py
+128 −0
@@ -0,0 +1,128 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_incremental.py | |
| 6 | +# Description : Tests — incremental compilation: diff, reversal + replacement entries. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Incremental compilation tests. | |
| 9 | + | |
| 10 | +The scenario CLAUDE.md names explicitly: "if an invoice changes, recompile | |
| 11 | +only the delta (like Git), with reversal entries generated automatically." | |
| 12 | +""" | |
| 13 | +from __future__ import annotations | |
| 14 | + | |
| 15 | +from datetime import date | |
| 16 | +from decimal import Decimal | |
| 17 | +from pathlib import Path | |
| 18 | + | |
| 19 | +from aic.incremental import diff_documents, event_fingerprint, recompile | |
| 20 | +from alsl.loader import load_policy_set | |
| 21 | +from core.events import AirDocument | |
| 22 | +from core.invariant import entry_imbalances | |
| 23 | +from kernel.ledger import Ledger | |
| 24 | + | |
| 25 | +ROOT = Path(__file__).resolve().parents[1] | |
| 26 | +POLICIES = load_policy_set(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 27 | + | |
| 28 | + | |
| 29 | +def _doc(*events: dict) -> AirDocument: | |
| 30 | + return AirDocument.model_validate({"events": list(events)}) | |
| 31 | + | |
| 32 | + | |
| 33 | +SALE = { | |
| 34 | + "id": "evt_inv_001", "type": "Sale", "date": date(2026, 7, 15), | |
| 35 | + "amount": {"amount": "1000.00", "currency": "CAD"}, | |
| 36 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 37 | +} | |
| 38 | +PURCHASE = { | |
| 39 | + "id": "evt_bill_001", "type": "Purchase", "date": date(2026, 7, 16), | |
| 40 | + "amount": {"amount": "400.00", "currency": "CAD"}, | |
| 41 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 42 | +} | |
| 43 | +SALE_CORRECTED = {**SALE, "amount": {"amount": "1200.00", "currency": "CAD"}} | |
| 44 | + | |
| 45 | + | |
| 46 | +def test_fingerprint_is_content_sensitive() -> None: | |
| 47 | + a = _doc(SALE).events[0] | |
| 48 | + b = _doc(SALE_CORRECTED).events[0] | |
| 49 | + c = _doc(SALE).events[0] | |
| 50 | + assert event_fingerprint(a) != event_fingerprint(b) | |
| 51 | + assert event_fingerprint(a) == event_fingerprint(c) | |
| 52 | + | |
| 53 | + | |
| 54 | +def test_diff_classification() -> None: | |
| 55 | + old = _doc(SALE, PURCHASE) | |
| 56 | + new = _doc(SALE_CORRECTED, {**PURCHASE, "id": "evt_bill_002"}) | |
| 57 | + diff = diff_documents(old, new) | |
| 58 | + assert diff.changed == ("evt_inv_001",) | |
| 59 | + assert diff.removed == ("evt_bill_001",) | |
| 60 | + assert diff.added == ("evt_bill_002",) | |
| 61 | + assert diff.unchanged == () | |
| 62 | + | |
| 63 | + | |
| 64 | +def test_unchanged_document_produces_empty_delta() -> None: | |
| 65 | + old = _doc(SALE, PURCHASE) | |
| 66 | + new = _doc(SALE, PURCHASE) | |
| 67 | + result = recompile(old, new, POLICIES) | |
| 68 | + assert result.diff.is_empty() | |
| 69 | + assert result.journal.entries == [] | |
| 70 | + | |
| 71 | + | |
| 72 | +def test_corrected_invoice_yields_reversal_plus_replacement() -> None: | |
| 73 | + result = recompile(_doc(SALE), _doc(SALE_CORRECTED), POLICIES) | |
| 74 | + | |
| 75 | + assert len(result.reversals) == 1 | |
| 76 | + assert len(result.new_entries) == 1 | |
| 77 | + reversal, replacement = result.reversals[0], result.new_entries[0] | |
| 78 | + | |
| 79 | + # reversal is the exact contra of the original 1000.00 compile | |
| 80 | + assert reversal.id == "rev_je_evt_inv_001" | |
| 81 | + assert reversal.reverses == "je_evt_inv_001" | |
| 82 | + credit_ar = [l for l in reversal.lines | |
| 83 | + if l.account.code == "1100" and l.side.value == "credit"] | |
| 84 | + assert credit_ar and credit_ar[0].amount.amount == Decimal("1149.75") | |
| 85 | + | |
| 86 | + # replacement carries the corrected figures under a revisioned id | |
| 87 | + assert replacement.id.startswith("je_evt_inv_001_r") | |
| 88 | + debit_ar = [l for l in replacement.lines | |
| 89 | + if l.account.code == "1100" and l.side.value == "debit"] | |
| 90 | + assert debit_ar and debit_ar[0].amount.amount == Decimal("1379.70") # 1200 * 1.14975 | |
| 91 | + | |
| 92 | + # every delta entry balances | |
| 93 | + for entry in result.journal.entries: | |
| 94 | + assert entry_imbalances(entry) == {} | |
| 95 | + | |
| 96 | + | |
| 97 | +def test_removed_event_yields_reversal_only() -> None: | |
| 98 | + result = recompile(_doc(SALE, PURCHASE), _doc(SALE), POLICIES) | |
| 99 | + assert [e.id for e in result.reversals] == ["rev_je_evt_bill_001"] | |
| 100 | + assert result.new_entries == [] | |
| 101 | + | |
| 102 | + | |
| 103 | +def test_delta_posts_cleanly_onto_the_ledger() -> None: | |
| 104 | + """Full lifecycle: post v1, post the delta, net effect == direct v2 compile.""" | |
| 105 | + from aic.compiler import compile_document | |
| 106 | + | |
| 107 | + old, new = _doc(SALE, PURCHASE), _doc(SALE_CORRECTED, PURCHASE) | |
| 108 | + | |
| 109 | + ledger = Ledger() | |
| 110 | + v1, _ = compile_document(old, POLICIES) | |
| 111 | + ledger.post_journal(v1, "v1") | |
| 112 | + delta = recompile(old, new, POLICIES) | |
| 113 | + ledger.post_journal(delta.journal, "v2-delta") | |
| 114 | + | |
| 115 | + direct = Ledger() | |
| 116 | + v2, _ = compile_document(new, POLICIES) | |
| 117 | + direct.post_journal(v2, "v2") | |
| 118 | + | |
| 119 | + assert ledger.balances() == direct.balances() | |
| 120 | + assert ledger.verify_chain() | |
| 121 | + | |
| 122 | + | |
| 123 | +def test_recompile_is_deterministic() -> None: | |
| 124 | + a = recompile(_doc(SALE, PURCHASE), _doc(SALE_CORRECTED), POLICIES) | |
| 125 | + b = recompile(_doc(SALE, PURCHASE), _doc(SALE_CORRECTED), POLICIES) | |
| 126 | + ids_a = [e.id for e in a.journal.entries] | |
| 127 | + ids_b = [e.id for e in b.journal.entries] | |
| 128 | + assert ids_a == ids_b | |
added
tests/test_ingestion.py
+152 −0
@@ -0,0 +1,152 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_ingestion.py | |
| 6 | +# Description : Tests — extraction, confidence routing, human approval queue (all offline). | |
| 7 | +# ============================================================================= | |
| 8 | +"""Ingestion tests. NO network, NO API key: everything runs on MockExtractor. | |
| 9 | +Core guarantees under test: | |
| 10 | +- the LLM layer only ever produces AIR (validated by schema), never entries; | |
| 11 | +- schema-invalid extractions ALWAYS route to a human; | |
| 12 | +- low confidence routes to a human; approval stamps approver + timestamp; | |
| 13 | +- approved documents still go through the deterministic compiler. | |
| 14 | +""" | |
| 15 | +from __future__ import annotations | |
| 16 | + | |
| 17 | +from datetime import datetime | |
| 18 | +from decimal import Decimal | |
| 19 | +from pathlib import Path | |
| 20 | + | |
| 21 | +import pytest | |
| 22 | + | |
| 23 | +from ingestion.approval import ApprovalQueue | |
| 24 | +from ingestion.extractor import ExtractionError, MockExtractor | |
| 25 | +from ingestion.pipeline import Route, ingest | |
| 26 | +from sdk.cli import main | |
| 27 | + | |
| 28 | +ROOT = Path(__file__).resolve().parents[1] | |
| 29 | +POLICY = str(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 30 | +HIGH = ROOT / "tests" / "fixtures" / "invoice_high_confidence.txt" | |
| 31 | +LOW = ROOT / "tests" / "fixtures" / "invoice_low_confidence.txt" | |
| 32 | + | |
| 33 | + | |
| 34 | +# --- extractor --------------------------------------------------------------------- | |
| 35 | +def test_mock_extractor_parses_fixture_format() -> None: | |
| 36 | + result = MockExtractor().extract(HIGH.read_text()) | |
| 37 | + assert result.confidence == Decimal("0.97") | |
| 38 | + (event,) = result.events | |
| 39 | + assert event["type"] == "Sale" | |
| 40 | + assert event["amount"] == {"amount": "999.99", "currency": "CAD"} | |
| 41 | + assert event["tax"] == {"jurisdiction": "CA-QC"} | |
| 42 | + | |
| 43 | + | |
| 44 | +def test_mock_extractor_empty_source_is_an_error() -> None: | |
| 45 | + with pytest.raises(ExtractionError): | |
| 46 | + MockExtractor().extract("nothing here") | |
| 47 | + | |
| 48 | + | |
| 49 | +# --- routing ----------------------------------------------------------------------- | |
| 50 | +def test_high_confidence_routes_to_auto_approved() -> None: | |
| 51 | + outcome = ingest(HIGH.read_text(), MockExtractor(), | |
| 52 | + ingested_at=datetime(2026, 8, 5, 9, 0)) | |
| 53 | + assert outcome.route is Route.AUTO_APPROVED | |
| 54 | + assert outcome.document is not None | |
| 55 | + event = outcome.document.events[0] | |
| 56 | + # traceability stamped into meta | |
| 57 | + assert event.meta is not None and event.meta.llm is not None | |
| 58 | + assert event.meta.llm.model == "mock" | |
| 59 | + assert str(event.meta.llm.confidence) == "0.97" | |
| 60 | + | |
| 61 | + | |
| 62 | +def test_low_confidence_routes_to_review() -> None: | |
| 63 | + outcome = ingest(LOW.read_text(), MockExtractor()) | |
| 64 | + assert outcome.route is Route.NEEDS_REVIEW | |
| 65 | + assert outcome.document is not None # schema-valid, just uncertain | |
| 66 | + assert any("below threshold" in r for r in outcome.reasons) | |
| 67 | + | |
| 68 | + | |
| 69 | +def test_schema_invalid_extraction_always_goes_to_review() -> None: | |
| 70 | + bad = "type: Sale\nid: evt_bad\ndate: not-a-date\namount: 10.00 CAD\nconfidence: 0.99\n" | |
| 71 | + outcome = ingest(bad, MockExtractor()) | |
| 72 | + assert outcome.route is Route.NEEDS_REVIEW # despite 0.99 confidence | |
| 73 | + assert outcome.document is None | |
| 74 | + assert outcome.validation_errors | |
| 75 | + | |
| 76 | + | |
| 77 | +def test_threshold_is_configurable() -> None: | |
| 78 | + outcome = ingest(LOW.read_text(), MockExtractor(), | |
| 79 | + threshold=Decimal("0.50")) | |
| 80 | + assert outcome.route is Route.AUTO_APPROVED | |
| 81 | + | |
| 82 | + | |
| 83 | +# --- approval queue ------------------------------------------------------------------- | |
| 84 | +def test_approval_flow_stamps_approver(tmp_path: Path) -> None: | |
| 85 | + queue = ApprovalQueue(tmp_path) | |
| 86 | + outcome = ingest(LOW.read_text(), MockExtractor()) | |
| 87 | + item_id = queue.submit(outcome, LOW.read_text()) | |
| 88 | + | |
| 89 | + assert [i.id for i in queue.pending()] == [item_id] | |
| 90 | + | |
| 91 | + document = queue.approve(item_id, approver="simon-pierre", | |
| 92 | + approved_at=datetime(2026, 8, 5, 10, 30)) | |
| 93 | + assert queue.pending() == [] | |
| 94 | + event = document.events[0] | |
| 95 | + assert event.meta is not None | |
| 96 | + assert event.meta.approver == "simon-pierre" | |
| 97 | + assert event.meta.timestamps is not None | |
| 98 | + assert event.meta.timestamps.approved is not None | |
| 99 | + | |
| 100 | + | |
| 101 | +def test_reject_moves_item_out_of_pending(tmp_path: Path) -> None: | |
| 102 | + queue = ApprovalQueue(tmp_path) | |
| 103 | + item_id = queue.submit(ingest(LOW.read_text(), MockExtractor()), | |
| 104 | + LOW.read_text()) | |
| 105 | + queue.reject(item_id, reason="duplicate of INV-000") | |
| 106 | + assert queue.pending() == [] | |
| 107 | + with pytest.raises(KeyError): | |
| 108 | + queue.approve(item_id, approver="x") | |
| 109 | + | |
| 110 | + | |
| 111 | +# --- end-to-end CLI -------------------------------------------------------------------- | |
| 112 | +def test_cli_ingest_auto_approve_posts_to_ledger(tmp_path: Path, capsys) -> None: | |
| 113 | + home = str(tmp_path / "books") | |
| 114 | + main(["init", "--home", home, "--policies", POLICY]) | |
| 115 | + assert main(["ingest", str(HIGH), "--home", home]) == 0 | |
| 116 | + out = capsys.readouterr().out | |
| 117 | + assert "auto-approved" in out and "+1 appended" in out | |
| 118 | + | |
| 119 | + | |
| 120 | +def test_cli_ingest_review_then_approve(tmp_path: Path, capsys) -> None: | |
| 121 | + home = str(tmp_path / "books") | |
| 122 | + main(["init", "--home", home, "--policies", POLICY]) | |
| 123 | + capsys.readouterr() | |
| 124 | + | |
| 125 | + assert main(["ingest", str(LOW), "--home", home]) == 0 | |
| 126 | + out = capsys.readouterr().out | |
| 127 | + assert "routed to human review" in out | |
| 128 | + item_id = next(w for w in out.split() if w.startswith("inbox_")) | |
| 129 | + | |
| 130 | + assert main(["inbox", "--home", home]) == 0 | |
| 131 | + assert item_id in capsys.readouterr().out | |
| 132 | + | |
| 133 | + assert main(["approve", item_id, "--home", home, | |
| 134 | + "--approver", "simon-pierre"]) == 0 | |
| 135 | + out = capsys.readouterr().out | |
| 136 | + assert "approved by simon-pierre" in out and "+1 appended" in out | |
| 137 | + | |
| 138 | + # the purchase (412.50 + GST 20.63 + QST 41.15) is now in the books | |
| 139 | + assert main(["report", "trial-balance", "--home", home]) == 0 | |
| 140 | + tb = capsys.readouterr().out | |
| 141 | + assert "412.50" in tb | |
| 142 | + | |
| 143 | + | |
| 144 | +def test_cli_reject(tmp_path: Path, capsys) -> None: | |
| 145 | + home = str(tmp_path / "books") | |
| 146 | + main(["init", "--home", home, "--policies", POLICY]) | |
| 147 | + main(["ingest", str(LOW), "--home", home]) | |
| 148 | + out = capsys.readouterr().out | |
| 149 | + item_id = next(w for w in out.split() if w.startswith("inbox_")) | |
| 150 | + assert main(["reject", item_id, "--home", home, "--reason", "not ours"]) == 0 | |
| 151 | + assert main(["inbox", "--home", home]) == 0 | |
| 152 | + assert "inbox empty" in capsys.readouterr().out | |
added
tests/test_llm_live.py
+74 −0
@@ -0,0 +1,74 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_llm_live.py | |
| 6 | +# Description : LIVE integration tests for the Claude extractor — skipped without ANTHROPIC_API_KEY. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Live Claude extractor tests. | |
| 9 | + | |
| 10 | +These hit the real Claude API and are SKIPPED unless ANTHROPIC_API_KEY is set | |
| 11 | +in the environment — the offline suite (mock extractor) never requires it. | |
| 12 | +The key is read from the environment only; it is never stored in the repo. | |
| 13 | + | |
| 14 | +Run: ANTHROPIC_API_KEY=sk-ant-... .venv/bin/python -m pytest tests/test_llm_live.py -v | |
| 15 | +""" | |
| 16 | +from __future__ import annotations | |
| 17 | + | |
| 18 | +import os | |
| 19 | +from decimal import Decimal | |
| 20 | +from pathlib import Path | |
| 21 | + | |
| 22 | +import pytest | |
| 23 | + | |
| 24 | +pytestmark = pytest.mark.skipif( | |
| 25 | + not os.environ.get("ANTHROPIC_API_KEY"), | |
| 26 | + reason="ANTHROPIC_API_KEY not set — live LLM tests are opt-in", | |
| 27 | +) | |
| 28 | + | |
| 29 | +ROOT = Path(__file__).resolve().parents[1] | |
| 30 | +POLICY = ROOT / "alsl" / "policies" / "ca-qc-2026.yaml" | |
| 31 | +INVOICE = ROOT / "tests" / "fixtures" / "invoice_realistic.txt" | |
| 32 | + | |
| 33 | + | |
| 34 | +def test_claude_extractor_reads_a_real_invoice() -> None: | |
| 35 | + from ingestion.extractor import ClaudeExtractor | |
| 36 | + from ingestion.pipeline import ingest | |
| 37 | + | |
| 38 | + outcome = ingest(INVOICE.read_text(encoding="utf-8"), ClaudeExtractor()) | |
| 39 | + | |
| 40 | + # extraction must be schema-valid AIR (the LLM never emits journal entries) | |
| 41 | + assert outcome.document is not None, outcome.validation_errors | |
| 42 | + (event,) = outcome.document.events | |
| 43 | + | |
| 44 | + # what a correct reading of this invoice must contain | |
| 45 | + assert event.type.value in ("Sale", "Purchase") # perspective may vary | |
| 46 | + assert event.date.isoformat() == "2026-07-28" | |
| 47 | + subtotal = event.subtotal() | |
| 48 | + assert subtotal is not None | |
| 49 | + # pre-tax amount, never the tax-included total, never a float artifact | |
| 50 | + assert subtotal.amount == Decimal("999.99") | |
| 51 | + assert subtotal.currency == "CAD" | |
| 52 | + assert event.tax is not None and event.tax.jurisdiction.upper().endswith("QC") | |
| 53 | + | |
| 54 | + # confidence must be an honest decimal in (0, 1] | |
| 55 | + assert Decimal("0") < outcome.extraction.confidence <= Decimal("1") | |
| 56 | + assert event.meta is not None and event.meta.llm is not None | |
| 57 | + | |
| 58 | + | |
| 59 | +def test_live_extraction_compiles_to_a_balanced_journal() -> None: | |
| 60 | + from aic.compiler import compile_document | |
| 61 | + from alsl.loader import load_policy_set | |
| 62 | + from core.invariant import entry_imbalances | |
| 63 | + from ingestion.extractor import ClaudeExtractor | |
| 64 | + from ingestion.pipeline import ingest | |
| 65 | + | |
| 66 | + outcome = ingest(INVOICE.read_text(encoding="utf-8"), ClaudeExtractor()) | |
| 67 | + assert outcome.document is not None | |
| 68 | + | |
| 69 | + journal, _ = compile_document(outcome.document, load_policy_set(POLICY)) | |
| 70 | + (entry,) = journal.entries | |
| 71 | + assert entry_imbalances(entry) == {} | |
| 72 | + # deterministic tax computation on the extracted subtotal: GST 50.00, QST 99.75 | |
| 73 | + amounts = sorted(str(l.amount.amount) for l in entry.lines) | |
| 74 | + assert "50.00" in amounts and "99.75" in amounts | |
added
tests/test_optimize.py
+130 −0
@@ -0,0 +1,130 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_optimize.py | |
| 6 | +# Description : Tests — optimization passes: fusion, netting, duplicate detection. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Optimization pass tests (Phase 6). | |
| 9 | + | |
| 10 | +The invariant is verified after every optimization pass by the pass manager; | |
| 11 | +these tests check the transformations themselves and their provenance.""" | |
| 12 | +from __future__ import annotations | |
| 13 | + | |
| 14 | +from datetime import date | |
| 15 | +from decimal import Decimal | |
| 16 | +from pathlib import Path | |
| 17 | + | |
| 18 | +from aic.compiler import compile_document | |
| 19 | +from alsl.loader import load_policy_set | |
| 20 | +from core.events import AirDocument | |
| 21 | + | |
| 22 | +ROOT = Path(__file__).resolve().parents[1] | |
| 23 | +POLICIES = load_policy_set(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 24 | + | |
| 25 | + | |
| 26 | +def _doc(*events: dict) -> AirDocument: | |
| 27 | + return AirDocument.model_validate({"events": list(events)}) | |
| 28 | + | |
| 29 | + | |
| 30 | +def _payment(i: int, amount: str = "125.00") -> dict: | |
| 31 | + return { | |
| 32 | + "id": f"evt_pay_{i:03d}", "type": "PaymentReceived", | |
| 33 | + "date": date(2026, 3, 15), "description": f"payout {i}", | |
| 34 | + "amount": {"amount": amount, "currency": "CAD"}, | |
| 35 | + } | |
| 36 | + | |
| 37 | + | |
| 38 | +# --- fusion -------------------------------------------------------------------- | |
| 39 | +def test_fusion_batches_identical_payments() -> None: | |
| 40 | + document = _doc(*[_payment(i) for i in range(50)]) | |
| 41 | + journal, diags = compile_document(document, POLICIES, optimize=True) | |
| 42 | + | |
| 43 | + (batch,) = journal.entries | |
| 44 | + assert batch.id.startswith("je_batch_") | |
| 45 | + assert batch.description == "Batch: 50 x PaymentReceived" | |
| 46 | + debit = next(l for l in batch.lines if l.side.value == "debit") | |
| 47 | + assert debit.amount.amount == Decimal("125.00") * 50 # 6250.00 | |
| 48 | + | |
| 49 | + # traceability survives fusion: the batch line derives from all 50 originals | |
| 50 | + assert debit.provenance_id is not None | |
| 51 | + node = journal.provenance.get(debit.provenance_id) | |
| 52 | + assert node.kind == "fusion" and len(node.inputs) == 50 | |
| 53 | + | |
| 54 | + assert any(d.code == "AIR-N800" for d in diags) | |
| 55 | + | |
| 56 | + | |
| 57 | +def test_fusion_keeps_different_days_apart() -> None: | |
| 58 | + events = [_payment(1), {**_payment(2), "date": date(2026, 3, 16)}] | |
| 59 | + journal, _ = compile_document(_doc(*events), POLICIES, optimize=True) | |
| 60 | + assert len(journal.entries) == 2 # nothing to fuse across dates | |
| 61 | + | |
| 62 | + | |
| 63 | +def test_default_pipeline_never_fuses() -> None: | |
| 64 | + document = _doc(*[_payment(i) for i in range(5)]) | |
| 65 | + journal, _ = compile_document(document, POLICIES) # optimize off | |
| 66 | + assert len(journal.entries) == 5 | |
| 67 | + | |
| 68 | + | |
| 69 | +# --- netting -------------------------------------------------------------------- | |
| 70 | +SALE = { | |
| 71 | + "id": "evt_net_sale", "type": "Sale", "date": date(2026, 3, 1), | |
| 72 | + "amount": {"amount": "1000.00", "currency": "CAD"}, | |
| 73 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 74 | +} | |
| 75 | + | |
| 76 | + | |
| 77 | +def _refund(amount: str) -> dict: | |
| 78 | + return { | |
| 79 | + "id": "evt_net_refund", "type": "Refund", "date": date(2026, 3, 5), | |
| 80 | + "related_event": "evt_net_sale", | |
| 81 | + "amount": {"amount": amount, "currency": "CAD"}, | |
| 82 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 83 | + } | |
| 84 | + | |
| 85 | + | |
| 86 | +def test_partial_refund_nets_into_one_entry() -> None: | |
| 87 | + journal, diags = compile_document( | |
| 88 | + _doc(SALE, _refund("250.00")), POLICIES, optimize=True) | |
| 89 | + | |
| 90 | + (entry,) = journal.entries | |
| 91 | + assert entry.id == "je_net_evt_net_sale" | |
| 92 | + amounts = {(l.account.code, l.side.value): str(l.amount.amount) | |
| 93 | + for l in entry.lines} | |
| 94 | + assert amounts[("4000", "credit")] == "750.00" # 1000 - 250 | |
| 95 | + assert amounts[("2310", "credit")] == "37.50" # 50.00 - 12.50 | |
| 96 | + assert amounts[("2320", "credit")] == "74.81" # 99.75 - 24.94 | |
| 97 | + assert amounts[("1100", "debit")] == "862.31" # balances | |
| 98 | + assert any(d.code == "AIR-N801" for d in diags) | |
| 99 | + | |
| 100 | + | |
| 101 | +def test_full_refund_nets_to_nothing() -> None: | |
| 102 | + journal, diags = compile_document( | |
| 103 | + _doc(SALE, _refund("1000.00")), POLICIES, optimize=True) | |
| 104 | + assert journal.entries == [] | |
| 105 | + note = next(d for d in diags if d.code == "AIR-N801") | |
| 106 | + assert "fully offset" in note.message | |
| 107 | + | |
| 108 | + | |
| 109 | +def test_refund_exceeding_sale_is_left_alone() -> None: | |
| 110 | + journal, _ = compile_document( | |
| 111 | + _doc(SALE, _refund("1500.00")), POLICIES, optimize=True) | |
| 112 | + assert len(journal.entries) == 2 # not nettable; both entries stay | |
| 113 | + | |
| 114 | + | |
| 115 | +# --- duplicate detection ------------------------------------------------------------- | |
| 116 | +def test_identical_content_different_ids_warns() -> None: | |
| 117 | + twin_a = {**SALE, "id": "evt_dup_a"} | |
| 118 | + twin_b = {**SALE, "id": "evt_dup_b"} | |
| 119 | + journal, diags = compile_document(_doc(twin_a, twin_b), POLICIES, | |
| 120 | + optimize=True) | |
| 121 | + warning = next(d for d in diags if d.code == "AIR-W800") | |
| 122 | + assert "evt_dup_a" in warning.message and "evt_dup_b" in warning.message | |
| 123 | + assert len(journal.entries) == 2 # nothing dropped — a human decides | |
| 124 | + | |
| 125 | + | |
| 126 | +def test_distinct_descriptions_are_not_duplicates() -> None: | |
| 127 | + a = {**SALE, "id": "evt_a", "description": "invoice 1"} | |
| 128 | + b = {**SALE, "id": "evt_b", "description": "invoice 2"} | |
| 129 | + _, diags = compile_document(_doc(a, b), POLICIES, optimize=True) | |
| 130 | + assert not any(d.code == "AIR-W800" for d in diags) | |
added
tests/test_quickbooks.py
+140 −0
@@ -0,0 +1,140 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_quickbooks.py | |
| 6 | +# Description : Tests — QuickBooks backend, entirely offline via the mock transport. | |
| 7 | +# ============================================================================= | |
| 8 | +"""QuickBooks backend tests. NO network, NO QuickBooks account: everything | |
| 9 | +runs against MockQboTransport, which simulates the QBO behaviors documented | |
| 10 | +in docs/research/erp-apis.md (requestid replay, error 6140, assigned Ids).""" | |
| 11 | +from __future__ import annotations | |
| 12 | + | |
| 13 | +from datetime import date | |
| 14 | +from decimal import Decimal | |
| 15 | +from pathlib import Path | |
| 16 | + | |
| 17 | +import pytest | |
| 18 | + | |
| 19 | +from aic.compiler import compile_document | |
| 20 | +from alsl.loader import load_policy_set | |
| 21 | +from backends.quickbooks.backend import QuickBooksBackend | |
| 22 | +from backends.quickbooks.mapper import ( | |
| 23 | + AccountMappingError, | |
| 24 | + contra_body, | |
| 25 | + doc_number, | |
| 26 | + map_entry, | |
| 27 | +) | |
| 28 | +from backends.quickbooks.transport import MockQboTransport, QboError, _dumps_exact | |
| 29 | +from core.events import AirDocument | |
| 30 | + | |
| 31 | +ROOT = Path(__file__).resolve().parents[1] | |
| 32 | +POLICIES = load_policy_set(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 33 | + | |
| 34 | + | |
| 35 | +def _journal(): | |
| 36 | + document = AirDocument.model_validate({ | |
| 37 | + "events": [{ | |
| 38 | + "id": "evt_qbo_sale", "type": "Sale", "date": date(2026, 7, 15), | |
| 39 | + "amount": {"amount": "1000.00", "currency": "CAD"}, | |
| 40 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 41 | + }] | |
| 42 | + }) | |
| 43 | + journal, _ = compile_document(document, POLICIES) | |
| 44 | + return journal | |
| 45 | + | |
| 46 | + | |
| 47 | +# --- mapper (pure, offline) ------------------------------------------------------ | |
| 48 | +def test_mapper_shapes_a_balanced_qbo_journal_entry() -> None: | |
| 49 | + journal = _journal() | |
| 50 | + body = map_entry(journal.entries[0]) | |
| 51 | + assert body["TxnDate"] == "2026-07-15" | |
| 52 | + assert body["DocNumber"] == "je_evt_qbo_sale" | |
| 53 | + postings = [l["JournalEntryLineDetail"]["PostingType"] for l in body["Line"]] | |
| 54 | + assert postings.count("Debit") == 1 and postings.count("Credit") == 3 | |
| 55 | + debits = sum(l["Amount"] for l in body["Line"] | |
| 56 | + if l["JournalEntryLineDetail"]["PostingType"] == "Debit") | |
| 57 | + credits = sum(l["Amount"] for l in body["Line"] | |
| 58 | + if l["JournalEntryLineDetail"]["PostingType"] == "Credit") | |
| 59 | + assert debits == credits == Decimal("1149.75") | |
| 60 | + assert all(isinstance(l["Amount"], Decimal) for l in body["Line"]) # never float | |
| 61 | + | |
| 62 | + | |
| 63 | +def test_doc_number_respects_qbo_21_char_limit() -> None: | |
| 64 | + long_id = "je_evt_" + "x" * 40 | |
| 65 | + dn = doc_number(long_id) | |
| 66 | + assert len(dn) <= 21 | |
| 67 | + assert dn == doc_number(long_id) # stable | |
| 68 | + | |
| 69 | + | |
| 70 | +def test_missing_account_mapping_is_a_clear_error() -> None: | |
| 71 | + journal = _journal() | |
| 72 | + with pytest.raises(AccountMappingError, match="1100"): | |
| 73 | + map_entry(journal.entries[0], account_map={"9999": {"value": "1"}}) | |
| 74 | + | |
| 75 | + | |
| 76 | +def test_contra_body_swaps_sides() -> None: | |
| 77 | + body = map_entry(_journal().entries[0]) | |
| 78 | + contra = contra_body(body) | |
| 79 | + assert contra["DocNumber"].startswith("R") | |
| 80 | + originals = [l["JournalEntryLineDetail"]["PostingType"] for l in body["Line"]] | |
| 81 | + contras = [l["JournalEntryLineDetail"]["PostingType"] for l in contra["Line"]] | |
| 82 | + assert all(a != b for a, b in zip(originals, contras)) | |
| 83 | + | |
| 84 | + | |
| 85 | +# --- mock transport behaviors ------------------------------------------------------- | |
| 86 | +def test_requestid_idempotency_never_double_posts() -> None: | |
| 87 | + backend = QuickBooksBackend() | |
| 88 | + payload = backend.compile(_journal()) | |
| 89 | + first = backend.post(payload, "batch-1") | |
| 90 | + replay = backend.post(payload, "batch-1") | |
| 91 | + assert first.details == replay.details | |
| 92 | + assert isinstance(backend.transport, MockQboTransport) | |
| 93 | + assert len(backend.transport.store) == 1 # one entity, not two | |
| 94 | + | |
| 95 | + | |
| 96 | +def test_duplicate_docnumber_with_new_requestid_raises_6140() -> None: | |
| 97 | + backend = QuickBooksBackend() | |
| 98 | + payload = backend.compile(_journal()) | |
| 99 | + backend.post(payload, "batch-1") | |
| 100 | + with pytest.raises(QboError, match="6140"): | |
| 101 | + backend.post(payload, "batch-2") # same DocNumber, different requestid | |
| 102 | + | |
| 103 | + | |
| 104 | +def test_unbalanced_body_rejected_by_mock() -> None: | |
| 105 | + transport = MockQboTransport() | |
| 106 | + with pytest.raises(QboError, match="6000"): | |
| 107 | + transport.create_journal_entry({ | |
| 108 | + "DocNumber": "bad", "TxnDate": "2026-01-01", | |
| 109 | + "Line": [{ | |
| 110 | + "Amount": Decimal("10"), "DetailType": "JournalEntryLineDetail", | |
| 111 | + "JournalEntryLineDetail": {"PostingType": "Debit", | |
| 112 | + "AccountRef": {"value": "1"}}, | |
| 113 | + }], | |
| 114 | + }, "r1") | |
| 115 | + | |
| 116 | + | |
| 117 | +def test_reverse_posts_contra_entries() -> None: | |
| 118 | + backend = QuickBooksBackend() | |
| 119 | + receipt = backend.post(backend.compile(_journal()), "batch-1") | |
| 120 | + reversal = backend.reverse(receipt) | |
| 121 | + assert isinstance(backend.transport, MockQboTransport) | |
| 122 | + assert len(backend.transport.store) == 2 | |
| 123 | + assert reversal.reversed_reference == receipt.reference | |
| 124 | + # reversing again with the same key replays, never double-posts | |
| 125 | + backend.reverse(receipt) | |
| 126 | + assert len(backend.transport.store) == 2 | |
| 127 | + | |
| 128 | + | |
| 129 | +def test_capabilities_declare_the_contract() -> None: | |
| 130 | + caps = QuickBooksBackend().capabilities() | |
| 131 | + assert caps.posts_remotely and not caps.native_reversal | |
| 132 | + assert caps.idempotency == "requestid" | |
| 133 | + | |
| 134 | + | |
| 135 | +# --- exact-decimal serialization (for the real transport, tested offline) ----------- | |
| 136 | +def test_dumps_exact_emits_decimal_literals_not_floats() -> None: | |
| 137 | + out = _dumps_exact({"Amount": Decimal("1149.75"), "note": 'keep "quotes"'}) | |
| 138 | + assert '"Amount": 1149.75' in out | |
| 139 | + assert '"note": "keep \\"quotes\\""' in out | |
| 140 | + assert "1149.750000" not in out # no float artifacts | |
added
tests/test_reconcile.py
+164 −0
@@ -0,0 +1,164 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_reconcile.py | |
| 6 | +# Description : Tests — camt.053/MT940/CSV parsers and bank reconciliation matching. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Bank reconciliation tests (Phase 6). | |
| 9 | + | |
| 10 | +Fixtures come from docs/research/bank-statement-formats.md (balance-consistent | |
| 11 | +statement: 25,000.00 - 1,150.00 + 3,449.93 = 27,299.93). The camt.053 and | |
| 12 | +MT940 fixtures describe the SAME statement, so parsing either must yield the | |
| 13 | +same transactions.""" | |
| 14 | +from __future__ import annotations | |
| 15 | + | |
| 16 | +from datetime import date | |
| 17 | +from decimal import Decimal | |
| 18 | +from pathlib import Path | |
| 19 | + | |
| 20 | +import pytest | |
| 21 | + | |
| 22 | +from aic.compiler import compile_document | |
| 23 | +from alsl.loader import load_policy_set | |
| 24 | +from core.events import AirDocument | |
| 25 | +from kernel.bank_formats import ( | |
| 26 | + BankFormatError, | |
| 27 | + parse_bank_csv, | |
| 28 | + parse_camt053, | |
| 29 | + parse_mt940, | |
| 30 | + parse_statement, | |
| 31 | +) | |
| 32 | +from kernel.ledger import Ledger | |
| 33 | +from kernel.reconcile import cash_movements, reconcile | |
| 34 | +from kernel.workspace import Workspace | |
| 35 | +from sdk.cli import main | |
| 36 | +from sdk.syscalls import AirKernel | |
| 37 | + | |
| 38 | +ROOT = Path(__file__).resolve().parents[1] | |
| 39 | +POLICY = str(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 40 | +FIXTURES = ROOT / "tests" / "fixtures" | |
| 41 | +CAMT = FIXTURES / "bank_statement.xml" | |
| 42 | +MT940 = FIXTURES / "bank_statement.mt940" | |
| 43 | +CSV = FIXTURES / "bank_statement.csv" | |
| 44 | + | |
| 45 | + | |
| 46 | +# --- parsers ---------------------------------------------------------------------- | |
| 47 | +def test_camt053_parses_signs_dates_and_references() -> None: | |
| 48 | + out_txn, in_txn = parse_camt053(CAMT) | |
| 49 | + assert out_txn.amount == Decimal("-1150.00") # DBIT -> money out | |
| 50 | + assert out_txn.currency == "CAD" | |
| 51 | + assert out_txn.date == date(2026, 8, 1) | |
| 52 | + assert out_txn.reference == "INV-2026-0042" # EndToEndId wins | |
| 53 | + assert "Acme" in out_txn.description # AddtlNtryInf | |
| 54 | + assert in_txn.amount == Decimal("3449.93") # CRDT -> money in | |
| 55 | + assert in_txn.date == date(2026, 8, 4) | |
| 56 | + | |
| 57 | + | |
| 58 | +def test_mt940_parses_the_same_statement_identically() -> None: | |
| 59 | + from_mt940 = parse_mt940(MT940) | |
| 60 | + from_camt = parse_camt053(CAMT) | |
| 61 | + assert [(t.date, t.amount, t.currency, t.reference) for t in from_mt940] == \ | |
| 62 | + [(t.date, t.amount, t.currency, t.reference) for t in from_camt] | |
| 63 | + # :86: free text became the description | |
| 64 | + assert "ACME" in from_mt940[0].description | |
| 65 | + | |
| 66 | + | |
| 67 | +def test_mt940_balance_integrity_is_enforced(tmp_path: Path) -> None: | |
| 68 | + tampered = MT940.read_text().replace("27299,93", "27300,00") | |
| 69 | + bad = tmp_path / "bad.mt940" | |
| 70 | + bad.write_text(tampered) | |
| 71 | + with pytest.raises(BankFormatError, match="does not balance"): | |
| 72 | + parse_mt940(bad) | |
| 73 | + | |
| 74 | + | |
| 75 | +def test_csv_parser_and_dispatcher_sniffing(tmp_path: Path) -> None: | |
| 76 | + txns = parse_bank_csv(CSV) | |
| 77 | + assert len(txns) == 3 and txns[2].amount == Decimal("-42.00") | |
| 78 | + # dispatcher: unknown suffix, content sniffed | |
| 79 | + renamed = tmp_path / "statement.txt" | |
| 80 | + renamed.write_text(MT940.read_text()) | |
| 81 | + assert len(parse_statement(renamed)) == 2 | |
| 82 | + with pytest.raises(BankFormatError, match="unrecognized"): | |
| 83 | + empty = tmp_path / "noise.txt" | |
| 84 | + empty.write_text("hello world") | |
| 85 | + parse_statement(empty) | |
| 86 | + | |
| 87 | + | |
| 88 | +# --- matching ---------------------------------------------------------------------- | |
| 89 | +def _books() -> Ledger: | |
| 90 | + """Books whose cash activity mirrors the fixture statement, plus one | |
| 91 | + ledger-only movement the bank never saw.""" | |
| 92 | + document = AirDocument.model_validate({"events": [ | |
| 93 | + {"id": "evt_rc_out", "type": "PaymentSent", "date": date(2026, 8, 2), | |
| 94 | + "description": "pay Acme invoice", | |
| 95 | + "amount": {"amount": "1150.00", "currency": "CAD"}}, | |
| 96 | + {"id": "evt_rc_in", "type": "PaymentReceived", "date": date(2026, 8, 4), | |
| 97 | + "description": "customer settles sale 7781", | |
| 98 | + "amount": {"amount": "3449.93", "currency": "CAD"}}, | |
| 99 | + {"id": "evt_rc_ghost", "type": "PaymentReceived", "date": date(2026, 8, 3), | |
| 100 | + "description": "cheque recorded, not yet deposited", | |
| 101 | + "amount": {"amount": "500.00", "currency": "CAD"}}, | |
| 102 | + ]}) | |
| 103 | + journal, _ = compile_document(document, load_policy_set(POLICY)) | |
| 104 | + ledger = Ledger() | |
| 105 | + ledger.post_journal(journal, "rc-books") | |
| 106 | + return ledger | |
| 107 | + | |
| 108 | + | |
| 109 | +def test_reconcile_matches_within_date_tolerance() -> None: | |
| 110 | + result = reconcile(parse_camt053(CAMT), cash_movements(_books())) | |
| 111 | + assert result.summary() == {"matched": 2, "unmatched_bank": 0, | |
| 112 | + "unmatched_ledger": 1} | |
| 113 | + # the 1150 bank debit (Aug 1) matched the books' payment dated Aug 2 | |
| 114 | + (pair,) = [(t, m) for t, m in result.matched if t.amount < 0] | |
| 115 | + assert pair[1].entry_id == "je_evt_rc_out" | |
| 116 | + # the undeposited cheque is flagged, not dropped | |
| 117 | + assert result.unmatched_ledger[0].entry_id == "je_evt_rc_ghost" | |
| 118 | + assert "DIFFERENCES FOUND" in result.render() | |
| 119 | + | |
| 120 | + | |
| 121 | +def test_reconcile_zero_tolerance_refuses_date_drift() -> None: | |
| 122 | + result = reconcile(parse_camt053(CAMT), cash_movements(_books()), | |
| 123 | + tolerance_days=0) | |
| 124 | + assert result.summary()["matched"] == 1 # only the same-day one | |
| 125 | + | |
| 126 | + | |
| 127 | +def test_reconcile_csv_flags_bank_only_fee() -> None: | |
| 128 | + result = reconcile(parse_bank_csv(CSV), cash_movements(_books())) | |
| 129 | + fees = [t for t in result.unmatched_bank if "FEE" in t.description] | |
| 130 | + assert len(fees) == 1 # bank fee not yet booked | |
| 131 | + | |
| 132 | + | |
| 133 | +# --- syscall + CLI ---------------------------------------------------------------------- | |
| 134 | +def _home_with_books(tmp_path: Path) -> Path: | |
| 135 | + home = tmp_path / "books" | |
| 136 | + Workspace.init(home, policies=POLICY) | |
| 137 | + kernel = AirKernel(home, actor="agent:reco") | |
| 138 | + kernel.create_economic_event({ | |
| 139 | + "id": "evt_rc_out", "type": "PaymentSent", "date": "2026-08-02", | |
| 140 | + "amount": {"amount": "1150.00", "currency": "CAD"}}) | |
| 141 | + kernel.create_economic_event({ | |
| 142 | + "id": "evt_rc_in", "type": "PaymentReceived", "date": "2026-08-04", | |
| 143 | + "amount": {"amount": "3449.93", "currency": "CAD"}}) | |
| 144 | + kernel.post() | |
| 145 | + return home | |
| 146 | + | |
| 147 | + | |
| 148 | +def test_reconcile_syscall_is_audited(tmp_path: Path) -> None: | |
| 149 | + home = _home_with_books(tmp_path) | |
| 150 | + kernel = AirKernel(home, actor="agent:reco") | |
| 151 | + summary = kernel.reconcile(MT940) | |
| 152 | + assert summary["matched"] == 2 and summary["clean"] is True | |
| 153 | + assert kernel.audit.records[-1]["syscall"] == "Reconcile" | |
| 154 | + assert kernel.audit.verify_chain() | |
| 155 | + | |
| 156 | + | |
| 157 | +def test_cli_reconcile_exit_codes(tmp_path: Path, capsys) -> None: | |
| 158 | + home = _home_with_books(tmp_path) | |
| 159 | + # clean statement -> 0 | |
| 160 | + assert main(["reconcile", str(MT940), "--home", str(home)]) == 0 | |
| 161 | + assert "CLEAN" in capsys.readouterr().out | |
| 162 | + # CSV includes an unbooked bank fee -> differences -> exit 2 | |
| 163 | + assert main(["reconcile", str(CSV), "--home", str(home)]) == 2 | |
| 164 | + assert "BANK?" in capsys.readouterr().out | |
added
tests/test_syscalls.py
+188 −0
@@ -0,0 +1,188 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_syscalls.py | |
| 6 | +# Description : Tests — agent syscalls, closed periods, and the hash-chained audit log. | |
| 7 | +# ============================================================================= | |
| 8 | +"""Syscall/kernel tests. | |
| 9 | + | |
| 10 | +Guarantees under test: | |
| 11 | +- agents get books access ONLY through syscalls, and every call (ok or error) | |
| 12 | + is one hash-chained audit record, in order; | |
| 13 | +- posting is idempotent and refuses closed periods; | |
| 14 | +- Reverse appends a contra entry, never edits; | |
| 15 | +- audit tampering is detected. | |
| 16 | +""" | |
| 17 | +from __future__ import annotations | |
| 18 | + | |
| 19 | +import json | |
| 20 | +from datetime import datetime, timezone | |
| 21 | +from pathlib import Path | |
| 22 | + | |
| 23 | +import pytest | |
| 24 | + | |
| 25 | +from kernel.audit import AuditLog | |
| 26 | +from kernel.workspace import Workspace | |
| 27 | +from sdk.cli import main | |
| 28 | +from sdk.syscalls import AirKernel, SyscallError | |
| 29 | + | |
| 30 | +ROOT = Path(__file__).resolve().parents[1] | |
| 31 | +POLICY = str(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 32 | + | |
| 33 | +SALE = { | |
| 34 | + "id": "evt_sc_sale", "type": "Sale", "date": "2026-02-10", | |
| 35 | + "amount": {"amount": "1000.00", "currency": "CAD"}, | |
| 36 | + "tax": {"jurisdiction": "CA-QC"}, | |
| 37 | +} | |
| 38 | + | |
| 39 | + | |
| 40 | +def _kernel(tmp_path: Path, actor: str = "agent:test") -> AirKernel: | |
| 41 | + home = tmp_path / "books" | |
| 42 | + if not (home / "meta.json").exists(): | |
| 43 | + Workspace.init(home, policies=POLICY) | |
| 44 | + tick = iter(range(10_000)) | |
| 45 | + return AirKernel(home, actor=actor, | |
| 46 | + clock=lambda: datetime(2026, 8, 5, 12, 0, next(tick), | |
| 47 | + tzinfo=timezone.utc)) | |
| 48 | + | |
| 49 | + | |
| 50 | +# --- happy path ----------------------------------------------------------------- | |
| 51 | +def test_full_agent_lifecycle_is_audited(tmp_path: Path) -> None: | |
| 52 | + kernel = _kernel(tmp_path) | |
| 53 | + | |
| 54 | + event_id = kernel.create_economic_event(SALE) | |
| 55 | + assert event_id == "evt_sc_sale" | |
| 56 | + diagnostics = kernel.validate() | |
| 57 | + assert not any(d.severity.value == "error" for d in diagnostics) | |
| 58 | + journal = kernel.compile() | |
| 59 | + assert len(journal.entries) == 1 | |
| 60 | + | |
| 61 | + receipt = kernel.post() | |
| 62 | + assert receipt["entries"] == 1 and receipt["appended"] == 1 | |
| 63 | + | |
| 64 | + report = kernel.generate_report("trial-balance") | |
| 65 | + assert "1149.75" in report | |
| 66 | + | |
| 67 | + calls = [(r["syscall"], r["status"]) for r in kernel.audit.records] | |
| 68 | + assert calls == [ | |
| 69 | + ("CreateEconomicEvent", "ok"), ("Validate", "ok"), ("Compile", "ok"), | |
| 70 | + ("Post", "ok"), ("GenerateReport", "ok"), | |
| 71 | + ] | |
| 72 | + assert kernel.audit.verify_chain() | |
| 73 | + | |
| 74 | + | |
| 75 | +def test_post_is_idempotent_and_archives_drafts(tmp_path: Path) -> None: | |
| 76 | + kernel = _kernel(tmp_path) | |
| 77 | + kernel.create_economic_event(SALE) | |
| 78 | + receipt = kernel.post() | |
| 79 | + # drafts staged -> archived on post; nothing left to post | |
| 80 | + with pytest.raises(SyscallError, match="no draft events"): | |
| 81 | + kernel.post() | |
| 82 | + archived = list((kernel.workspace.root / "documents").glob("posted_*")) | |
| 83 | + assert len(archived) == 1 | |
| 84 | + assert receipt["entry_ids"] == ["je_evt_sc_sale"] | |
| 85 | + | |
| 86 | + | |
| 87 | +def test_reverse_appends_contra(tmp_path: Path) -> None: | |
| 88 | + kernel = _kernel(tmp_path) | |
| 89 | + kernel.create_economic_event(SALE) | |
| 90 | + kernel.post() | |
| 91 | + contra_id = kernel.reverse("je_evt_sc_sale") | |
| 92 | + assert contra_id == "rev_je_evt_sc_sale" | |
| 93 | + # books net to zero after the reversal | |
| 94 | + report = kernel.generate_report("trial-balance", "csv") | |
| 95 | + total = [r for r in report.splitlines() if r.startswith("TOTAL")][0] | |
| 96 | + assert total.split(",")[-1] == total.split(",")[-2] | |
| 97 | + assert kernel.audit.verify_chain() | |
| 98 | + | |
| 99 | + | |
| 100 | +# --- refusals (audited errors) ------------------------------------------------------ | |
| 101 | +def test_invalid_event_is_refused_and_audited(tmp_path: Path) -> None: | |
| 102 | + kernel = _kernel(tmp_path) | |
| 103 | + with pytest.raises(Exception): | |
| 104 | + kernel.create_economic_event({ | |
| 105 | + "id": "evt_bad", "type": "Sale", "date": "2026-02-10", | |
| 106 | + "amount": {"amount": 19.99, "currency": "CAD"}, # float: forbidden | |
| 107 | + }) | |
| 108 | + (record,) = kernel.audit.records | |
| 109 | + assert record["syscall"] == "CreateEconomicEvent" | |
| 110 | + assert record["status"] == "error" | |
| 111 | + assert kernel.audit.verify_chain() | |
| 112 | + | |
| 113 | + | |
| 114 | +def test_closed_period_refuses_posting(tmp_path: Path) -> None: | |
| 115 | + kernel = _kernel(tmp_path) | |
| 116 | + kernel.close_period("2026-02") | |
| 117 | + kernel.create_economic_event(SALE) # dated 2026-02-10 | |
| 118 | + with pytest.raises(SyscallError, match="AIR-E700"): | |
| 119 | + kernel.post() | |
| 120 | + statuses = {r["syscall"]: r["status"] for r in kernel.audit.records} | |
| 121 | + assert statuses["ClosePeriod"] == "ok" | |
| 122 | + assert statuses["Post"] == "error" | |
| 123 | + # the drafts survive: the agent can restage into an open period | |
| 124 | + assert len(list((kernel.workspace.root / "drafts").glob("*.json"))) == 1 | |
| 125 | + | |
| 126 | + | |
| 127 | +def test_reverse_unknown_entry_is_refused(tmp_path: Path) -> None: | |
| 128 | + kernel = _kernel(tmp_path) | |
| 129 | + with pytest.raises(SyscallError): | |
| 130 | + kernel.reverse("je_nonexistent") | |
| 131 | + assert kernel.audit.records[-1]["status"] == "error" | |
| 132 | + | |
| 133 | + | |
| 134 | +def test_unknown_report_is_refused(tmp_path: Path) -> None: | |
| 135 | + kernel = _kernel(tmp_path) | |
| 136 | + with pytest.raises(SyscallError, match="unknown report"): | |
| 137 | + kernel.generate_report("profit-magic") | |
| 138 | + | |
| 139 | + | |
| 140 | +# --- audit log integrity --------------------------------------------------------------- | |
| 141 | +def test_audit_log_tampering_is_detected(tmp_path: Path) -> None: | |
| 142 | + kernel = _kernel(tmp_path) | |
| 143 | + kernel.create_economic_event(SALE) | |
| 144 | + kernel.post() | |
| 145 | + | |
| 146 | + audit_path = kernel.workspace.root / "audit.jsonl" | |
| 147 | + lines = audit_path.read_text().splitlines() | |
| 148 | + record = json.loads(lines[0]) | |
| 149 | + record["actor"] = "agent:impostor" # rewrite history | |
| 150 | + lines[0] = json.dumps(record, sort_keys=True, separators=(",", ":")) | |
| 151 | + audit_path.write_text("\n".join(lines) + "\n") | |
| 152 | + | |
| 153 | + assert not AuditLog(path=audit_path).verify_chain() | |
| 154 | + | |
| 155 | + | |
| 156 | +def test_audit_log_persists_across_sessions(tmp_path: Path) -> None: | |
| 157 | + kernel = _kernel(tmp_path) | |
| 158 | + kernel.create_economic_event(SALE) | |
| 159 | + kernel.post() | |
| 160 | + # a second agent session on the same home continues the same chain | |
| 161 | + kernel2 = AirKernel(kernel.workspace.root, actor="agent:second") | |
| 162 | + kernel2.generate_report("balance-sheet") | |
| 163 | + assert len(kernel2.audit.records) == 3 | |
| 164 | + assert {r["actor"] for r in kernel2.audit.records} == \ | |
| 165 | + {"agent:test", "agent:second"} | |
| 166 | + assert kernel2.audit.verify_chain() | |
| 167 | + | |
| 168 | + | |
| 169 | +# --- demo agent + CLI ----------------------------------------------------------------- | |
| 170 | +def test_demo_agent_end_to_end(tmp_path: Path, capsys) -> None: | |
| 171 | + from sdk.demo_agent import run | |
| 172 | + home = tmp_path / "books" | |
| 173 | + Workspace.init(home, policies=POLICY) | |
| 174 | + assert run(str(home), actor="agent:demo") == 0 | |
| 175 | + out = capsys.readouterr().out | |
| 176 | + assert "refused as expected" in out # closed-period enforcement | |
| 177 | + assert "chain VALID" in out | |
| 178 | + | |
| 179 | + | |
| 180 | +def test_cli_audit_lists_and_verifies(tmp_path: Path, capsys) -> None: | |
| 181 | + home = tmp_path / "books" | |
| 182 | + Workspace.init(home, policies=POLICY) | |
| 183 | + kernel = AirKernel(home, actor="agent:test") | |
| 184 | + kernel.create_economic_event(SALE) | |
| 185 | + kernel.post() | |
| 186 | + assert main(["audit", "--home", str(home)]) == 0 | |
| 187 | + out = capsys.readouterr().out | |
| 188 | + assert "CreateEconomicEvent" in out and "hash chain VALID" in out | |
added
tests/test_workspace.py
+105 −0
@@ -0,0 +1,105 @@ | ||
| 1 | +# ============================================================================= | |
| 2 | +# Projet : AIR — Accounting Intermediate Representation | |
| 3 | +# Auteur : Simon-Pierre Boucher | |
| 4 | +# Contact : contact@spboucher.ai | |
| 5 | +# Fichier : test_workspace.py | |
| 6 | +# Description : Tests — the AIR home (managed local data) and the standalone CLI flows. | |
| 7 | +# ============================================================================= | |
| 8 | +"""AIR home + CLI tests: init, compile into the home, archives, status, | |
| 9 | +incremental recompile, statements, and offline QBO export — all without any | |
| 10 | +third-party system.""" | |
| 11 | +from __future__ import annotations | |
| 12 | + | |
| 13 | +import json | |
| 14 | +from pathlib import Path | |
| 15 | + | |
| 16 | +import pytest | |
| 17 | + | |
| 18 | +from kernel.workspace import Workspace, WorkspaceError | |
| 19 | +from sdk.cli import main | |
| 20 | + | |
| 21 | +ROOT = Path(__file__).resolve().parents[1] | |
| 22 | +POLICY = str(ROOT / "alsl" / "policies" / "ca-qc-2026.yaml") | |
| 23 | +DEMO = str(ROOT / "tests" / "fixtures" / "demo_document.yaml") | |
| 24 | + | |
| 25 | + | |
| 26 | +def test_workspace_init_open_and_guardrails(tmp_path: Path) -> None: | |
| 27 | + home = tmp_path / "books" | |
| 28 | + ws = Workspace.init(home, name="acme", policies=POLICY) | |
| 29 | + assert ws.meta["name"] == "acme" | |
| 30 | + with pytest.raises(WorkspaceError, match="already initialized"): | |
| 31 | + Workspace.init(home) | |
| 32 | + with pytest.raises(WorkspaceError, match="no AIR home"): | |
| 33 | + Workspace.open(tmp_path / "elsewhere") | |
| 34 | + | |
| 35 | + | |
| 36 | +def test_cli_full_standalone_lifecycle(tmp_path: Path, capsys) -> None: | |
| 37 | + home = str(tmp_path / "books") | |
| 38 | + | |
| 39 | + assert main(["init", "--home", home, "--policies", POLICY]) == 0 | |
| 40 | + assert main(["compile", DEMO, "--home", home]) == 0 | |
| 41 | + out = capsys.readouterr().out | |
| 42 | + assert "8 entries" in out and "document archived" in out | |
| 43 | + | |
| 44 | + # replaying the same document is idempotent (0 appended) | |
| 45 | + assert main(["compile", DEMO, "--home", home]) == 0 | |
| 46 | + assert "+0 appended" in capsys.readouterr().out | |
| 47 | + | |
| 48 | + # data is managed inside the home | |
| 49 | + ws = Workspace.open(home) | |
| 50 | + status = ws.status() | |
| 51 | + assert status["entries"] == 8 | |
| 52 | + assert status["chain_valid"] is True | |
| 53 | + assert status["documents_archived"] == 1 # content-addressed, no duplicate | |
| 54 | + | |
| 55 | + # statements straight from the home | |
| 56 | + assert main(["report", "trial-balance", "--home", home]) == 0 | |
| 57 | + tb = capsys.readouterr().out | |
| 58 | + assert "TOTAL" in tb | |
| 59 | + | |
| 60 | + assert main(["status", "--home", home]) == 0 | |
| 61 | + assert "chain_valid" in capsys.readouterr().out | |
| 62 | + | |
| 63 | + | |
| 64 | +def test_cli_incremental_into_home(tmp_path: Path, capsys) -> None: | |
| 65 | + home = str(tmp_path / "books") | |
| 66 | + main(["init", "--home", home, "--policies", POLICY]) | |
| 67 | + main(["compile", DEMO, "--home", home]) | |
| 68 | + capsys.readouterr() | |
| 69 | + | |
| 70 | + corrected = tmp_path / "demo_v2.yaml" | |
| 71 | + corrected.write_text( | |
| 72 | + Path(DEMO).read_text().replace('amount: "333.33"', 'amount: "349.99"'), | |
| 73 | + encoding="utf-8", | |
| 74 | + ) | |
| 75 | + assert main(["recompile", DEMO, str(corrected), "--home", home]) == 0 | |
| 76 | + out = capsys.readouterr().out | |
| 77 | + assert "~1 changed" in out and "1 reversal(s)" in out | |
| 78 | + | |
| 79 | + status = Workspace.open(home).status() | |
| 80 | + assert status["entries"] == 10 # 8 + reversal + replacement | |
| 81 | + assert status["documents_archived"] == 2 | |
| 82 | + assert status["chain_valid"] is True | |
| 83 | + | |
| 84 | + | |
| 85 | +def test_cli_qbo_export_needs_no_quickbooks(tmp_path: Path, capsys) -> None: | |
| 86 | + home = str(tmp_path / "books") | |
| 87 | + main(["init", "--home", home, "--policies", POLICY]) | |
| 88 | + capsys.readouterr() | |
| 89 | + assert main(["compile", DEMO, "--home", home, "--backend", "qbo-export"]) == 0 | |
| 90 | + out = capsys.readouterr().out | |
| 91 | + assert "QBO JSON export" in out | |
| 92 | + | |
| 93 | + exports = list(Workspace.open(home).exports_dir.glob("qbo_journal_*.json")) | |
| 94 | + assert len(exports) == 1 | |
| 95 | + data = json.loads(exports[0].read_text()) | |
| 96 | + assert len(data["journal_entries"]) == 8 | |
| 97 | + assert data["journal_entries"][0]["Line"] # QBO JournalEntry shape | |
| 98 | + | |
| 99 | + | |
| 100 | +def test_cli_missing_policies_is_helpful(tmp_path: Path, capsys) -> None: | |
| 101 | + home = str(tmp_path / "books") | |
| 102 | + main(["init", "--home", home]) # no default policies | |
| 103 | + capsys.readouterr() | |
| 104 | + assert main(["compile", DEMO, "--home", home]) == 1 | |
| 105 | + assert "no policy set" in capsys.readouterr().err | |
| 106 | ||