Repository Public MIT
1<!--2ORIX Calc3File: docs/mathematics/reference-tests.md4Author: Simon-Pierre Boucher5Contact: contact@spboucher.ai6-->7# Mathematical validation report89How the calculation engines are checked against independent references: where the references come from, how many cases10there are, which tolerances the tests assert, and what is *not* independently validated.1112This page describes the evidence in the repository. It is not a status page: run the suites with `Scripts/test-all.sh`13(see [`docs/development/testing.md`](../development/testing.md)); the latest recorded results are kept in14`RELEASE-REPORT.md` at the repository root. Per-engine detail is in each package's `VALIDATION.md`:15[Financial](../../Packages/ORIXFinancial/VALIDATION.md) · [Statistics](../../Packages/ORIXStatistics/VALIDATION.md) ·16[Units](../../Packages/ORIXUnits/VALIDATION.md) · [Symbolic](../../Packages/ORIXSymbolic/VALIDATION.md) ·17[Graph](../../Packages/ORIXGraph/VALIDATION.md).1819## Inventory2021`@Test` functions counted in each package's `Tests/` directory (`grep -rE '@Test\b'`); a parameterised or table-driven test22counts once, however many reference cases it loops over. The last column is the figure stated in the package's23`VALIDATION.md`, where there is one.2425| Engine package | `@Test` | `@Suite` | Reference cases | Stated in `VALIDATION.md` |26|---|--:|--:|---|---|27| `ORIXNumerics` | 12 | 2 | closed forms in the test source | — |28| `ORIXExpression` | 37 | 9 | exact expected strings and closed forms in the test source | — |29| `ORIXFinancial` | 67 | 8 | 108 cases in `reference.json` + textbook and Excel-documentation values | 67 tests in 8 suites |30| `ORIXStatistics` | 85 | 15 | 5 343 special-function values, 10 164 distribution values, 8 + 4 data sets, 132 inference cases, 18 regression cases, 6 bootstrap cases | 85 tests in 15 suites |31| `ORIXUnits` | 60 | 11 | 841 conversions, 962 rate-convention values | 60 tests in 11 suites |32| `ORIXSymbolic` | 58 | 10 | sympy / mpmath corpus (see table below) | 58 tests in 10 suites |33| `ORIXGraph` | 117 | 20 | analytic values and analytically known discontinuities | 117 tests in 20 suites |34| `ORIXWorkspace` | 11 | 3 | — (behavioural) | — |35| `ORIXMathCamera` | 3 | 2 | 30 printed expressions × 3 fonts | — |3637Feature packages and the app are behavioural and rendering tests, not mathematical references; they are listed in38`docs/development/testing.md`.3940## How the references are produced4142| Engine | Generator | Libraries | Output (in the package) | Loaded by the tests from (in the package) |43|---|---|---|---|---|44| Financial | `Packages/ORIXFinancial/Reference/generate_reference.py` | mpmath (50 digits), numpy, scipy 1.17.1, `decimal` | `Reference/reference.json`, console log `Reference/reference_run.txt` | `Tests/ORIXFinancialTests/Resources/reference.json` (a test fails if the two copies differ) |45| Statistics | `Packages/ORIXStatistics/Reference/gen_*.py`, `reflib.py`, driven by `generate_all.sh` | mpmath 1.3.0 (50 digits; 60 for the regression solve), scipy 1.17.1, numpy 2.4.4, statsmodels 0.14.6, `fractions`, Python 3.14.4 (`Reference/output/VERSIONS.txt`) | `Reference/output/*.json` | `Tests/ORIXStatisticsTests/ReferenceData/` (mirrored by the script) |46| Units | `Packages/ORIXUnits/Reference/generate_reference.py` | `fractions.Fraction`, `decimal` (60 digits) | `Reference/reference_conversions.tsv`, `reference_rates.tsv` | `Tests/ORIXUnitsTests/ReferenceData.swift` (embedded table) |47| Symbolic | `Packages/ORIXSymbolic/Reference/generate_reference.py` | sympy 1.14.0, mpmath 1.3.0 | `Reference/reference.json` | `Tests/ORIXSymbolicTests/Resources/reference.json` |48| Graph | none: expectations are analytic | — | — | test sources and `Tests/ORIXGraphTests/Support.swift` |49| Expression, Numerics | none: exact strings and closed forms | — | — | test sources |5051Each financial reference is computed by a different route from the Swift code (period-by-period compounding against the52closed form, `mpmath.polyroots` against the scan + Brent, `mpmath.diff` against the analytic duration sums, vectorised53numpy trees against scalar loops). mpmath, not scipy, is the primary reference for statistical functions because scipy54is itself inaccurate in places at the accuracy tested; the disagreements found are recorded in the JSON under55`scipy_disagreements`.5657## Standard arithmetic5859`Packages/ORIXExpression/Tests/ORIXExpressionTests/EngineTests.swift`, suite `StandardArithmetic`. Expected values are60exact display strings.6162| Input | Expected | What it guards |63|---|---|---|64| `25 × 4` | `100` | basic arithmetic |65| `2 ^ 3 ^ 2` | `512` | right-associative power |66| `-2 ^ 2` / `(-2) ^ 2` | `−4` / `4` | unary minus below power |67| `6 ÷ 2(1 + 2)` | `9` | implicit multiplication at the same level as ÷ |68| `0.1 + 0.2` | `0.3` | exact decimal arithmetic |69| `0.1 + 0.2 == 0.3` | `true` | exact comparison |70| `1/3 + 1/3 + 1/3` | `1` | exact rationals |71| `200 + 10%` | `220` (contextual), `200.1` (strict) | percent convention |72| `100 + 5% + 5%` | `110.25` | percentage of the running total |73| `2^100` | `1267650600228229401496703205376` | arbitrary-precision integers |74| `25!` | `15511210043330985984000000` | exact factorial |75| `1e400 / 1e399` | `10` | no intermediate overflow |76| `1/0`, `5 mod 0`, `0^-1` | `CalcError.divisionByZero` | errors, not infinities |77| `exp(1000)` | `CalcError` | overflow is an error |7879## Scientific8081Same file, suites `Scientific`, `ComplexNumbers`, `Matrices`, `Formatting`. Floating-point comparisons use a relative82tolerance of 1e-12 unless a test states otherwise.8384| Input | Expected | What it guards |85|---|---|---|86| `sin(30)`, `cos(60)` in degrees | `0.5` | exact special angles |87| `sin(180)`, `cos(90)` in degrees; `sin(pi)`, `cos(pi/2)` | `0` | snapping at multiples of π/2 |88| `tan(90)` in degrees; `tan(pi/2)` | `CalcError` | pole is a domain error |89| `sqrt(1/4)` | `0.5` | exact root of an exact value |90| `27^(2/3)`, `(-8)^(1/3)` | `9`, `−2` | exact rational powers |91| `log(81, 3)`, `log2(1/8)` | `4`, `−3` | exact logarithms |92| `nCr(100, 50)` | `100891344545564193334812497256` | exact combinatorics |93| `isprime(2^61 - 1)` | `true` | number theory on large integers |94| `sqrt(-1)`, `ln(0)`, `asin(2)`, `0/0` | `CalcError` | real-mode domain errors |95| `sin(x)² + cos(x)²` over sample points | 1 within 1e-12 | identities |9697## Financial9899Tolerance is `|obtained − expected| ≤ tol · max(1, |expected|)` unless stated. "Python" means `reference.json`.100101| Case | Source | Expected | Tolerance |102|---|---|---|---|103| TVM: 100 000, 6 % monthly, 360 payments → PMT | standard textbook case | −599.55 | 1e-9 |104| TVM: 1 000 at 5 % for 10 years → FV | standard textbook case | 1 628.89 | 1e-9 |105| BEGIN vs END: PV of 15 payments of 500 at 6 % | property + Python | BEGIN = END × 1.06 | 1e-13; 1e-12 vs Python |106| P/Y ≠ C/Y: `TVMInputs(n: 300, iy: 5, pv: 300 000, py: 12, cy: 2)` → PMT | project brief; Python | −1 744.81 | exact cent |107| P/Y ≠ C/Y round trip: `periodicRate(iy: 5, py: 12, cy: 2)` → nominal | property | 5 % | 1e-12 |108| `tvm` set: 11 worksheets × 5 unknowns (END/BEGIN, P/Y ≠ C/Y, daily compounding, zero, negative and 1e-7 % rates) | Python | each solved variable | 1e-9 (I/Y in %: 1e-8); periodic rate 1e-14 |109| Canadian mortgage 300 000, 5 % semi-annual, 25 y, monthly | project brief; Python | 1 744.81 | exact cent |110| Same loan, monthly compounding (US) | Python | 1 753.77 | exact cent |111| `loan_schedules` set: 11 schedules (accelerated, prepayments, rate changes with both policies, override, 5-year term) | Python `decimal`, `ROUND_HALF_UP` | payments, totals, probe rows | < 0.005 |112| IRR of −1000, 2300, −1320 | project brief; Python polynomial roots | 10 % and 20 % (both reported) | 1e-12 |113| IRR of −1000, 2200, −1210 | property | double root found | — |114| Multiple-rate TVM worksheet | property | refused with both candidates | — |115| Excel `PRICE` example (5.75 %, 6.5 %, semi-annual, 30/360) | Microsoft Excel documentation | 94.63436 | 5e-6 |116| Excel `YIELD`, `DURATION`, `MDURATION`, `XIRR` examples | Microsoft Excel documentation | 6.5 %; 10.9191453; 5.736; 0.373362535 | 5e-8; 5e-8; 5e-4; 5e-9 |117| BSM call / put, S = 42, K = 40, r = 10 %, σ = 20 %, T = 0.5 | Hull | 4.76 / 0.81 | 2 decimals |118| American put, CRR 5 steps, S = K = 50, r = 10 %, σ = 40 %, T = 5/12 | Hull | 4.49 | 2 decimals |119| `bsm` set: 8 markets × call/put | mpmath 50 digits | price, Greeks, implied volatility | price 1e-12·S + 1e-10·price; Greeks 1e-10·S; IV 1e-10 or price-noise/vega |120| `crr` set: 6 markets × call/put, 5 … 2 000 steps | vectorised numpy | European and American values | 1e-10 |121| `bonds` set: 10 dated bonds, every day-count convention | mpmath, `datetime`, `brentq` | accrued, prices, durations, convexity, yields | 1e-12 prices and yields; 1e-11 durations / convexity |122123## Graphing124125Expectations are analytic; the tests re-evaluate the function between the vertices the sampler returned126(`hiddenBreaks`, `spans`, `uncovered`, `maxScreenError`, `bounded`). Window 800 × 600, x ∈ [−10, 10] unless noted.127128| Function | Expected output | Tolerance |129|---|---|---|130| `1/x` | 2 polylines, pole at 0 with limits −∞ / +∞ | pole located to 1e-12 |131| `tan(x)` | 7 polylines, 6 poles at (k + ½)π | 1e-12 |132| `floor(x)` | 21 horizontal pieces, 20 jumps on the integers | exact |133| `sin(x)/x` | 1 polyline, removable discontinuity with hole (0, 1) | — |134| `(x² − 1)/(x − 1)` | removable at 1, hole value 2 | 1e-6 |135| `sqrt(x)` | starts at exactly (0, 0); undefined interval [−10, 0] | boundary to 1e-15 off the grid |136| `sin(1000x)` | no polyline, one band of 800 columns | — |137| 12 smooth functions | polyline within half a pixel of the curve | < 0.5 px |138| `x² + y² = 25` at 400 × 400 | one closed polyline, area 25π | ± 0.05 |139| `(x − y)² = 0` | nothing drawn, `reliable == false` with a reason | — |140| roots of `x³ − x`, `√2`, `cos x` | analytic values | 1e-12 … 1e-13 |141| `∫₀^π sin = 2`, `∫ e^(−x²)` on [−8, 8] = √π | analytic values | 1e-12 |142| `∫` across a pole (`1/x`, `tan x`, …) | refused with a message | — |143144## Statistics145146| Case | Source | Asserted tolerance |147|---|---|---|148| Special functions, whole grid (shapes 1e-5 … 1e6, tails to 1e-300) | mpmath 50 digits | 1e-15 … 2e-12 relative, per function |149| 14 distributions, 81 parameter sets: pdf/pmf, cdf, survival, quantiles | mpmath 50 digits | 1e-12 relative |150| Discrete quantiles (736 cases) | mpmath | exact integers |151| `normQ(0.975)` = 1.9599639845400538 | mpmath | 2e-15 |152| t(5) `ppf(0.975)` = 2.5705818356363146 | mpmath | 1e-12 |153| Nine quantile definitions, 1 080 values | `numpy.quantile` | 1e-13 |154| One-sample, Welch, paired t; proportions; χ²; ANOVA | scipy, statsmodels | 1e-9 … 1e-10 |155| Longley regression coefficients | 60-digit mpmath solve (agrees with the NIST certified values) | 1e-10 |156| Simple regression, prediction intervals, curve fits | scipy `linregress`, statsmodels, numpy | 1e-9 … 1e-11 |157| BCa interval on fixed replicates | Efron's formulas in numpy / scipy | 1e-9 |158| SplitMix64 and xoshiro256** output vectors | published vectors | exact |159| Every sampler, n = 200 000, fixed seeds | moments and empirical CDF | 5 standard errors |160161## Units162163| Case | Source | Tolerance |164|---|---|---|165| `1 mi → km` = 1.609344; `1 in → m` = 0.0254; `1 lb → kg` = 0.45359237 | Python `Fraction` from the primary definitions | exact |166| `1 gal (US) → L` = 3.785411784; `1 gal (imp) → L` = 4.54609 | same | exact |167| `1 atm → Pa` = 101325; `1 BTU → J` = 1055.05585262; `1 nmi → m` = 1852 | same | exact |168| 590 terminating conversions | same | equal to the last digit |169| 251 non-terminating conversions | same | 33 significant digits |170| `Double` path | same | 2 ulp |171| Round trip a → b → a | property | exact when the forward result terminated, else 1e-36 |172| 962 rate-convention values | Python `decimal`, 60 digits | 2e-13 relative |173174## Symbolic175176Symbolic forms differ between systems, so comparisons are semantic: both results are evaluated at pseudo-random points177and must agree to 1e-8 … 1e-10. Integrals are compared with mpmath quadrature of the integrand.178179| Area | Cases | Reference |180|---|---|---|181| Derivatives | 83, 6 points each; 150 random trees | sympy `diff`; Ridders derivative |182| Indefinite integrals | 88, 5 intervals each (each also re-differentiated) | mpmath quadrature |183| Definite integrals | 20 + 20 structural | mpmath quadrature |184| Simplification | 136 corpus + 600 random trees | value preservation, idempotence |185| Limits | 37 + 16 structural | sympy `limit` |186| Taylor coefficients | 20 series (orders 3–7) | sympy |187| Polynomial equations | 18 | sympy `nroots` |188| Factorisation | 18 + 12 random | sympy `factor_list` |189| Linear systems | 9 (incl. 8 × 8 Hilbert) | exact substitution |190| ODEs | 7 tests | closed forms with error bounds |191192Structural expectations cover the canonical simple cases (`d/dx x²` prints `2x`).193194## What is not independently validated195196Assembled from the limitations stated in the `VALIDATION.md` files.197198**Independence of the references**199- The Python references for Financial and Units are independent *implementations*, written in the same work session from200 the same reading of the conventions. They catch arithmetic, precision and algorithm errors; they cannot catch a201 convention misunderstood in both places (the 30/360 US February rule, the month-end coupon rule, Actual/360 and202 Actual/365 Fixed period handling, the accelerated-payment definition; a wrong belief about what a unit *is*).203- No QuantLib, numpy-financial, Bloomberg or physical-calculator cross-check was run. No value was checked against a204 BA II Plus or its manual. The five Excel documentation examples are the only third-party bond / XIRR figures, and they205 were quoted from memory of the Microsoft documentation, not re-downloaded.206- NIST SP 811 guard values and most unit definitions were written from memory of the standard. Two sources were fetched207 and read: the CODATA 2022 atomic mass constant and Schedules II and III of the Canada Weights and Measures Act.208 ISO 4217 names and minor-unit digits were not diffed against the SIX list.209- The published SplitMix64 / xoshiro256** vectors and the NIST Longley certified values were typed from memory; the210 tests rely on the 60-digit solve, and the agreement with the typed constants is asserted separately.211- Graph expectations are analytic, not from an external plotting system.212213**Not covered**214- NIST StRD Norris is not included; degree-10 polynomial problems (Filip, Wampler 4–5) were not attempted.215- No formal random-number battery (TestU01, PractRand) was run. The end-to-end bootstrap comparison is statistical216 (tolerance 6 % of the interval width).217- Excel `PRICE` / `YIELD` behaviour with one coupon or less remaining (simple-interest formula) is not reproduced and218 not validated.219- The ECB parser is tested against a document in the real structure with made-up rates, not against a live download.220- LaTeX output of symbolic results was not checked beyond what the printer guarantees; behaviour with user-defined221 functions inside symbolic expressions is not validated.222- Symbolic verification by sampling is evidence, not proof (1e-7 relative on up to 10 points).223- Graph detection of sub-pixel features, of zero sets without a sign change and of narrow poles before integration is224 heuristic; see the limits in `Packages/ORIXGraph/VALIDATION.md` §5.225- The Math Camera benchmark uses rendered images of printed expressions, not photographs.226227**Platform**228- The engine `VALIDATION.md` records state that the engine tests were run with `swift test` on macOS (arm64) only and229 were not executed on an iOS simulator or device. Nothing was run on a physical device. Performance figures come from230 one Apple Silicon Mac.231