SPB Git

Repository Public MIT

precision.md 2.7 KB 40 lines markdown

1<!--2ORIX Calc3File: docs/architecture/precision.md4Author: Simon-Pierre Boucher5Contact: contact@spboucher.ai6-->7# Precision strategy89| Kind of calculation | Representation | Consequence |10|---|---|---|11| Whole numbers | Arbitrary-precision integers (BigInt) | `2^200`, `100!`, `nCr(100, 50)` are exact, digit for digit |12| Decimals and fractions | Exact rationals on BigInt | `0.1 + 0.2 = 0.3` exactly; `1/3 + 1/3 + 1/3 = 1`; a Hilbert-matrix determinant is exact |13| Roots and powers of exact values | Exact when the result is rational (`√(1/4)`, `27^(2/3)`), otherwise double | No false exactness |14| Special angles | Exact table: sin 30° = 1/2, cos 90° = 0, tan 45° = 1; sin π = 0 by snapping the double nearest to k·π/2 | `tan 90°` is a domain error, not 1.6e16 |15| Transcendental functions | IEEE-754 binary64 (libm) | 15–16 significant digits, labelled "≈ floating point" |16| Complex numbers | `Complex<Double>` (Swift Numerics), principal branches | |17| Linear algebra | Gauss–Jordan on typed values: exact for exact entries; partial pivoting for doubles; LAPACK `dgeev` for eigenvalues | Residuals reported as verification checks |18| Money schedules | Double for rates, Foundation `Decimal` for rounding each amount to the cent | Lender statements can differ by a cent |19| Unit conversion | Exact rational factors, 38-digit decimal results | `1 mi = 1.609344 km` exactly |20| Statistics | Double, with special functions validated against SciPy / mpmath | See `Packages/ORIXStatistics/VALIDATION.md` |2122## Limits (stated, not hidden)2324- Exact non-integer rationals are kept up to 8192 bits of numerator or denominator; beyond that the calculation continues in25  double precision **with a precision note** in the verification report.26- Exact integers are limited to 4,000,000 bits (about 1.2 million digits); a larger result is an overflow error.27- Factorials are exact up to 50000!.28- Transcendental functions are not arbitrary precision. `Foundation.Decimal` is a 38-digit decimal float, not an29  arbitrary-precision type; it is used only where decimal rounding is the specification (money, unit display).30- Doubles are displayed with at most 17 significant digits; the default display is 12, which hides the last-bit noise of31  binary floating point without altering the stored value.3233## Never silently wrong3435- NaN and unexpected infinities are converted to errors at the point where they arise.36- Display rounding is computed from the exact stored value and never written back.37- Comparisons between an exact value and a double are done exactly (the double is converted to its exact rational value).38- Conversion from an exact rational to a double is correctly rounded (round-to-nearest-even with a sticky bit, including39  subnormals), verified against known hard cases.40
Files main
Menu

Theme