Precision strategy
| Kind of calculation | Representation | Consequence |
|---|---|---|
| Whole numbers | Arbitrary-precision integers (BigInt) | 2^200, 100!, nCr(100, 50) are exact, digit for digit |
| Decimals and fractions | Exact rationals on BigInt | 0.1 + 0.2 = 0.3 exactly; 1/3 + 1/3 + 1/3 = 1; a Hilbert-matrix determinant is exact |
| Roots and powers of exact values | Exact when the result is rational (√(1/4), 27^(2/3)), otherwise double |
No false exactness |
| Special angles | Exact table: sin 30° = 1/2, cos 90° = 0, tan 45° = 1; sin π = 0 by snapping the double nearest to k·π/2 | tan 90° is a domain error, not 1.6e16 |
| Transcendental functions | IEEE-754 binary64 (libm) | 15–16 significant digits, labelled "≈ floating point" |
| Complex numbers | Complex<Double> (Swift Numerics), principal branches |
|
| Linear algebra | Gauss–Jordan on typed values: exact for exact entries; partial pivoting for doubles; LAPACK dgeev for eigenvalues |
Residuals reported as verification checks |
| Money schedules | Double for rates, Foundation Decimal for rounding each amount to the cent |
Lender statements can differ by a cent |
| Unit conversion | Exact rational factors, 38-digit decimal results | 1 mi = 1.609344 km exactly |
| Statistics | Double, with special functions validated against SciPy / mpmath | See Packages/ORIXStatistics/VALIDATION.md |
Limits (stated, not hidden)
- Exact non-integer rationals are kept up to 8192 bits of numerator or denominator; beyond that the calculation continues in double precision with a precision note in the verification report.
- Exact integers are limited to 4,000,000 bits (about 1.2 million digits); a larger result is an overflow error.
- Factorials are exact up to 50000!.
- Transcendental functions are not arbitrary precision.
Foundation.Decimalis a 38-digit decimal float, not an arbitrary-precision type; it is used only where decimal rounding is the specification (money, unit display). - Doubles are displayed with at most 17 significant digits; the default display is 12, which hides the last-bit noise of binary floating point without altering the stored value.
Never silently wrong
- NaN and unexpected infinities are converted to errors at the point where they arise.
- Display rounding is computed from the exact stored value and never written back.
- Comparisons between an exact value and a double are done exactly (the double is converted to its exact rational value).
- Conversion from an exact rational to a double is correctly rounded (round-to-nearest-even with a sticky bit, including subnormals), verified against known hard cases.