ORIXStatistics — validation record
Everything below was measured on this code, by the test suite, against references produced by the scripts in Reference/.
Nothing in the tests is an "expected" number taken on faith. The literal constants in the test sources are exact closed forms
or values printed by mpmath, with two exceptions that were typed from memory and are corroborated rather than trusted: the
published SplitMix64 / xoshiro256** output vectors (the implementation reproduces all 15 values) and the NIST Longley certified
values (a 60-digit solve reproduces them; see limitation 1).
How the references are produced
| File | Produces | Engine |
|---|---|---|
Reference/reflib.py |
shared high-precision kernels | mpmath at 50 digits (precision raised by 340 digits when a complement is tiny) |
Reference/gen_special.py → special.json |
5 343 special-function values | mpmath; scipy.special as sanity check |
Reference/gen_distributions.py → distributions.json |
10 164 pdf/pmf, cdf, survival, quantile values + moments | mpmath (quantiles: exact inverse of the double probability by a secant solve in 50 digits); scipy.stats as starting point and sanity check |
Reference/gen_descriptive.py → descriptive.json |
8 data sets, 4 paired sets | numpy, scipy.stats, statistics; mean and variances in exact rational arithmetic (fractions) |
Reference/gen_inference.py → inference.json |
132 test / interval cases | scipy.stats, statsmodels |
Reference/gen_regression.py → regression.json |
18 regression cases | statsmodels OLS + OLSInfluence, scipy linregress, numpy lstsq/polyfit, and a 60-digit mpmath solve of the normal equations |
Reference/gen_bootstrap.py → bootstrap.json |
6 bootstrap cases | numpy replicates, Efron's formulas, scipy.stats.bootstrap |
Versions used: see Reference/output/VERSIONS.txt (python 3.14.4, scipy 1.17.1, numpy 2.4.4, mpmath 1.3.0, statsmodels 0.14.6).
Reference/generate_all.sh regenerates every file into Reference/output/ and mirrors it into
Tests/ORIXStatisticsTests/ReferenceData/ (the copy bundled with the test target). All generators are seeded, so the output
is reproducible.
Why mpmath and not scipy as the primary reference for functions: scipy is itself wrong in places at the accuracy tested here.
Found while building the references (and recorded in the JSON under scipy_disagreements):
| Case | scipy 1.17.1 | 50-digit value | scipy relative error |
|---|---|---|---|
gammainc(1e6, 995000) |
2.749568445799219e-07 | 2.749580359270071e-07 | 4e-6 |
betaincinv(30, 5, 1e-300) |
1.0193e-11 | 6.9897e-11 | factor 6.9 |
t.ppf(1e-200, 3) |
−2.3976e66 | −4.7953e66 | factor 2 |
uniform(loc=-2, scale=7).sf(4.999999999993) |
9.999778782798785e-13 | 9.999620179509553e-13 | 1.6e-5 |
geom(0.9).ppf(0.999999999) |
9 | 10 in exact arithmetic (p is one ulp above cdf(9)) | see "discrete quantiles" |
Measured accuracy — special functions
Worst relative error over the whole grid of special.json (test SpecialFunctionReferenceTests; the asserted tolerance is in
the last column). Grid: shapes from 1e-5 to 1e6 (gamma) and 0.01 to 1e5 (beta), arguments from the far lower tail to the far
upper tail, probabilities from 1e-300 to 1 − 1e-16.
| Function | Cases | Worst relative error | Worst case | Asserted |
|---|---|---|---|---|
beta |
36 | 5.3e-15 | beta[50.0, 50.0] |
2e-14 |
regularizedBeta, a shape > 1000 |
143 | 6.2e-14 | betaI[0.007408658771797605, 1000.0, 100000.0] |
5e-13 |
regularizedBeta, shapes ≤ 1000 |
1229 | 4.5e-13 | betaI[0.999, 1000.0, 0.01] |
2e-12 |
regularizedBetaComplement, a shape > 1000 |
144 | 8.9e-14 | betaIc[0.014574111337532955, 1000.0, 100000.0] |
5e-13 |
regularizedBetaComplement, shapes ≤ 1000 |
1247 | 4.1e-13 | betaIc[0.03950695847036844, 0.01, 20.0] |
2e-12 |
inverseRegularizedBetaComplement |
521 | 2.4e-14 | betaIcinv[0.999, 0.05, 0.5] |
5e-13 |
inverseRegularizedBeta |
617 | 1.1e-13 | betaIinv[1e-10, 0.05, 0.5] |
5e-13 |
inverseErfc |
14 | 2.2e-16 | erfcinv[1e-08] |
2e-15 |
inverseErf |
26 | 4.9e-16 | erfinv[0.001] |
2e-15 |
gamma |
16 | 1.9e-16 | gamma[-10.1] |
1e-15 |
regularizedGammaP |
458 | 1.1e-13 | gammaP[100000.0, 90000.0] |
5e-13 |
inverseRegularizedGammaP |
125 | 8.6e-14 | gammaPinv[0.001, 0.7] |
5e-13 |
regularizedGammaQ |
457 | 5.4e-14 | gammaQ[10000.0, 12000.0] |
5e-13 |
inverseRegularizedGammaQ |
135 | 9.2e-15 | gammaQinv[0.05, 0.99] |
5e-13 |
logBeta |
81 | 5.9e-16 | logBeta[10.0, 0.5] |
2e-15 |
logGamma |
20 | 3.4e-16 | logGamma[1.5] |
1e-15 |
normalCDF (to z = −38) |
17 | 1.2e-13 | normCDF[-30.0] |
5e-13 |
normalQuantile(p) |
30 | 4.4e-16 | normQ[0.4] |
2e-15 |
normalQuantile(upperTail:) |
9 | 4.2e-16 | normQupper[0.3] |
2e-15 |
normalSurvival (to z = 37) |
18 | 9.8e-14 | normSF[37.0] |
5e-13 |
Reading: everything that is a thin wrapper over libm (lgamma, tgamma, erf, erfc) or a refinement on top of it is at
rounding level (≤ 6e-16), except Φ in the extreme tails where libm's own erfc is the limit (9e-15 at |z| = 10, 1.2e-13 at
|z| = 30–37). The incomplete gamma family stays below 1.2e-13 and the incomplete beta family below 4.5e-13 over the
whole grid, tails to 1e-300 included; the two worst incomplete-beta cases involve a shape of 0.01 (limitation 3 below).
The specification target was 1e-12 in the central range.
Ingredients that matter for these numbers (all in SpecialFunctions.swift):
the prefactors x^a·e^(−x)/Γ(a) and x^a·y^b/B(a, b) are evaluated through the Stirling remainder and log1p(u) − u, not through
exp(a·ln x − x − lnΓ(a)), so large shapes do not cancel; the beta continued fraction is the even contraction written with
a·y − b·x + 1 (DiDonato & Morris / Boost), which removes the cancellation of the textbook fraction when x is close to 1
(that fraction was measured at 3e-10 for Student t with ν = 1e7 and was replaced); Q(a, x) for a < 1, x < 1 is assembled from
expm1 and a dedicated lgamma(1 + a) series; every inverse iterates on the logarithm of the smaller tail inside a bracket
and falls back to Brent (ORIXNumerics.RootFinder).
Measured accuracy — distributions
Worst relative error per distribution over distributions.json (test DistributionReferenceTests, asserted tolerance 1e-12
for every row; the specification asked for 1e-10 in the central range with looser far tails). "Central" = reference value or
probability ≥ 1e-12; "far tail" = below, down to 1e-300 for values and 1e-200 for quantile probabilities. Discrete quantiles are
compared as exact integers (736 cases, 0 mismatches).
| Distribution | Parameter sets | Cases | pdf / pmf | cdf & survival | quantiles (both tails) | far-tail worst |
|---|---|---|---|---|---|---|
| normal | 4 | 456 | 2.5e-15 | 3.2e-15 | 6.4e-16 | 6.5e-15 |
| Student t (ν = 0.5 … 1e10) | 14 | 1920 | 4.4e-15 | 9.9e-15 | 2.9e-15 | 2.7e-14 |
| chi-square (k = 0.5 … 1e5) | 10 | 1318 | 8.4e-15 | 6.0e-15 | 4.4e-15 | 3.8e-14 |
| F | 10 | 1181 | 7.9e-15 | 7.1e-15 | 9.4e-15 | 2.6e-14 |
| gamma (shape 0.1 … 1000) | 7 | 915 | 1.9e-14 | 7.5e-15 | 2.9e-14 | 2.6e-14 |
| beta | 8 | 996 | 3.2e-14 | 5.7e-15 | 3.0e-14 | 6.5e-14 |
| exponential | 3 | 426 | 1.9e-15 | 8.5e-16 | 3.4e-16 | 1.8e-15 |
| uniform | 2 | 248 | 0.0e+00 | 4.3e-16 | 2.2e-15 | 0.0e+00 |
| binomial (n ≤ 10 000) | 7 | 740 | 1.0e-14 | 8.6e-15 | exact integers | 7.9e-15 |
| Poisson (λ ≤ 10 000) | 6 | 828 | 8.3e-15 | 9.3e-15 | exact integers | 7.8e-15 |
| geometric (trials) | 3 | 357 | 2.6e-15 | 2.3e-15 | exact integers | 5.3e-15 |
| geometric (failures) | 1 | 119 | 2.6e-15 | 1.9e-15 | exact integers | 2.7e-15 |
| negative binomial | 4 | 536 | 8.9e-15 | 7.8e-15 | exact integers | 9.6e-15 |
| discrete uniform | 2 | 124 | 0.0e+00 | 0.0e+00 | exact integers | 0.0e+00 |
logPDF was compared in absolute terms (|Δ|/max(1, |value|) ≤ 3.6e-15 everywhere). Moments (mean, variance) of all 81
parameter sets agree with scipy to 1e-12, with nil / .infinity where scipy reports nan / inf.
Reference cases (extract)
A readable extract of the comparisons the tests make; the full sets are the JSON files. "Obtained" is what this build returns.
| Case | Source | Expected | Obtained | Relative error | Asserted |
|---|---|---|---|---|---|
gammaP(0.5, 0.25) |
mpmath 50 d | 0.5204998778130465 | 0.5204998778130466 | 2.1e-16 | 5e-13 |
gammaQ(100, 500) |
mpmath 50 d | 1.5008794119250894e-106 | 1.5008794119251446e-106 | 3.7e-14 | 5e-13 |
gammaP(1000000, 997000) |
mpmath 50 d | 0.0013381041673135997 | 0.0013381041673135553 | 3.3e-14 | 5e-13 |
gammaQ(1E-05, 1) |
mpmath 50 d | 2.1938617914074714e-06 | 2.1938617914074706e-06 | 3.9e-16 | 5e-13 |
betaI(0.25, 2, 5) |
mpmath 50 d | 0.466064453125 | 0.46606445312499994 | 1.2e-16 | 2e-12 |
betaIc(0.999, 100, 100) |
mpmath 50 d | 4.104506907877991e-242 | 4.104506907877423e-242 | 1.4e-13 | 2e-12 |
betaI(0.001, 100, 100000) |
mpmath 50 d | 0.5172649144182913 | 0.5172649144182917 | 8.6e-16 | 2e-12 |
normQ(1E-300) |
mpmath 50 d | -37.0470962993612 | -37.0470962993612 | 0.0e+00 | 2e-15 |
normQ(0.975) |
mpmath 50 d | 1.9599639845400538 | 1.959963984540054 | 1.1e-16 | 2e-15 |
erfinv(0.999999) |
mpmath 50 d | 3.458910737275499 | 3.4589107372754984 | 1.3e-16 | 2e-15 |
gammaPinv(10, 1E-100) |
mpmath 50 d | 4.5287286883032135e-10 | 4.528728688303208e-10 | 1.3e-15 | 5e-13 |
betaIinv(1E-300, 30, 5) |
mpmath 50 d | 6.989673513975212e-11 | 6.989673513975239e-11 | 3.9e-15 | 5e-13 |
logBeta(10000000, 10000000) |
mpmath 50 d | -13862950.404734597 | -13862950.404734597 | 0.0e+00 | 2e-15 |
logGamma(-100.5) |
mpmath 50 d | -364.90096830942736 | -364.90096830942736 | 0.0e+00 | 1e-15 |
normal(0, 1) ppf(0.975) |
mpmath 50 d | 1.9599639845400538 | 1.959963984540054 | 1.1e-16 | 1e-12 |
normal(0, 1) isf(1E-200) |
mpmath 50 d | 30.20559417957964 | 30.205594179579645 | 1.2e-16 | 1e-12 |
t(5) ppf(0.975) |
mpmath 50 d | 2.5705818356363146 | 2.570581835636315 | 1.7e-16 | 1e-12 |
t(3) ppf(1E-200) |
mpmath 50 d | -4.795275720469223e+66 | -4.795275720469242e+66 | 3.9e-15 | 1e-12 |
t(10000000) isf(0.025) |
mpmath 50 d | 1.9599642217672055 | 1.9599642217672057 | 1.1e-16 | 1e-12 |
t(29.5) cdf(-3.390611053854093) |
mpmath 50 d | 0.0010000000000000009 | 0.0010000000000000018 | 8.7e-16 | 1e-12 |
chi2(10) isf(0.05) |
mpmath 50 d | 18.307038053275146 | 18.307038053275146 | 0.0e+00 | 1e-12 |
chi2(100000) ppf(0.01) |
mpmath 50 d | 98962.56777829022 | 98962.56777829026 | 4.4e-16 | 1e-12 |
chi2(3) sf(30.664849706213598) |
mpmath 50 d | 1.0000000000000006e-06 | 9.999999999999995e-07 | 1.1e-15 | 1e-12 |
F(3, 10) isf(0.05) |
mpmath 50 d | 3.7082648190468444 | 3.7082648190468435 | 2.4e-16 | 1e-12 |
F(250, 1000) cdf(1.1327757729147325) |
mpmath 50 d | 0.9000000000000004 | 0.9 | 3.7e-16 | 1e-12 |
gamma(2.5, 0.5) ppf(0.5) |
mpmath 50 d | 4.351460191095527 | 4.3514601910955255 | 4.1e-16 | 1e-12 |
beta(2, 5) ppf(0.9) |
mpmath 50 d | 0.5103163065514916 | 0.5103163065514917 | 2.2e-16 | 1e-12 |
beta(500, 300) sf(0.6643038093511993) |
mpmath 50 d | 0.009999999999999834 | 0.009999999999999842 | 8.7e-16 | 1e-12 |
binomial(100, 0.3) cdf(30) |
mpmath 50 d | 0.5491236007687905 | 0.549123600768791 | 8.1e-16 | 1e-12 |
binomial(10000, 0.5) sf(5301) |
mpmath 50 d | 8.108211552360942e-10 | 8.108211552360937e-10 | 6.4e-16 | 1e-12 |
binomial(100, 0.3) ppf(0.95) |
mpmath 50 d | 38.0 | 38.0 | 0.0e+00 | exact |
poisson(4.5) pmf(4) |
mpmath 50 d | 0.1898076205401244 | 0.1898076205401244 | 0.0e+00 | 1e-12 |
poisson(10000) cdf(9304) |
mpmath 50 d | 9.866065294581133e-13 | 9.866065294581145e-13 | 1.2e-15 | 1e-12 |
negativeBinomial(5.5, 0.7) sf(3) |
mpmath 50 d | 0.23148168919205292 | 0.23148168919205306 | 6.0e-16 | 1e-12 |
geometric(0.2) ppf(0.5) |
mpmath 50 d | 4.0 | 4.0 | 0.0e+00 | exact |
| one-sample t (n = 8): t | scipy ttest_1samp |
0.8484810168315478 | 0.8484810168315494 | 2.0e-15 | 1e-10 |
| … p-value | scipy | 0.4242313636551264 | 0.4242313636551256 | 2.0e-15 | 1e-10 |
| … CI lower | scipy confidence_interval |
37.75190080097814 | 37.75190080097814 | 0.0e+00 | 1e-10 |
| Welch t: Welch–Satterthwaite df | scipy ttest_ind(equal_var=False) |
21.769922023224975 | 21.76992202322498 | 1.6e-16 | 1e-10 |
| … p-value | scipy | 0.0061691384318944675 | 0.0061691384318945134 | 7.5e-15 | 1e-10 |
| Wilson 95% CI, x = 7, n = 20: lower | scipy binomtest.proportion_ci |
0.1811918241010821 | 0.18119182410108206 | 1.5e-16 | 1e-9 |
| Clopper–Pearson 95% CI: upper | scipy binomtest.proportion_ci(exact) |
0.5921885345328283 | 0.592188534532828 | 3.7e-16 | 1e-9 |
| χ² independence 2×3: statistic | scipy chi2_contingency |
10.859433596275702 | 10.859433596275702 | 0.0e+00 | 1e-10 |
| … p-value | scipy | 0.004384337279348025 | 0.0043843372793480224 | 5.9e-16 | 1e-10 |
| … Cramér's V | scipy contingency.association |
0.32953654723377346 | 0.32953654723377346 | 0.0e+00 | 1e-10 |
| χ² 2×2 with Yates: statistic | scipy chi2_contingency(correction=True) |
3.831932773109244 | 3.8319327731092434 | 1.2e-16 | 1e-10 |
| one-way ANOVA (3 groups): F | scipy f_oneway |
4.437025859172229 | 4.43702585917223 | 2.0e-16 | 1e-10 |
| … p-value | scipy | 0.02405024962743939 | 0.02405024962743936 | 1.3e-15 | 1e-10 |
| exact binomial test x = 7, n = 20, p₀ = 0.5: p | scipy binomtest |
0.26317596435546875 | 0.26317596435546886 | 4.2e-16 | 1e-9 |
| Longley β(intercept) | 60-digit solve = NIST StRD | -3482258.6345958184 | -3482258.634593751 | 5.9e-13 | 1e-10 |
| Longley β(GNPDEFL) | 60-digit solve = NIST StRD | 15.061872271373323 | 15.061872271241008 | 8.8e-12 | 1e-10 |
| Longley β(GNP) | 60-digit solve = NIST StRD | -0.03581917929259102 | -0.035819179292534656 | 1.6e-12 | 1e-10 |
| Longley β(UNEMP) | 60-digit solve = NIST StRD | -2.020229803816825 | -2.0202298038160094 | 4.0e-13 | 1e-10 |
| Longley β(ARMED) | 60-digit solve = NIST StRD | -1.033226867173592 | -1.0332268671732106 | 3.7e-13 | 1e-10 |
| Longley β(POP) | 60-digit solve = NIST StRD | -0.05110410565358071 | -0.051104105653569605 | 2.2e-13 | 1e-10 |
| Longley β(YEAR) | 60-digit solve = NIST StRD | 1829.151464613552 | 1829.1514646124872 | 5.8e-13 | 1e-10 |
| Longley SE(intercept) | 60-digit solve | 890420.3836073726 | 890420.3836076737 | 3.4e-13 | 1e-10 |
| Longley residual SD | statsmodels | 304.8540735621171 | 304.85407356205235 | 2.1e-13 | 1e-10 |
| Longley R² | statsmodels | 0.9954790045772911 | 0.995479004577293 | 1.9e-15 | 1e-10 |
| Longley max Cook's distance | statsmodels OLSInfluence |
0.6139168381069071 | 0.6139168381918801 | 1.4e-10 | 1e-6 |
| Longley Durbin–Watson | statsmodels | 2.559487689281959 | 2.5594876892808527 | 4.3e-13 | 1e-7 |
| Longley max VIF | statsmodels | 1788.5134827182983 | 1788.513482718443 | 8.1e-14 | 1e-8 |
| simple regression (n = 12): slope | scipy linregress |
1.1493924321384048 | 1.1493924321384046 | 1.9e-16 | 1e-11 |
| … SE(slope) | scipy linregress |
0.06996369402133151 | 0.06996369402133194 | 6.1e-15 | 1e-9 |
| … prediction interval upper at x₀ = 7.3 | statsmodels get_prediction |
15.028417392889148 | 15.028417392889148 | 0.0e+00 | 1e-10 |
| ExpReg b (y = a·bˣ) | scipy linregress on ln y |
1.3449014573079836 | 1.3449014573079836 | 0.0e+00 | 1e-9 |
| ExpReg R² (on ln y) | scipy | 0.9763059486168801 | 0.9763059486168802 | 1.1e-16 | 1e-9 |
| ExpReg R² (on y) | numpy | 0.9788123520760399 | 0.9788123520760401 | 2.3e-16 | 1e-9 |
| BCa lower limit on fixed replicates | Efron (1987) in numpy/scipy | 2.934641388510817 | 2.934641388510817 | 0.0e+00 | 1e-9 |
| sample skewness G₁ (n = 200) | scipy skew(bias=False) |
3.143565196647018 | 3.1435651966470184 | 1.4e-16 | 1e-11 |
| sample excess kurtosis G₂ | scipy kurtosis(bias=False) |
17.60326024533058 | 17.603260245330578 | 2.0e-16 | 1e-11 |
| quantile 0.9, R-7 | numpy quantile(method='linear') |
3.855808847171133 | 3.855808847171133 | 0.0e+00 | 1e-13 |
| quantile 0.9, R-6 | numpy quantile(method='weibull') |
3.8887385752664327 | 3.8887385752664327 | 0.0e+00 | 1e-13 |
| sample variance of 1e9 + small values | exact rational arithmetic | 2.141736102534665 | 2.141736102534665 | 0.0e+00 | 1e-13 |
Longley: the worst coefficient error of this package's QR is 8.8e-12 against the 60-digit solution; numpy.linalg.lstsq (SVD)
on the same data is at 1.0e-11. On the well-conditioned sets the coefficient error is ≤ 6e-15. The test additionally asserts that
our error never exceeds 10× numpy's on any data set.
Bootstrap end to end (lognormal sample, n = 30, statistic = mean, 95% BCa, 20 000 resamples, different random streams):
this package (2.8989, 4.5433), scipy.stats.bootstrap (2.9031, 4.5123); bootstrap standard error 0.4089 vs 0.4059.
The interval arithmetic itself is exact on shared replicates (row above).
What the test suite checks (85 tests, 15 suites)
| Area | Checks |
|---|---|
| Special functions | whole mpmath grid; Acklam's raw approximation shows its published 1.15e-9 bound (guards against a mistyped coefficient); P + Q = 1, I_x(a,b) + I_{1−x}(b,a) = 1; closed forms; inverse ∘ forward round trips down to p = 1e-250; combinatorics; domain errors (poles, negative shapes, arguments and probabilities out of range, overflow) |
| Distributions | whole mpmath grid (pdf/pmf, logpdf, cdf, survival, quantile, upper-tail quantile) for 14 distributions and 81 parameter sets; cdf(quantile(p)) and survival(quantile(upperTail: q)) round trips; ∫pdf = Δcdf by Gauss–Kronrod (ORIXNumerics); mean and variance by tanh–sinh integration of the quantile function and by summing the pmf; moments vs scipy; discrete-quantile definition (cdf(k) ≥ p and cdf(k − 1) < p) on every discrete distribution; tail-aware interval probabilities; degenerate parameters; unbounded densities return +∞; parameter and probability errors for every type |
| Random numbers | SplitMix64 published vector (seed 1234567); xoshiro256** published vector (state 1, 2, 3, 4); same seed ⇒ same stream, copies continue identically, streams differ; uniform ∈ [0, 1), openUniform ∈ (0, 1); uniformInteger passes a χ² test; every sampler (21 distribution / parameter combinations, both algorithm branches of binomial and Poisson) — sample mean and variance within 5 standard errors and empirical CDF within 5 binomial standard errors at 7 quantiles, n = 200 000, fixed seeds |
| Descriptive | all nine quantile definitions vs numpy.quantile (8 data sets × 15 probabilities × 9 methods = 1 080 values); Sturges / Freedman–Diaconis / fixed histograms vs numpy.histogram; skewness and kurtosis (both conventions) vs scipy; ranks, Spearman with ties, covariance, z-scores, geometric / harmonic / weighted means; Tukey and TI quartiles; mode conventions; compensated summation under catastrophic cancellation; error paths (empty, n too small, NaN, undefined) |
| Inference | every interval and test vs scipy / statsmodels for all three alternatives (see table); "Stats" input equals "Data" input; decision ⇔ null value outside the interval; verification warnings fire exactly when the condition holds (expected count < 5, n·p < 10, n < 30, Yates requested on a larger table); explanations contain the computed numbers; error paths |
| Regression | simple regression vs linregress and statsmodels, and vs the independent QR path; multiple regression (5 data sets, with and without intercept) vs a 60-digit solve and statsmodels diagnostics; Longley; six curve models vs numpy / scipy with R² on both scales; exact fits; rank deficiency (exact combination, duplicate, constant column) with the explanatory message; error paths |
| Simulation | π/4 and two-dice experiments within their Monte Carlo standard error; bit-identical results for the same seed; async = sync; progress callbacks; cooperative cancellation (CancellationError); scenario table and one-way sensitivity on a loan model; sampling distribution of the mean (σ/√n); bootstrap interval arithmetic vs the reference on identical replicates; end-to-end bootstrap vs scipy.stats.bootstrap; seed reproducibility; paired resampling; error paths |
Run: swift build && swift test in Packages/ORIXStatistics (zero warnings). ORIX_ACCURACY_REPORT=1 swift test prints the
accuracy tables above (SPC|… and ACC|… lines).
Limitations and things that were not validated (stated plainly)
- NIST StRD Norris is not included. Its 36 observations could not be reproduced from memory with certainty, so no
"certified" Norris value appears anywhere. Longley is covered: the data come from
statsmodels.datasets.longley, the reference is a 60-digit solve of exactly those numbers, and that solve agrees with the NIST certified coefficients, residual standard deviation and R² to all 15 printed digits (for UNEMP and ARMED after the factor 100 by which that copy's units differ). The certified values were typed from memory, which is why the 60-digit solve — not the typed constants — is the reference the tests rely on; the agreement of the two is asserted separately. - Higher-degree polynomial fits use raw powers of x. The QR on equilibrated columns handles the degree-5 test case to 1e-6 relative, but NIST-style degree-10 problems (Filip, Wampler 4–5) were not attempted and should not be expected to work in double precision; a verification warning is attached from degree 4 upwards.
- Tiny beta shapes. When one shape of the incomplete beta function is below about 0.01 and the requested tail is the small complement of a value near 1, up to log10(1/tail) digits are lost (worst measured: 4.5e-13 at shape 0.01). This reaches Student t / F with fewer than 0.02 degrees of freedom, not ordinary use.
- Quantiles next to a finite support bound (e.g. the 1 − 1e-12 quantile of a Beta, or the 1e-12 quantile of a
Uniform(2, 9)) are limited by the spacing of doubles around the bound, not by the algorithm: the value returned is the
correctly rounded neighbour, and
cdf(quantile(p))can differ from p by one ulp of x times the density. - Discrete quantiles and rounding.
quantile(p)compares against p·(1 − 64ε) (R's convention), so a p that equals a CDF value up to rounding returns that k. In exact arithmetic on the binary value of p the answer can be k + 1; such cases (2 of 736 in the grid) are flaggedaltin the reference and both answers are accepted. scipy makes the same choice. - Very large parameters. Distribution functions were validated up to shapes of 1e5 (gamma family to 1e6, Student t to
ν = 1e10 through Hill's expansion, binomial to n = 10 000 and spot-checked at n = 1e6 against mpmath, Poisson to λ = 1e4 and
spot-checked at 1e6). Beyond that the series / continued fractions still converge (iteration caps grow as √parameter) but
were not measured. Inputs to pmf with n above ~1e9 lose digits through
1 − pand are conditioned accordingly. - Random generators. The two generators reproduce published vectors and every sampler passes moment and CDF checks at n = 200 000; no formal battery (TestU01, PractRand) was run here — xoshiro256** has passed them in the literature. The generator is not cryptographic.
- Bootstrap. The interval formulas are validated exactly on fixed replicates; the end-to-end comparison with
scipy.stats.bootstrapis statistical (different random streams) with a tolerance of 6% of the interval width. BCa needs n extra evaluations of the statistic (jackknife) and is undefined when all replicates fall on one side of the estimate (an explicit error suggests the percentile method). - Exact binomial test sums the pmf over all outcomes (O(n) time) and is limited to 10 million trials.
- Assumption checks are what they say. Normality cannot be verified from summary statistics; the verification report marks it as an assumption (warning under n = 30) rather than pretending to have tested it. No normality test (Shapiro–Wilk, Anderson–Darling) is implemented.
- Not implemented (outside the brief): non-central distributions, non-parametric tests (Mann–Whitney, Wilcoxon, Kruskal–Wallis), two-way ANOVA, post-hoc comparisons, weighted or logistic regression, non-linear least squares (the exponential and power models are the linearised calculator fits, labelled as such).
- Platform. Built and tested on macOS (arm64, Swift 6.4.2, Swift 6 language mode, strict concurrency). The package
declares iOS 26; it uses only Foundation / libm (
lgamma_r,tgamma,erf,erfc), so results are expected to be identical on iOS arm64, but the tests were not executed on a device or simulator here.