SPB Git

Repository Public

testing.md 24.2 KB 321 lines markdown

1# Testing23The test strategy by layer, how to run the suites, where the fixtures come from, which tests are gated,4and what the automated tests do not cover.56Related documents: [setup.md](setup.md), [performance.md](performance.md),7[accessibility.md](accessibility.md), [localization.md](localization.md),8[../mathematics/reference-tests.md](../mathematics/reference-tests.md) (the reference sources and their9case counts), [../release/checklist.md](../release/checklist.md).1011## 1. Where the numbers are1213Test counts and pass/fail status change with every commit. The authoritative records are14**`docs/reports/tests.json`**, written by `Scripts/test-all.sh` (see section 3), and15**`docs/reports/app-tests.json`** for the app-hosted bundle. The table at the end of this document16([section 8](#8-recorded-test-run)) is copied from those two files: 22 packages, 1,270 test functions,17no failed package, plus 9 app-hosted tests with no failure.1819## 2. Strategy by layer2021<!-- diagram: testing-layers -->22```mermaid23flowchart TD24    A["App integration tests<br/>XCTest hosted in the app<br/>App/Tests"] --> B["Feature packages<br/>view models and adapters<br/>Feature*/Tests"]25    B --> C["UI kit<br/>session, layout, accessibility tree, contrast<br/>SheetUI/Tests"]26    B --> D["Domain libraries<br/>SheetAnalytics, SheetData, SheetCharts,<br/>SheetPython, SheetIntelligence, SheetIO"]27    B --> F["Function libraries<br/>FunctionsNumeric, FunctionsText, FunctionsLookup"]28    C --> E["Core<br/>SheetCore, SheetFormat, SheetEngine"]29    D --> E30    F --> E31```32SVG fallback: ![Test layers](../assets/diagrams/testing-layers.svg)3334Each box is tested by its own suite; an arrow means "is built on". The domain and function libraries35are the two layers whose expected values come from external reference programs (see36[../mathematics/reference-tests.md](../mathematics/reference-tests.md)).3738| Layer | Packages | What the tests are | External reference |39|---|---|---|---|40| Core model and engine | `SheetCore`, `SheetEngine` | hand-written expectations: addresses, parser, semantics, recalculation, editing with undo, input parsing, a performance smoke test; cross-library model and registry checks | none |41| Number formats | `SheetFormat` | LibreOffice fixture replay with named deviation rules, hand-written edge cases, a fuzz test, a micro-benchmark suite | LibreOffice |42| Worksheet functions | `FunctionsNumeric`, `FunctionsText`, `FunctionsLookup` | fixture replay (one parameterised test per fixture file), Microsoft documentation examples, hand-written edge cases, a test that fails when a registered function has no reference | LibreOffice, SciPy, mpmath, NumPy, statsmodels, Python `decimal`, Excel-computed public data |43| File formats | `SheetIO` | read and write tests per format, ZIP and XML hardening (hostile archives, fuzzing of truncated and corrupted files), native-format integrity, interoperability with other programs, gated performance tests | openpyxl, pandas, LibreOffice |44| Statistics, finance, what-if, audit | `SheetAnalytics` (targets `StatisticsTests`, `FinanceModelsTests`, `WhatIfTests`, `AuditTests`, shared `TestSupport`) | fixture replay against Python libraries; exact hand-derived expectations for the deterministic analyses | SciPy, NumPy, pandas, statsmodels, scikit-learn, `arch`, Python `decimal` |45| Data engine | `SheetData` | engine, import, transform, pivot and sheet-bridge tests on seeded synthetic files | pandas (no DuckDB on the reference side) |46| Charts | `SheetCharts` | numeric algorithms against Python references; every chart kind × difficult data rendered to SVG (parsed back with `XMLParser`), PDF and PNG; structural checks on the display list; vector-PDF heuristics; JSON round trips | NumPy, SciPy, matplotlib |47| Python runtime | `SheetPython` | execution, API, discovery, records and security tests that run **real interpreters**; each sandbox denial is asserted, and the same probe is asserted to succeed without the sandbox | the interpreters found on the Mac |48| Intelligence | `SheetIntelligence` | context building, validators and post-checks with a fake backend; `RealModelTests` against the on-device model, opt-in (`ORIX_AI_EVAL=1`, section 5) | the on-device model (measurement, not a reference) |49| UI kit | `SheetUI` | `AxisLayoutTests`, `SessionTests` (edit / commit / undo, navigation, merge-aware selection, pointing, find, background recalculation, print pagination), `ContrastTests`, `LocalizationTests`, `AccessibilityTreeTests` | WCAG 2.1 contrast formula |50| Feature panels | `FeatureAudit`, `FeatureCharts`, `FeatureData`, `FeatureAnalysis`, `FeaturePython`, `FeatureIntelligence`, `FeatureAutomation` | view models, providers and adapters driven without a window; localization completeness; integration with a real `WorkbookSession` | — |51| Application | `App/Tests` | XCTest bundle hosted inside the running app: real documents, windows and grid, synthesized key and mouse events | — |52| Templates | `SheetTemplates` | structure of each template, key outputs against `expected.json`, save and reload, LibreOffice recalculation of every template, thumbnails (section 7) | numpy / scipy, LibreOffice |53| Automation engine | `SheetAutomation` | workflow model, engine, steps, outputs, template merge, validation, the on-disk store, the `orix-sheet` command line (section 7) | — |5455Source: the test files under `Packages/*/Tests` and `App/Tests` (file and suite names), `Packages/SheetCharts/README.md` ("Tests"), `Packages/SheetPython/SECURITY.md`, `Packages/SheetAnalytics/VALIDATION.md`, `Packages/SheetIO/COMPATIBILITY.md`.5657The packages use **Swift Testing** (`@Test`, `@Suite`, `#expect`); only the app-hosted bundle uses58XCTest (`final class AppIntegrationTests: XCTestCase`).5960## 3. Running the tests6162### Everything, with a report6364```sh65Scripts/test-all.sh                      # every package, sequentially66Scripts/test-all.sh SheetCore SheetIO    # only the packages named67Scripts/build.sh test                    # same as Scripts/test-all.sh68```6970What `Scripts/test-all.sh` does, read from the script:71721. For each package, in this order — `SheetCore SheetFormat FunctionsNumeric FunctionsText73   FunctionsLookup SheetEngine SheetIO SheetCharts SheetData SheetAnalytics SheetPython74   SheetIntelligence SheetAutomation SheetTemplates SheetUI FeatureAudit FeatureCharts FeatureData75   FeatureAnalysis FeaturePython FeatureIntelligence FeatureAutomation` — it runs76   `ORIX_AI_EVAL=0 swift test` (a **debug** build) and stores the output, stripped of colour codes, in77   `build/test-logs/<Package>.log`. A package without a `Package.swift` is skipped.782. The test count is the sum of every `Test run with N tests` line (Swift Testing; a package may have79   several test targets) plus the last `Executed N tests` line (XCTest). A parameterised Swift Testing function counts once, whatever the80   number of arguments; a fixture file with thousands of cases is therefore one test.813. A package is `passed` when every such line contains "passed" **and** the log contains neither `error: `82   nor `signal code`; otherwise it is `FAILED`. A failing package does not stop the others.834. It prints one line per package (name, tests, status, seconds) and a total, and writes84   `docs/reports/tests.json`:85   `{"date", "total", "failedPackages", "packages": [{"package", "tests", "status", "seconds"}]}`.865. The exit status is 1 when at least one package failed.8788The header comment of the script says it writes `tests.json` "+ a table"; the table is printed on89standard output only, it is not written to a file.9091The on-device model evaluation is opt-in (`ORIX_AI_EVAL=1`, section 5); any other value, including the92`0` the script sets, leaves it skipped. It takes minutes (mean 28.2 s per request over 20 requests in93`Packages/SheetIntelligence/EVALUATION.md`).9495### One package, one test9697```sh98cd Packages/FunctionsLookup && swift test99cd Packages/SheetCore && swift test --filter RecalcTests100cd Packages/SheetIO && swift test --filter HardeningTests101```102103Use `--scratch-path <dir>` when another build of the same package is running.104105### The app-hosted tests106107`project.yml` declares the `ORIXSheetTests` bundle (sources in `App/Tests`, `TEST_HOST` = the Debug app)108and adds it to the `test` action of the `ORIXSheet` scheme, which passes `-ApplePersistenceIgnoreState YES`109and `-ORIXTesting YES` to the host app. No script in `Scripts/` runs it: `Scripts/build.sh test` and110`Scripts/test-all.sh` cover the packages only. The command is:111112```sh113Scripts/build.sh generate114xcodebuild test -project ORIXSheet.xcodeproj -scheme ORIXSheet -destination 'platform=macOS,arch=arm64'115```116117The recorded result is in `docs/reports/app-tests.json`: 9 tests, 0 failures118(2026-11-21T08:48:35Z). It is a separate file; the app-hosted tests are not part of119`docs/reports/tests.json`.120121`AppIntegrationTests.swift` contains nine tests: features registered; typing formulas in the grid and122undo; mouse selection and redraw; native save and reopen is lossless; foreign files are imported as123untitled and export is preflighted; corrupted and unsupported files fail cleanly; printing produces a124paginated vector PDF; the menu bar has no duplicate shortcuts; a large sheet (50 000 rows × 4 columns,125100 000 formulas) recalculates in the background and repaints under a bound.126127## 4. Fixtures and generators128129Reference values are committed; `swift test` needs neither Python nor LibreOffice to replay them.130131| Package | Fixtures (`Tests/<Target>/Fixtures/`) | Generators (`Tests/<Target>/Generators/`) |132|---|---|---|133| FunctionsNumeric | 24 JSON files (`lo_*.json`, `scipy_*.json`, `numpy_descriptive.json`, `statsmodels_regression.json`, `decimal_rounding.json`, `cmath_complex.json`), 1.5 MiB | `gen_libreoffice.py`, `gen_scipy.py`, `common.py`, `data.py`, `overrides.py`, `coverage.py` |134| FunctionsText | `function_references.json`, 0.8 MiB | `generate_function_fixtures.py`, `lo_recalc.py` |135| FunctionsLookup | `excel_reference.json`, `financial.json`, `securities.json`, `lookup_lo.json`, `arrays.json`, 9.6 MiB | `gen_lookup.py`, `gen_arrays.py`, `gen_financial.py`, `gen_securities.py`, `gen_excel_reference.py`, `lo_harness.py`, `pyref.py` |136| SheetFormat | `libreoffice_formats.json`, 0.2 MiB | `generate_format_fixtures.py`, `lo_recalc.py` |137| SheetAnalytics | 7 JSON files for `StatisticsTests`, `finance.json` for `FinanceModelsTests`, 1.3 MiB | `gen_core.py`, `gen_descriptive_inference.py`, `gen_regression.py`, `gen_timeseries.py`, `gen_multivariate.py`, `gen_resampling.py`, `gen_tukey.py`, `gen_finance.py`, `gen_common.py` |138| SheetData | 14 files: CSV dialects, Parquet, JSON Lines, SQLite, Arrow, `expected.json`, 1.9 MiB | `generate_fixtures.py` |139| SheetCharts | `reference.json`, `colormaps.json`, 0.2 MiB | `gen_fixtures.py`, `gen_colormaps.py` |140| SheetIO | 7 files: XLSX from openpyxl and LibreOffice, an ODS, a raw XLSX, a macro workbook, 0.1 MiB | `gen_openpyxl_corpus.py`, `gen_raw_xlsx.py`, `gen_libreoffice_corpus.sh`, `dump_xlsx.py` |141| SheetTemplates | `expected.json` | `gen_expected.py` (reads `inputs.json`) |142143Source: `find Packages -path '*Fixtures*' -type f` and `find Packages -path '*/Generators/*'`, sizes summed with `stat`, run on 2026-11-21.144145The other packages have no fixture directory: their tests build their inputs in code (synthetic146workbooks, temporary files).147148How each corpus is built, what the references are and how to regenerate them is described in149[../mathematics/reference-tests.md](../mathematics/reference-tests.md). Two rules apply to anyone adding150a function or a format:151152- add reference cases to the generator, not expected values copied from the implementation;153- when a reference program is known to deviate from Excel's documented behaviour, record the Excel value154  **and** the reference's value with a reason (the `overrides.py`, `excel` source tag and155  `ExcelDeviations.swift` mechanisms), so that the deviation stays visible.156157## 5. Gated and conditional tests158159### Opt-in by environment variable160161| Variable | Package / test | Effect |162|---|---|---|163| `ORIX_AI_EVAL=1` | `SheetIntelligence`, `RealModelTests.oversizedPromptMapsToContextWindowExceeded` and `RealModelTests.fullEvaluation` | enables the two tests that generate text with the on-device model; `fullEvaluation` runs the 20 requests and rewrites `Packages/SheetIntelligence/EVALUATION.md`. With the variable unset or set to anything else (`0` included) both are reported as skipped, the model generates nothing and the file is not rewritten |164| `ORIX_BENCH=1` | `SheetIO`, `PerformanceTests` | enables the 1,000,000-cell and 5,000,000-cell runs: `ORIX_BENCH=1 swift test -c release -Xswiftc -enable-testing --filter PerformanceTests` |165| `ORIX_DATA_LARGE_TESTS=1` | `SheetData`, `millionRowPipeline` | enables the 1,000,000-row end-to-end test: `ORIX_DATA_LARGE_TESTS=1 swift test --filter millionRow` |166| `FN_DUMP=<dir>` | `FunctionsNumeric`, `ReferenceFixtureTests` | writes every mismatch of each fixture to `<dir>/<fixture>.txt` (to review a regenerated corpus) |167| `ORIX_TEMPLATES_SKIP_LO` | `SheetTemplates`, `LibreOfficeCrossCheck` | skips the LibreOffice recalculation of the templates |168| `ORIX_TEMPLATES_INPUTS=<file>` | `SheetTemplates`, `exportGeneratorInputs` | writes the inputs read by `gen_expected.py` |169| `ORIX_TEMPLATES_DUMP=<file>` | `SheetTemplates`, `dumpNamedValues` | debug aid: the value of every defined name of every template |170| `ORIX_TEMPLATES_THUMBS=<dir>` | `SheetTemplates`, `Thumbnails` | also writes the thumbnail PNGs |171| `ORIX_TEMPLATES_NO_FEATURES`, `ORIX_TEMPLATES_NO_PREVIEW` | `SheetTemplates/Package.swift` | build the package without the sibling feature packages / without the preview harness |172| `ORIX_SHEET_SUPPORT_DIR` | `SheetPython`, `SheetAutomation` | points the application-support directory at a temporary location |173174Source: `grep -rn 'ProcessInfo.processInfo.environment' Packages --include='*.swift'` and the files it lists, run on 2026-11-21.175176### Disabled when a tool or a permission is missing177178These tests are **skipped, not failed**, when their precondition is not met. A green run on a machine179without the tools has therefore verified less than a green run on the build machine.180181| Condition | Tests affected |182|---|---|183| `/opt/homebrew/bin/python3` is not executable (`Tools.hasPython`) | `SheetIO`: three `InteropTests` tests and one `CSVTests` test (the checks that run openpyxl and pandas on files SheetIO wrote) |184| `/opt/homebrew/bin/soffice` is not executable (`Tools.hasLibreOffice`) | `SheetIO`: one `InteropTests` test and one `ODSTests` test; `SheetTemplates`: the LibreOffice cross-check returns early |185| No Python ≥ 3.9 interpreter found (`Interp.any`) | most of `SheetPython` (execution, security, discovery, API) and four `FeaturePython` integration tests |186| No interpreter with pandas + numpy + matplotlib (`Interp.scientific`) | the DataFrame, plotting and large-data tests of `SheetPython` and `FeaturePython` |187| No interpreter without pandas (`Interp.bare`, normally `/usr/bin/python3`) | one `SheetPython` API test |188| polars not installed | one `SheetPython` API test |189| The on-device model is unavailable (with `ORIX_AI_EVAL=1`) | the two opt-in `RealModelTests` return early after printing the reason; `EVALUATION.md` is not written |190| The test runner is not trusted for the accessibility API (`AXIsProcessTrusted()`) | the whole `AccessibilityTreeTests` suite of `SheetUI` |191192Source: `grep -rn '\.enabled(' Packages/*/Tests`, `Packages/SheetIO/Tests/SheetIOTests/TestSupport.swift`, `Packages/SheetPython/Tests/SheetPythonTests/Support.swift`, `Packages/SheetIntelligence/Tests/SheetIntelligenceTests/RealModelTests.swift`, `Packages/SheetUI/Tests/SheetUITests/AccessibilityTests.swift`.193194Note on `RealModelTests`: `availabilityIsReported` always runs. It asks the system whether the model is195available and generates nothing; that query is the only contact with the real backend in a plain196`swift test` or a `Scripts/test-all.sh` run.197198## 6. UI and automation testing199200### What is automated201202- **Session-level tests without a window** (`SheetUI/SessionTests`): the `WorkbookSession` is the main203  actor object behind every window; editing, committing, undo, navigation, selection across merged204  cells, formula pointing, the find model, background recalculation and print pagination are tested on205  it directly.206- **Accessibility tree of a real view** (`SheetUI/AccessibilityTreeTests`): a `WorkbookView` is put in an207  offscreen window and read through the system accessibility API; the numbers found (elements, controls,208  controls without a label) are reported in [accessibility.md](accessibility.md).209- **Contrast** (`SheetUI/ContrastTests`): every text / background pair of the design tokens is measured210  in light and dark and the test fails under 4.5:1 for text and 3:1 for glyphs and structural lines.211- **Feature view models** (`Feature*/Tests`): each panel exposes its state as a model that the tests212  drive without rendering; `FeatureData` adds a `DataAutomation` entry point used by its tests and by213  the `data-preview` benchmark.214- **App-hosted integration** (`App/Tests`): real `WorkbookDocument`s and windows inside the running app;215  key events are created with `NSEvent.keyEvent` and delivered to `GridView.keyDown`, mouse selection216  uses the grid's own geometry, saving and reopening goes through the document layer, printing is217  rendered to a PDF and its page count checked.218- **Menu and shortcut consistency**: `testMenuBarHasNoDuplicateShortcuts` walks the main menu and219  fails on any key equivalent used twice;220  `-ORIXDumpMenus <json>` writes the menu tree for the localization checks221  ([localization.md](localization.md)).222- **Localization**: `Scripts/localize.py --check` and the `LocalizationTests` suites of `SheetUI` and223  `FeatureAudit`.224225### What is done by eye226227- **Screenshots from the app itself.** The app (`-ORIXShot`) and every preview executable (`--shot`)228  render their window to a PNG with `cacheDisplay(in:to:)` and exit. These images were reviewed by a229  person in light and dark, English and French ([accessibility.md](accessibility.md) lists which230  screens). Nothing compares them automatically: there is **no snapshot or golden-image test**.231232### Limits233234- **There is no XCUITest target.** No test launches the app as a separate process and drives it through235  the accessibility interface; Apple's `performAccessibilityAudit()` was not run236  ([accessibility.md](accessibility.md)).237- **Synthesized events are not real input.** The app-hosted tests call `keyDown` on the grid with238  constructed `NSEvent`s and feed the rest of the text through the session "like the field editor239  does"; the system text-input path (input methods, dead keys, the real first-responder chain) is not240  exercised.241- **VoiceOver was not listened to**; Full Keyboard Access, Voice Control and Switch Control were not242  exercised; Increase Contrast, Differentiate Without Colour and Reduce Motion were not switched on243  during a run.244- **Settings, Page Setup, Help and the compatibility report** were not put through the accessibility245  tree audit.246- **No test opens a file written by ORIX Sheet in Microsoft Excel.**247- **Drag and drop, the open and save panels, printing to a physical printer, window restoration and248  multi-window behaviour** have no automated test.249- **Timing assertions are loose on purpose.** `testLargeSheetStaysResponsive` asserts a worst repaint250  under 50 ms ("generous bound for CI noise") and `testMouseSelectionAndRedraw` under 100 ms; they catch251  regressions of an order of magnitude, not of a few percent. Use the benchmarks of252  [performance.md](performance.md) for measurements.253254## 7. Templates and automation255256`SheetTemplates`, `SheetAutomation` and `FeatureAutomation` are built into the app (`project.yml`,257`App/Sources/Features.swift`) and are part of the `Scripts/test-all.sh` package list.258259| Package | Test files | What they cover, by suite name |260|---|---|---|261| SheetTemplates | `StructureTests.swift` (`Library`, `Structure`), `NumericTests.swift` (`Numeric`), `IOTests.swift` (`InputOutput`), `LibreOfficeTests.swift` (`LibreOfficeCrossCheck`), `ThumbnailTests.swift` (`Thumbnails`), `GeneratorInputs.swift` (`GeneratorInputs`) | structure of each template, key outputs against `expected.json` computed with numpy / scipy, save and reload, LibreOffice recalculation with one recorded accepted difference, thumbnail rendering. See [../templates.md](../templates.md) |262| SheetAutomation | `ModelTests.swift`, `EngineTests.swift`, `StepTests.swift`, `OutputTests.swift`, `TemplateMergeTests.swift` (`TemplateTests`, `MergeTests`), `ValidatorStoreTests.swift` (`ValidatorTests`, `StoreTests`), `CLITests.swift` | workflow model, engine, steps, outputs, template merge, validation, the on-disk store, the `orix-sheet` command line. See [../architecture/automation.md](../architecture/automation.md) |263| FeatureAutomation | `ModelTests.swift`, `EditingTests.swift` (`EditingTests`, `ScheduleTests`), `LocalizationIntentTests.swift` (`LocalizationTests`, `IntentServiceTests`) | the panel's view model, workflow editing, the LaunchAgent property list of a schedule, localization completeness, the service behind the App Intents |264265Source: `ls` and `grep -n '@Suite'` over the three `Tests` directories.266267## 8. Recorded test run268269Copied from `docs/reports/tests.json` (run of 2026-11-21T08:47:01Z by `Scripts/test-all.sh`) and270`docs/reports/app-tests.json` (2026-11-21T08:48:35Z).271272| Package | Test functions | Status | Seconds | Test targets |273|---|---:|---|---:|---|274| SheetCore | 32 | passed | 2 | `SheetCoreTests` |275| SheetFormat | 28 | passed | 4 | `SheetFormatTests` |276| FunctionsNumeric | 31 | passed | 7 | `FunctionsNumericTests` |277| FunctionsText | 19 | passed | 5 | `FunctionsTextTests` |278| FunctionsLookup | 49 | passed | 8 | `FunctionsLookupTests` |279| SheetEngine | 4 | passed | 5 | `SheetEngineTests` |280| SheetIO | 66 | passed | 4 | `SheetIOTests` |281| SheetCharts | 43 | passed | 10 | `SheetChartsTests` |282| SheetData | 64 | passed | 4 | `SheetDataTests` |283| SheetAnalytics | 310 | passed | 9 | `StatisticsTests`, `FinanceModelsTests`, `WhatIfTests`, `AuditTests` |284| SheetPython | 57 | passed | 12 | `SheetPythonTests` |285| SheetIntelligence | 62 | passed | 3 | `SheetIntelligenceTests` |286| SheetAutomation | 78 | passed | 4 | `SheetAutomationTests` |287| SheetTemplates | 29 | passed | 9 | `SheetTemplatesTests` |288| SheetUI | 20 | passed | 6 | `SheetUITests` |289| FeatureAudit | 60 | passed | 11 | `FeatureAuditTests` |290| FeatureCharts | 51 | passed | 10 | `FeatureChartsTests` |291| FeatureData | 57 | passed | 6 | `FeatureDataTests` |292| FeatureAnalysis | 125 | passed | 8 | `AnalysisKitTests`, `StatsBasicTests`, `StatsModelsTests`, `FinanceStudioTests`, `WhatIfToolsTests` |293| FeaturePython | 40 | passed | 16 | `FeaturePythonTests` |294| FeatureIntelligence | 17 | passed | 10 | `FeatureIntelligenceTests` |295| FeatureAutomation | 28 | passed | 6 | `FeatureAutomationTests` |296| **Total, 22 packages** | **1,270** | 0 failed packages | | |297| App (XCTest hosted in the app) | 9 | 0 failures | | `ORIXSheetTests` |298299The counts are **test functions**, as reported by the test runner (`Test run with N tests`). Many of300them are parameterised over fixtures: a function that replays a fixture file with thousands of301reference cases counts once. The number of reference cases behind the function libraries is in302[../mathematics/reference-tests.md](../mathematics/reference-tests.md). A test disabled by a missing303tool or an unset environment variable (section 5) is still counted by the runner as a test of the run;304`tests.json` does not record how many were skipped.305306## 9. Writing a new test307308- Put pure logic in the library target and test it without AppKit. A view-model test that needs the main309  actor is marked `@MainActor`; suites that share global state (interpreters, the application-support310  directory, the accessibility API) are marked `.serialized`.311- A test that needs an external tool must be **disabled with a comment** when the tool is missing312  (`.enabled(if:)`), never silently pass by returning early without saying so. `RealModelTests` uses313  the `.enabled` form for its opt-in tests; `LibreOfficeCrossCheck` returns early instead, and when314  touching it, prefer the `.enabled` form so that the skip is visible in the report.315- A test that takes more than a few seconds belongs behind an environment variable (section 5) so that316  `Scripts/test-all.sh` stays usable as a pre-commit check.317- Performance numbers are printed (`[bench]`, `BENCHMARK`, `[timing]` prefixes), not asserted, except for318  the loose bounds mentioned in section 6.319- Never write expected values by running the code under test and pasting its output.320321
Files main
Menu

Theme