Repository Public License
1<!-- ORIX Clean / File: docs/architecture/storage-accounting.md / Author: Simon-Pierre Boucher / Contact: contact@spboucher.ai -->2# Storage accounting34How ORIX Clean counts bytes, why it keeps three sizes per file, and why the space a cleanup is expected to5return is given as a range. Sources: `StorageAccounting/Support.swift`, `Volumes.swift`, `Queries.swift`,6`Treemap.swift`, and the size fields filled by the scanner.78## Three sizes910| Size | Field | Kernel source | Question it answers |11|---|---|---|---|12| Logical ("content size") | `logicalSize` | `ATTR_FILE_TOTALSIZE` | How many bytes of content does the file hold, all forks included? |13| Allocated ("size on disk") | `allocatedSize` | `ATTR_FILE_ALLOCSIZE` | How many bytes of blocks are attributed to this file? |14| Private | `privateSize` | `ATTR_CMNEXT_PRIVATESIZE` | How many bytes would the volume get back if this file alone were removed? -1 when the filesystem does not say |1516For a single file measured outside a scan, `Measure.item` uses `lstat` (`st_size`, `st_blocks * 512`) and17`getattrlist` with `ATTR_CMNEXT_PRIVATESIZE` and `FSOPT_NOFOLLOW`. For a directory it runs a two-thread scan18and reads the root totals; `incomplete` is set when that scan reported any issue, in which case the sizes are a19lower bound.2021The interface lets the user choose which size drives the map and lists (`SizeKind.allocated`, the default, or22`.logical`); it shows both in detail views. Sizes are formatted with decimal units, as Finder does23(`ByteFormat.string`, `.byteCount(style: .file)`).2425## Why the three differ2627| Case | Logical vs allocated | Private | How the code represents it |28|---|---|---|---|29| Ordinary file | Allocated is logical rounded up to the block size | Equal to allocated | No flag |30| **APFS clone** (copy made by Finder, `cp -c`, `clonefile`) | Each clone reports its full allocated size | Close to zero while blocks are shared; grows as either copy is modified | `mayShareBlocks` from `EF_MAY_SHARE_BLOCKS`; clone id kept in `tree.cloneIDs` |31| **Hard link** (several names, one inode) | Every name reports the same sizes | Not meaningful per name | `hardLinked` on all names; `hardLinkDuplicate` on every name after the first seen, which then adds nothing to totals |32| **Sparse file** | Allocated is far below logical | Follows allocated | `sparse` from `EF_IS_SPARSE` |33| **Compressed file** (transparent compression) | Allocated can be far below logical | Follows allocated | `compressed` from `UF_COMPRESSED` |34| **Dataless file** (evicted iCloud or File Provider item) | Logical is the full size, allocated near zero | Near zero | `dataless` from `SF_DATALESS`; never opened for reading |35| File also referenced by a **local snapshot** | Unchanged | Blocks pinned by a snapshot are not private | No dedicated flag; the file may show `mayShareBlocks` |3637Consequences enforced in code:3839- Directory totals sum logical, allocated and positive private sizes of descendants, skipping hard-link40 duplicates, so a folder with three names of one 300 kB file reports 300 kB (test `hardLinksAreCountedOnce`).41- A directory's private total is a lower bound: blocks shared only between two files of the same folder are42 private to the folder but to neither file.43- The treemap skips hard-link duplicates and zero-size children, so areas add up to the parent's total.44- Category totals (`TreeQuery.categories`) skip hard-link duplicates; files inside a package are attributed45 to the package's category (an `.app` to Applications).4647## Recoverable space is a range4849```mermaid50flowchart LR51 Files[Files selected for removal] --> Min[Sum of private sizes]52 Files --> Max[Sum of allocated sizes]53 Min --> Est[RecoverableEstimate minimum to maximum]54 Max --> Est55 Est --> Exact{minimum equals maximum}56 Exact -->|yes| One[shown as one figure]57 Exact -->|no| Range[shown as A to B]58 Trash[Moved to the Trash] --> Later[nothing is freed until the Trash is emptied]59 Snap[Local snapshots] --> Later2[freed blocks may stay pinned until the snapshot expires]60```6162`RecoverableEstimate` has a `minimum` and a `maximum`:6364- **Maximum** is the sum of allocated sizes. It is reached only if nothing in the selection shares blocks with65 a file that stays.66- **Minimum** is the sum of private sizes: what the filesystem attributes to these files alone. When private67 size is unknown (-1), the allocated size is used for both ends and the estimate is exact but possibly68 overstated; reports say so in their limitations.69- In a duplicate group, a member that has other hard-linked names contributes zero to both ends70 (`DuplicateGroup.recoverable(removing:)`): removing one name frees nothing.7172A single figure cannot be correct in general. If two clones are both selected, the truth is near the maximum;73if only one is, it is near the minimum; private sizes are per file and do not tell which blocks are shared74with which other file. The range states what is known. The estimate is also only about the files: the volume75gets the space back when the Trash is emptied, and blocks still referenced by a local snapshot stay in use76until the snapshot is deleted by the system. The cleanup results show free space reported by macOS before and77after, without claiming the difference is due to the cleanup.7879## Volumes, free space and purgeable space8081`Volumes.mounted()` lists mounted volumes (hidden ones skipped) with `URLResourceValues` and `statfs`.8283| Field | Source | Meaning |84|---|---|---|85| `total` | `.volumeTotalCapacityKey` | Capacity |86| `available` | `.volumeAvailableCapacityKey` | Free blocks reported right now |87| `availableForImportantUsage` | `.volumeAvailableCapacityForImportantUsageKey`, never below `available` | What the system says it could make available |88| `purgeable` | difference of the two above | Space the system could reclaim on demand |89| `format` | `statfs.f_fstypename` | Filesystem type |90| `isReadOnly` | resource value or `MNT_RDONLY`; forced false for `/` | The startup volume is a sealed snapshot, but the Data volume behind it is writable |9192Purgeable space is reported, not managed: it is whatever macOS considers reclaimable (caches it controls,93local snapshots). The application cannot list it, attribute it to files or purge it. Used space is94`total - available`, so on APFS it includes space used by other volumes of the same container; the figure is95the system's, shown as such.9697Snapshots are not enumerated (see `docs/research/macos-storage-capability-matrix.md`). Their effect appears98only indirectly, in purgeable space and in the limitation text of reports.99100## Queries on the tree101102| Function | Behaviour |103|---|---|104| `TreeQuery.largestFiles` | One pass over files with a fixed-capacity min-heap (default 500). Filters: size bounds, modified before or after, categories, extensions, case-insensitive name substring, subtree. Files inside packages are excluded unless asked |105| `TreeQuery.largestFolders` | Depth-first walk; packages count as one item and are not descended into |106| `TreeQuery.categories` | Totals per `FileCategory` (video, audio, image, archive, disk image, installer, document, code, application, other) from a fixed extension table |107| `Treemap.layout` | Squarified layout (Bruls, Huizing, van Wijk) with level of detail |108109### Treemap level of detail110111The map is drawn with SwiftUI `Canvas`, not Metal. `Treemap.layout` produces only what can be seen:112113- a folder is opened only if its rectangle is at least 26 points on each side and depth is under 7;114- packages are leaves unless the user enables package contents (the layout root is always opened);115- children whose area would be under 14 square points are merged into one "smaller items" cell that carries116 the count and total size of what it stands for;117- with a `visible` rectangle, subtrees that do not intersect the viewport are skipped (zoom and pan).118119Measured on a 467,907-node tree at 1600 by 1000: 1,798 cells in 0.162 ms on average. With a layout this cheap120and a cell count this small, a custom Metal renderer was not justified.121122## Invariants123124- No size is estimated from a name or an extension; categories are the only thing derived from extensions.125- `RecoverableEstimate.minimum <= maximum` (the initializer clamps).126- Totals shown for a folder equal the sum over its regular files (checked under concurrent modification by the127 test `filesChangingAndDisappearingDuringScan`).128- Allocated total of a scan matched `du -sk` exactly on the test tree and on a 31k-object tree.129130## Failure handling131132- Unreadable folders have unknown size: they contribute nothing and are listed as issues.133- A filesystem that rejects the extended attributes yields `sharingMeasured == false`; private sizes are then134 treated as unknown everywhere.135- Volumes with zero capacity or unreadable resource values are omitted.136137## Not handled138139- Which file shares blocks with which: only "may share" and a clone id per file are known.140- Space used by snapshots, by other volumes of the same APFS container, or by the system's own purgeable data.141- Per-file accounting on non-APFS filesystems beyond logical and allocated size.142- Directory entries' own blocks.143