Repository Public License
1<!--2ORIX Secure3File: docs/architecture/storage.md4Description: Storage overview, owned-storage cleanup, photo cleanup pipeline, file cleanup and the file inspector.5Author: Simon-Pierre Boucher6Contact: contact@spboucher.ai7-->89# ORIX Secure — Storage, cleanup and file inspection1011Sources: `ORIXSecureEngines/StorageAnalysisEngine.swift`, `ORIXSecureEngines/SystemDeviceProbe.swift`12(`volumeCapacity`), package `ORIXMediaCleanup`, package `ORIXFileInspection`; app screens under13`App/Sources/Features/Storage`, `PhotoCleanup` and `FileScanner`.1415## 1. Storage overview1617`SystemDeviceProbe.volumeCapacity(at:)` reads four `URLResourceKey` values of the home directory into a18`VolumeCapacity` of `Reading<Int64>`:1920| Field | Key | Rule |21|---|---|---|22| `total` | `volumeTotalCapacityKey` | |23| `available` | `volumeAvailableCapacityKey` | Free space as the file system reports it. |24| `availableForImportantUsage` | `volumeAvailableCapacityForImportantUsageKey` | A value of 0 is treated as not reported. |25| `availableForOpportunisticUsage` | `volumeAvailableCapacityForOpportunisticUsageKey` | A value of 0 is treated as not reported. |2627`bestAvailable` is the important-usage figure when measured, else `available`; `usedFraction` is28`(total − bestAvailable) / total`, clamped to 0…1. Privacy manifest: `DiskSpace`, reasons `85F4.1` and `E174.1`.2930`StorageAnalysisEngine.capacityFinding` produces one finding, `storage.capacity`:3132| Condition (`fraction = free / total`, `free = bestAvailable`) | State | Severity |33|---|---|---|34| `total` or `bestAvailable` not measured | `notAvailable` | — |35| `fraction < criticalFraction` (0.05) or `free < criticalBytes` (2 000 000 000) | `attention` — "Storage is almost full" | `moderate` |36| `fraction < lowFraction` (0.10) or `free < lowBytes` (5 000 000 000) | `attention` — "Storage is getting low" | `low` |37| otherwise | `verified` | `none` |3839The thresholds are the app's own rule, quoted in the finding. `StorageOverview.notExposed` carries the40`storage.categories` platform limit: iOS gives no per-app breakdown, and none is estimated.4142## 2. Owned storage and its cleanup4344`OwnedLocations` (supplied by `AppFolders.owned`) maps four `OwnedStorageEntry.Kind`s to folders.45`StorageAnalysisEngine.measure(_:)` walks a folder and sums `totalFileAllocatedSize` (falling back to46`fileSize`) of regular files; a missing path measures as zero.4748| Kind | Folder | `isSafeToClear` |49|---|---|---|50| `exports` | `tmp/ORIXSecureExports` | yes |51| `caches` | `Caches/ORIXSecure` | yes |52| `temporary` | `tmp/ORIXSecureWork` | yes |53| `database` | `Application Support/ORIXSecure/Database` | no |5455The `database` entry (reports, answers, workflows) is measured and displayed only. `clear(_ kinds:)`:56571. accepts only kinds marked safe to clear; `.database` is always refused and reported in `errors` (there is58 no parameter to override this);592. measures the folder (`before`), removes each direct child, measures again (`after`);603. reports `freedBytes = Σ max(0, before − after)` and `filesRemoved` likewise, plus per-item errors.6162The freed amount is a measured difference. The app only calls `clear([.caches, .temporary, .exports])`63(Storage screen after a confirmation, Settings, "Erase All App Data"). Reports, answers and workflows are64deleted only through `LocalStore` (`deleteReport`, `deleteAllReports`, `deleteEverything`).6566## 3. Photo cleanup pipeline6768```mermaid69flowchart TD70 A["Access<br/>PhotoLibraryAccess.current() / requestAccess()<br/>PHPhotoLibrary, .readWrite"] -->|"limited or authorized"| B71 A -->|"denied, restricted, not determined"| X["MediaLibraryError.accessDenied<br/>pipeline never prompts by itself"]72 B["Incremental scan — PhotoLibraryScanner actor<br/>PHAsset.fetchAssets, batches of 200"] --> C{"FingerprintCache hit?<br/>key: localIdentifier + modificationDate"}73 C -->|yes| D["reuse ImageFingerprint"]74 C -->|no| E["128 px thumbnail, network disabled<br/>ImageFingerprinter.fingerprint"]75 E --> D76 D --> F["CleanupAnalyzer.analyze — first pass<br/>no sizes, no content hashes"]77 F --> G["measurementCandidates<br/>group members + 30 longest videos"]78 G --> H["PhotoSizeMeasurer.measure<br/>public API only, never downloads from iCloud"]79 H --> I["CleanupAnalyzer.analyze — second pass<br/>with byteSize and SHA-256"]80 I --> J["User selection — CleanupSelection<br/>preselected: extra copies of exact duplicates only"]81 J --> K["CleanupPlan<br/>measured bytes, unknown sizes, warnings"]82 K --> L["PHAssetChangeRequest.deleteAssets<br/>system confirmation dialog"]83 L --> M["Verification: fetch the identifiers again"]84 M --> N["CleanupResult<br/>deleted = no longer fetched;<br/>bytesFreedImmediately = 0"]85 N --> O["Re-scan from cache when something was deleted"]86```8788`CleanupPipeline.run` implements scan → analyse → measure → analyse and `CleanupExecutor.execute` the89deletion and verification. The engine works against the `MediaLibrary` protocol (in-memory library in tests).9091### 3.1 Access9293Only read-write authorization is used (`PHPhotoLibrary.authorizationStatus(for: .readWrite)`); add-only is94never requested. With `limited` access every result carries `coverageNote`: results cover the selected photos95only. The prompt appears when the user taps "Choose Photo Access", or when a workflow runs a `photos.scan`96step while access is `notDetermined`.9798### 3.2 Scan and cache99100`PhotoLibraryScanner.Options` defaults: `batchSize` 200, `thumbnailPixelSize` 128,101`concurrentThumbnailRequests` 4, `includeAllBurstAssets` true, `includeHiddenAssets` false. Images and videos102are fetched sorted by creation date. Thumbnails are requested with `isNetworkAccessAllowed = false`, first in103`.fastFormat`, then `.highQualityFormat` if no image came back; an asset with no local thumbnail counts as a104fingerprint failure. The image is fingerprinted inside the callback and not retained.105106`FingerprintCache` is one binary file, `orix-media-fingerprints.cache`, in `Caches/ORIXSecure`: magic107`OXFP`, version (`currentVersion` = 1), count, records (identifier, modification stamp, dHash, pHash, mean108luma, 48 colour bytes), FNV-1a 64 checksum. A file that is unreadable, corrupt or of another version is109discarded and the reason reported in `ScanStatistics.cacheDiscardReason`. An entry is a hit only when the110asset's `modificationDate` matches. After a complete scan the cache is pruned to the identifiers still111present and saved atomically. It stores hashes, never pixels.112113### 3.3 Fingerprint algorithms114115`ImageFingerprinter.fingerprint(of:)` draws any image into a 64×64 sRGB bitmap, then derives:116117| Component | Computation |118|---|---|119| `dHash` (64 bits) | 9×8 grayscale reduction; one bit per horizontal neighbour comparison. |120| `pHash` (64 bits) | 32×32 grayscale reduction; separable DCT-II limited to the 8×8 lowest frequencies; bit set where the coefficient exceeds the median of the 63 non-DC coefficients; bit 0 always 0. |121| `meanLuma` | Mean of the 32×32 grayscale reduction. |122| `colorSignature` (48 bytes) | Mean R, G, B of each cell of a 4×4 grid. |123124`colorDistance` is the mean absolute difference of the signatures after removing the overall brightness offset.125126### 3.4 Analysis and thresholds127128`CleanupAnalyzer.analyze` is deterministic (canonical order: capture date, then identifier) and cancellable.129Steps, in order: exact duplicates (identical `contentHash`), camera bursts (`burstIdentifier`), probable130duplicates (same kind, dimensions, capture second and identical fingerprints, at least one member unhashed),131visual similarity, plain lists (screenshots, videos, Live Photos).132133Two fingerprints are similar (`areSimilar`) when all four limits hold. `CleanupConfiguration` defaults:134135| Property | Default | Meaning |136|---|---|---|137| `dHashThreshold` | 8 | Max dHash Hamming distance. |138| `pHashThreshold` | 10 | Max pHash Hamming distance. |139| `colorThreshold` | 8 | Max brightness-compensated colour distance (0…255). |140| `lumaThreshold` | 48 | Max mean-luminance difference. |141| `similarTimeWindow` | 180 s | Consecutive photos at most this far apart form a session. |142| `similarLookback` | 500 | A group stops accepting members after this many later photos. |143| `includeScreenshotsInSimilarity` | false | Screenshots are listed, not hashed for similarity. |144| `globalPassEnabled` | false | Optional cross-session pass with `globalDHashThreshold` 2 and `globalPHashThreshold` 4. |145| `burstLikeInterval` / `burstLikeMinimumCount` | 2 s / 3 | A similar group whose consecutive photos are within 2 s and has ≥ 3 members is "burst-like". |146| `largeVideoMinimumBytes` | 100 000 000 | Measured size for "large video". |147| `longVideoMinimumDuration` | 60 s | "Long video". |148| `oldScreenshotAgeDays` | nil | Optional "old screenshots" category. |149150The source comment attributes these defaults to a synthetic benchmark in the test suite (60 generated scenes).151Clustering is greedy complete linkage: a photo joins a group only if it matches every member, so a slowly152drifting sequence cannot chain.153154Categories (`CleanupCategoryKind`) and their confidence: `exactDuplicates`, `bursts`, `screenshots`,155`oldScreenshots`, `largeVideos`, `longVideos`, `livePhotos` are `measured`; `probableDuplicates`, `similar`,156`burstLike` are `heuristic`. Live Photos are informational and propose nothing to delete.157158### 3.5 Keeper rules159160`CleanupAnalyzer.chooseKeeper(among:)` narrows the contenders rule by rule and reports the `KeeperRule` that161decided: `favorite` → `edited` → `highestResolution` → `largestMeasuredSize` (applied only when every162remaining contender has a measured size) → `earliest` → `identifierOrder` (stable tie-break). Only exact163duplicates are preselected (`preselectedIDs` = every copy but the keeper); heuristic groups preselect nothing.164165### 3.6 Size measurement — public API only166167`PhotoSizeMeasurer` (`maxConcurrent` 2, clamped 1…3; `videoHashByteLimit` 250 000 000):168169- Photos and other resources: every `PHAssetResource` is streamed with170 `PHAssetResourceManager.requestData`, `isNetworkAccessAllowed = false`; bytes are counted and the primary171 resource is hashed with SHA-256 in the same pass.172- Videos: `PHImageManager.requestAVAsset` with network disabled; when the result is an `AVURLAsset`, the size173 is the file's `URLResourceKey.fileSizeKey`. Videos up to the hash limit are additionally streamed once for174 their hash. Non-file-backed videos fall back to the resource stream.175- `PHPhotosError.networkAccessRequired` or `PHImageResultIsInCloudKey` gives `MediaMeasurement.inCloudOnly`:176 size unavailable, `isLocallyAvailable = measured(false)`. Nothing is downloaded.177178**Rule: the undocumented `fileSize` key of `PHAssetResource` (key-value coding) is not used.** The package179contains no `value(forKey:)` call; the only `fileSize` in it is `URLResourceValues.fileSize` of a video file180URL. A size is therefore either counted byte by byte or read from a documented URL resource value, or it is181`unavailable`.182183Totals keep measured and unmeasured apart: `CleanupGroup.recoverable` and `CleanupTotals.recoverable` are184`unavailable` as soon as one candidate's size is unknown, while `measuredPortion` / `measuredBytes` hold the185partial sum. Unmeasured videos are ranked by `VideoSizeEstimate` (duration × pixel count; boundaries at 60 M,186250 M and 1 000 M pixel-seconds) in a separate `largeVideos:estimated` group of `heuristic` confidence that187shows a size class, never a byte value.188189### 3.7 Plan, deletion, Recently Deleted190191`CleanupPlan` lists the selected identifiers known to the analysis, `unknownIDs` (never sent), measured bytes,192`unknownSizeIDs`, and `warnings`: groups in which every member is selected, selected favourites, unknown193identifiers. `CleanupExecutor.execute`:1941951. returns `nothingRequested` / `noAccess` without calling the library when applicable;1962. drops identifiers no longer in the library (`CleanupSkip.notInLibrary`);1973. sends one `PHAssetChangeRequest.deleteAssets` for exactly the remaining identifiers — iOS shows its own198 confirmation; declining maps to `.cancelledByUser`;1994. fetches the identifiers again; only those no longer returned count as `deleted`, whatever the library200 reported. Outcomes: `completed`, `partial`, `cancelledByUser`, `failed`.201202Recently Deleted semantics are encoded in the types: `CleanupResult.bytesFreedImmediately` is the constant 0;203the measured size of deleted items is named `movedToRecentlyDeleted`; `CleanupStatements.recentlyDeletedNote`204and `freedNow` state that items stay 30 days in Recently Deleted and that storage is reclaimed only after205that or when the album is emptied. The plan sheet shows the note before confirmation and the cleanup report206never states an amount of freed space. The app cannot empty Recently Deleted.207208## 4. File cleanup (user-chosen folder)209210`FileCleanupView` opens a folder with `fileImporter` (security-scoped URL) and runs, in the package:211212- `LargeFileScanner.scan` — 50 largest files (`resultCount`), totals per extension, up to 500 000 files;213 ranks by allocated size when the system reports it.214- `DuplicateFinder.findDuplicates(inFolder:)` — groups by size, then SHA-256 of the first 64 KiB215 (`prefixLength`), then full SHA-256. Defaults: `minimumFileSize` 1, `maxFiles` 200 000, hidden files and216 packages skipped; hard links to an already counted file are ignored; symbolic links are not followed.217 It states facts and removes nothing. Its `notInspected` notes that APFS clones may free less than announced.218- `FileRemover.remove(_:)` — after a confirmation dialog, removes exactly the listed URLs one by one,219 permanently (no trash), refuses folders unless `allowDirectories`, removes a symbolic link rather than its220 target, verifies each item is gone and reports `bytesFreed` per item. The view saves a `.cleanup` report.221222## 5. File inspector223224`FileInspector.inspect(fileAt:)` reads one user-selected file through bounded `ByteSource` reads (no single225read above 8 MiB) and never executes, renders or decompresses it.226227- **Type from content**: `MagicDetector` matches a signature table on the first 64 KiB, then probes RIFF/ISO228 media brands, PE, Mach-O vs. Java class, DMG trailer and text. ZIP containers are refined by entry names229 (`refineZip`). `FileNameAnalyzer` adds name checks (double extension, bidi/invisible characters, look-alike230 extension, padding) and the extension-versus-content comparison (`file.type.extensionMismatch`,231 `file.type.executableDisguised`).232- **Hash**: streamed SHA-256 (`FileHasher`, 1 MiB chunks, cancellable).233- **ZIP without extraction**: `ZipInspector` searches the end-of-central-directory record backwards in the234 last 22 + 65 535 bytes, handles ZIP64, then walks the central directory only. It records declared sizes,235 methods, encryption, traversal paths, symbolic links, duplicate names, nested archives, executables,236 overlapping entries, prepended bytes and OOXML macro parts. Local headers are not compared with the central237 directory (stated in `notInspected`). TAR headers and the GZIP header/trailer get an equivalent bounded walk.238- **Bomb limits** (`InspectionOptions`): `maxArchiveEntries` 100 000 (walk stops, `file.zip.tooManyEntries`),239 `maxReportedArchiveEntries` 500, `maxOverallCompressionRatio` 100 (`file.zip.ratio.overall`),240 `maxEntryCompressionRatio` 1000 (`file.zip.ratio.entry`), `maxDeclaredUncompressedBytes` 10 GiB241 (`file.zip.declaredSize`), `maxImagePixels` 268 435 456 (`file.image.dimensions`). Ratios come from declared242 sizes; nothing is inflated to check them.243- **PDF raw scan**: `PDFInspector.scan` counts the name tokens `JavaScript`, `JS`, `OpenAction`, `AA`,244 `Launch`, `EmbeddedFile`, `RichMedia`, `XFA`, `AcroForm`, `Encrypt`, `ObjStm` (including `#xx`-escaped245 spellings) in at most `rawScanWindowBytes` (8 MiB) at each end of the file, checks the `%PDF-` header offset246 and the `%%EOF` marker and trailing bytes. Where PDFKit exists and the file is ≤ 256 MiB it adds page count,247 encryption/lock state, attributes and up to 200 link addresses from at most 500 pages. Token presence is248 reported as presence; code is not analysed.249250`FileInspector.standingLimits` is appended to every inspection: no malware signature database, nothing251executed or rendered, macros by presence only, encrypted content unreadable, archives by directory only.252253## 6. Verification status254255- Package tests on macOS cover fingerprints, cache, analyzer, plan/executor with an in-memory library, and256 the file parsers including fuzz tests. They were not re-run while writing this document.257- The Photos adaptors (`PhotoLibraryScanner`, `PhotoSizeMeasurer`, `PhotoDeleter`) were run in the iOS258 Simulator only (`Scripts/seed-photos.sh` seeds a library). Not verified on a physical device: iCloud-only259 originals, limited access, large libraries, real bursts and Live Photos, the system deletion dialog, and260 measurement timing. Volume readings in the Simulator describe the host Mac's volume.261