SPB Git

Repository Public License

storage.md 16.6 KB

# ORIX Secure — Storage, cleanup and file inspection

Sources: ORIXSecureEngines/StorageAnalysisEngine.swift, ORIXSecureEngines/SystemDeviceProbe.swift (volumeCapacity), package ORIXMediaCleanup, package ORIXFileInspection; app screens under App/Sources/Features/Storage, PhotoCleanup and FileScanner.

# 1. Storage overview

SystemDeviceProbe.volumeCapacity(at:) reads four URLResourceKey values of the home directory into a VolumeCapacity of Reading<Int64>:

Field Key Rule
total volumeTotalCapacityKey
available volumeAvailableCapacityKey Free space as the file system reports it.
availableForImportantUsage volumeAvailableCapacityForImportantUsageKey A value of 0 is treated as not reported.
availableForOpportunisticUsage volumeAvailableCapacityForOpportunisticUsageKey A value of 0 is treated as not reported.

bestAvailable is the important-usage figure when measured, else available; usedFraction is (total − bestAvailable) / total, clamped to 0…1. Privacy manifest: DiskSpace, reasons 85F4.1 and E174.1.

StorageAnalysisEngine.capacityFinding produces one finding, storage.capacity:

Condition (fraction = free / total, free = bestAvailable) State Severity
total or bestAvailable not measured notAvailable —
fraction < criticalFraction (0.05) or free < criticalBytes (2 000 000 000) attention — "Storage is almost full" moderate
fraction < lowFraction (0.10) or free < lowBytes (5 000 000 000) attention — "Storage is getting low" low
otherwise verified none

The thresholds are the app's own rule, quoted in the finding. StorageOverview.notExposed carries the storage.categories platform limit: iOS gives no per-app breakdown, and none is estimated.

# 2. Owned storage and its cleanup

OwnedLocations (supplied by AppFolders.owned) maps four OwnedStorageEntry.Kinds to folders. StorageAnalysisEngine.measure(_:) walks a folder and sums totalFileAllocatedSize (falling back to fileSize) of regular files; a missing path measures as zero.

Kind Folder isSafeToClear
exports tmp/ORIXSecureExports yes
caches Caches/ORIXSecure yes
temporary tmp/ORIXSecureWork yes
database Application Support/ORIXSecure/Database no

The database entry (reports, answers, workflows) is measured and displayed only. clear(_ kinds:):

  1. accepts only kinds marked safe to clear; .database is always refused and reported in errors (there is no parameter to override this);
  2. measures the folder (before), removes each direct child, measures again (after);
  3. reports freedBytes = Σ max(0, before − after) and filesRemoved likewise, plus per-item errors.

The freed amount is a measured difference. The app only calls clear([.caches, .temporary, .exports]) (Storage screen after a confirmation, Settings, "Erase All App Data"). Reports, answers and workflows are deleted only through LocalStore (deleteReport, deleteAllReports, deleteEverything).

# 3. Photo cleanup pipeline

flowchart TD
    A["Access<br/>PhotoLibraryAccess.current() / requestAccess()<br/>PHPhotoLibrary, .readWrite"] -->|"limited or authorized"| B
    A -->|"denied, restricted, not determined"| X["MediaLibraryError.accessDenied<br/>pipeline never prompts by itself"]
    B["Incremental scan — PhotoLibraryScanner actor<br/>PHAsset.fetchAssets, batches of 200"] --> C{"FingerprintCache hit?<br/>key: localIdentifier + modificationDate"}
    C -->|yes| D["reuse ImageFingerprint"]
    C -->|no| E["128 px thumbnail, network disabled<br/>ImageFingerprinter.fingerprint"]
    E --> D
    D --> F["CleanupAnalyzer.analyze — first pass<br/>no sizes, no content hashes"]
    F --> G["measurementCandidates<br/>group members + 30 longest videos"]
    G --> H["PhotoSizeMeasurer.measure<br/>public API only, never downloads from iCloud"]
    H --> I["CleanupAnalyzer.analyze — second pass<br/>with byteSize and SHA-256"]
    I --> J["User selection — CleanupSelection<br/>preselected: extra copies of exact duplicates only"]
    J --> K["CleanupPlan<br/>measured bytes, unknown sizes, warnings"]
    K --> L["PHAssetChangeRequest.deleteAssets<br/>system confirmation dialog"]
    L --> M["Verification: fetch the identifiers again"]
    M --> N["CleanupResult<br/>deleted = no longer fetched;<br/>bytesFreedImmediately = 0"]
    N --> O["Re-scan from cache when something was deleted"]

CleanupPipeline.run implements scan → analyse → measure → analyse and CleanupExecutor.execute the deletion and verification. The engine works against the MediaLibrary protocol (in-memory library in tests).

# 3.1 Access

Only read-write authorization is used (PHPhotoLibrary.authorizationStatus(for: .readWrite)); add-only is never requested. With limited access every result carries coverageNote: results cover the selected photos only. The prompt appears when the user taps "Choose Photo Access", or when a workflow runs a photos.scan step while access is notDetermined.

# 3.2 Scan and cache

PhotoLibraryScanner.Options defaults: batchSize 200, thumbnailPixelSize 128, concurrentThumbnailRequests 4, includeAllBurstAssets true, includeHiddenAssets false. Images and videos are fetched sorted by creation date. Thumbnails are requested with isNetworkAccessAllowed = false, first in .fastFormat, then .highQualityFormat if no image came back; an asset with no local thumbnail counts as a fingerprint failure. The image is fingerprinted inside the callback and not retained.

FingerprintCache is one binary file, orix-media-fingerprints.cache, in Caches/ORIXSecure: magic OXFP, version (currentVersion = 1), count, records (identifier, modification stamp, dHash, pHash, mean luma, 48 colour bytes), FNV-1a 64 checksum. A file that is unreadable, corrupt or of another version is discarded and the reason reported in ScanStatistics.cacheDiscardReason. An entry is a hit only when the asset's modificationDate matches. After a complete scan the cache is pruned to the identifiers still present and saved atomically. It stores hashes, never pixels.

# 3.3 Fingerprint algorithms

ImageFingerprinter.fingerprint(of:) draws any image into a 64×64 sRGB bitmap, then derives:

Component Computation
dHash (64 bits) 9×8 grayscale reduction; one bit per horizontal neighbour comparison.
pHash (64 bits) 32×32 grayscale reduction; separable DCT-II limited to the 8×8 lowest frequencies; bit set where the coefficient exceeds the median of the 63 non-DC coefficients; bit 0 always 0.
meanLuma Mean of the 32×32 grayscale reduction.
colorSignature (48 bytes) Mean R, G, B of each cell of a 4×4 grid.

colorDistance is the mean absolute difference of the signatures after removing the overall brightness offset.

# 3.4 Analysis and thresholds

CleanupAnalyzer.analyze is deterministic (canonical order: capture date, then identifier) and cancellable. Steps, in order: exact duplicates (identical contentHash), camera bursts (burstIdentifier), probable duplicates (same kind, dimensions, capture second and identical fingerprints, at least one member unhashed), visual similarity, plain lists (screenshots, videos, Live Photos).

Two fingerprints are similar (areSimilar) when all four limits hold. CleanupConfiguration defaults:

Property Default Meaning
dHashThreshold 8 Max dHash Hamming distance.
pHashThreshold 10 Max pHash Hamming distance.
colorThreshold 8 Max brightness-compensated colour distance (0…255).
lumaThreshold 48 Max mean-luminance difference.
similarTimeWindow 180 s Consecutive photos at most this far apart form a session.
similarLookback 500 A group stops accepting members after this many later photos.
includeScreenshotsInSimilarity false Screenshots are listed, not hashed for similarity.
globalPassEnabled false Optional cross-session pass with globalDHashThreshold 2 and globalPHashThreshold 4.
burstLikeInterval / burstLikeMinimumCount 2 s / 3 A similar group whose consecutive photos are within 2 s and has ≥ 3 members is "burst-like".
largeVideoMinimumBytes 100 000 000 Measured size for "large video".
longVideoMinimumDuration 60 s "Long video".
oldScreenshotAgeDays nil Optional "old screenshots" category.

The source comment attributes these defaults to a synthetic benchmark in the test suite (60 generated scenes). Clustering is greedy complete linkage: a photo joins a group only if it matches every member, so a slowly drifting sequence cannot chain.

Categories (CleanupCategoryKind) and their confidence: exactDuplicates, bursts, screenshots, oldScreenshots, largeVideos, longVideos, livePhotos are measured; probableDuplicates, similar, burstLike are heuristic. Live Photos are informational and propose nothing to delete.

# 3.5 Keeper rules

CleanupAnalyzer.chooseKeeper(among:) narrows the contenders rule by rule and reports the KeeperRule that decided: favorite → edited → highestResolution → largestMeasuredSize (applied only when every remaining contender has a measured size) → earliest → identifierOrder (stable tie-break). Only exact duplicates are preselected (preselectedIDs = every copy but the keeper); heuristic groups preselect nothing.

# 3.6 Size measurement — public API only

PhotoSizeMeasurer (maxConcurrent 2, clamped 1…3; videoHashByteLimit 250 000 000):

  • Photos and other resources: every PHAssetResource is streamed with PHAssetResourceManager.requestData, isNetworkAccessAllowed = false; bytes are counted and the primary resource is hashed with SHA-256 in the same pass.
  • Videos: PHImageManager.requestAVAsset with network disabled; when the result is an AVURLAsset, the size is the file's URLResourceKey.fileSizeKey. Videos up to the hash limit are additionally streamed once for their hash. Non-file-backed videos fall back to the resource stream.
  • PHPhotosError.networkAccessRequired or PHImageResultIsInCloudKey gives MediaMeasurement.inCloudOnly: size unavailable, isLocallyAvailable = measured(false). Nothing is downloaded.

Rule: the undocumented fileSize key of PHAssetResource (key-value coding) is not used. The package contains no value(forKey:) call; the only fileSize in it is URLResourceValues.fileSize of a video file URL. A size is therefore either counted byte by byte or read from a documented URL resource value, or it is unavailable.

Totals keep measured and unmeasured apart: CleanupGroup.recoverable and CleanupTotals.recoverable are unavailable as soon as one candidate's size is unknown, while measuredPortion / measuredBytes hold the partial sum. Unmeasured videos are ranked by VideoSizeEstimate (duration × pixel count; boundaries at 60 M, 250 M and 1 000 M pixel-seconds) in a separate largeVideos:estimated group of heuristic confidence that shows a size class, never a byte value.

# 3.7 Plan, deletion, Recently Deleted

CleanupPlan lists the selected identifiers known to the analysis, unknownIDs (never sent), measured bytes, unknownSizeIDs, and warnings: groups in which every member is selected, selected favourites, unknown identifiers. CleanupExecutor.execute:

  1. returns nothingRequested / noAccess without calling the library when applicable;
  2. drops identifiers no longer in the library (CleanupSkip.notInLibrary);
  3. sends one PHAssetChangeRequest.deleteAssets for exactly the remaining identifiers — iOS shows its own confirmation; declining maps to .cancelledByUser;
  4. fetches the identifiers again; only those no longer returned count as deleted, whatever the library reported. Outcomes: completed, partial, cancelledByUser, failed.

Recently Deleted semantics are encoded in the types: CleanupResult.bytesFreedImmediately is the constant 0; the measured size of deleted items is named movedToRecentlyDeleted; CleanupStatements.recentlyDeletedNote and freedNow state that items stay 30 days in Recently Deleted and that storage is reclaimed only after that or when the album is emptied. The plan sheet shows the note before confirmation and the cleanup report never states an amount of freed space. The app cannot empty Recently Deleted.

# 4. File cleanup (user-chosen folder)

FileCleanupView opens a folder with fileImporter (security-scoped URL) and runs, in the package:

  • LargeFileScanner.scan — 50 largest files (resultCount), totals per extension, up to 500 000 files; ranks by allocated size when the system reports it.
  • DuplicateFinder.findDuplicates(inFolder:) — groups by size, then SHA-256 of the first 64 KiB (prefixLength), then full SHA-256. Defaults: minimumFileSize 1, maxFiles 200 000, hidden files and packages skipped; hard links to an already counted file are ignored; symbolic links are not followed. It states facts and removes nothing. Its notInspected notes that APFS clones may free less than announced.
  • FileRemover.remove(_:) — after a confirmation dialog, removes exactly the listed URLs one by one, permanently (no trash), refuses folders unless allowDirectories, removes a symbolic link rather than its target, verifies each item is gone and reports bytesFreed per item. The view saves a .cleanup report.

# 5. File inspector

FileInspector.inspect(fileAt:) reads one user-selected file through bounded ByteSource reads (no single read above 8 MiB) and never executes, renders or decompresses it.

  • Type from content: MagicDetector matches a signature table on the first 64 KiB, then probes RIFF/ISO media brands, PE, Mach-O vs. Java class, DMG trailer and text. ZIP containers are refined by entry names (refineZip). FileNameAnalyzer adds name checks (double extension, bidi/invisible characters, look-alike extension, padding) and the extension-versus-content comparison (file.type.extensionMismatch, file.type.executableDisguised).
  • Hash: streamed SHA-256 (FileHasher, 1 MiB chunks, cancellable).
  • ZIP without extraction: ZipInspector searches the end-of-central-directory record backwards in the last 22 + 65 535 bytes, handles ZIP64, then walks the central directory only. It records declared sizes, methods, encryption, traversal paths, symbolic links, duplicate names, nested archives, executables, overlapping entries, prepended bytes and OOXML macro parts. Local headers are not compared with the central directory (stated in notInspected). TAR headers and the GZIP header/trailer get an equivalent bounded walk.
  • Bomb limits (InspectionOptions): maxArchiveEntries 100 000 (walk stops, file.zip.tooManyEntries), maxReportedArchiveEntries 500, maxOverallCompressionRatio 100 (file.zip.ratio.overall), maxEntryCompressionRatio 1000 (file.zip.ratio.entry), maxDeclaredUncompressedBytes 10 GiB (file.zip.declaredSize), maxImagePixels 268 435 456 (file.image.dimensions). Ratios come from declared sizes; nothing is inflated to check them.
  • PDF raw scan: PDFInspector.scan counts the name tokens JavaScript, JS, OpenAction, AA, Launch, EmbeddedFile, RichMedia, XFA, AcroForm, Encrypt, ObjStm (including #xx-escaped spellings) in at most rawScanWindowBytes (8 MiB) at each end of the file, checks the %PDF- header offset and the %%EOF marker and trailing bytes. Where PDFKit exists and the file is ≤ 256 MiB it adds page count, encryption/lock state, attributes and up to 200 link addresses from at most 500 pages. Token presence is reported as presence; code is not analysed.

FileInspector.standingLimits is appended to every inspection: no malware signature database, nothing executed or rendered, macros by presence only, encrypted content unreadable, archives by directory only.

# 6. Verification status

  • Package tests on macOS cover fingerprints, cache, analyzer, plan/executor with an in-memory library, and the file parsers including fuzz tests. They were not re-run while writing this document.
  • The Photos adaptors (PhotoLibraryScanner, PhotoSizeMeasurer, PhotoDeleter) were run in the iOS Simulator only (Scripts/seed-photos.sh seeds a library). Not verified on a physical device: iCloud-only originals, limited access, large libraries, real bursts and Live Photos, the system deletion dialog, and measurement timing. Volume readings in the Simulator describe the host Mac's volume.
Files main
Menu

Theme