OCR engine
Text recognition in ORIX Scan is Apple's Vision framework, called on the device. The code is in
Packages/ORIXKit/Sources/ORIXKit/OCR/.
Local by construction
OCREngine creates a VNRecognizeTextRequest and performs it with a VNImageRequestHandler on a CGImage held in memory.
The repository contains no networking code: there is no URLSession, no socket and no third-party SDK in ORIXKit,
ORIXDesign or the app target, and the package has no dependencies. Recognized text is written to a JSON file next to the
page (ocr/<page>.json) and to the on-device SwiftData and Spotlight indexes. It leaves the device only when the user
shares an export.
Pipeline
flowchart TD
Page["PageRecord + PageEdit"] --> Render["DocumentProcessor.renderedImage<br/>(crop, straighten, rotate, filter; max 4200 px)"]
Render --> BW{"filter is blackWhite?"}
BW -- yes --> Grey["recognize the greyscale render instead"]
BW -- no --> Req
Grey --> Req["VNRecognizeTextRequest"]
Req --> Obs["observations: text, confidence, line box"]
Obs --> Words["word boxes via boundingBox(for:)"]
Words --> Order["ReadingOrder.sorted (XY-cut)"]
Order --> Result["OCRPage with geometryKey"]
Result --> Save[("ocr/<page>.json")]
Save --> Text["plainText, search index"]
Save --> Tables["TableExtractor"]
Save --> Layer["PDF text layer"]
What OCREngine configures
OCREngine.recognize(_:options:geometryKey:progress:) takes an upright CGImage and OCROptions:
| Option | Effect on the request |
|---|---|
mode: .fast |
recognitionLevel = .fast |
mode: .accurate |
recognitionLevel = .accurate |
mode: .automatic |
recognitionLevel = .accurate and automaticallyDetectsLanguage = true |
languages |
Filtered against supportedRecognitionLanguages() of the request, then set as recognitionLanguages when not empty. With no usable language, automatic language detection is turned on |
usesLanguageCorrection |
Passed through |
Languages are never hard-coded. OCREngine.supportedLanguages(mode:) returns what the installed Vision revision
reports for the level of that mode. The Settings language list (LanguagePicker) is built from that call, so the app cannot
advertise a language the system does not support.
Cancellation. recognize checks for cancellation first, then wraps the work in withTaskCancellationHandler. A small
lock-protected box gives the handler the in-flight VNRequest so it can call cancel(). A cancelled run throws
ORIXError.cancelled.
Progress. When a progress closure is supplied, the request's progressHandler forwards Vision's fraction.
DocumentProcessor.recognize scales it across the pages of the document.
OCRPage
| Field | Meaning |
|---|---|
imageWidth, imageHeight |
Pixel size of the image that was recognized |
lines |
[OCRLine] in reading order |
mode, languages |
What was requested |
geometryKey |
Geometry of the edit the boxes belong to |
recognizedAt, duration |
Time stamp and seconds spent |
An OCRLine carries text, recognizedText, confidence (Vision's value for the top candidate), a Quad, and words.
Each OCRWord is a whitespace-separated token (OCREngine.tokenRanges, punctuation stays attached) with the box Vision
returns for that range. All coordinates are converted from Vision's bottom-left origin to the top-left, normalized space of
Quad.
Reading order
ReadingOrder is a recursive XY-cut over line bounding boxes. At each level it first tries to split into columns along
vertical whitespace bands at least max(1.8 × median line height, 0.025) wide, then into rows along horizontal bands of
1.4 × the median line height, to a depth of 12. A column split at the top level is accepted only when every group has at
least two lines, which keeps a short stray line from creating a false column. Leaves are sorted top to bottom, and left to
right within a visual row. OCRPage.plainText joins the lines of a block with newlines and blocks with a blank line.
Skew measurement
OCRPage.skewDegrees is the median baseline angle of lines with at least six characters. For a line with three or more
words the angle is the least-squares slope of the word-box centers, corrected for the image aspect ratio; shorter lines fall
back to the line box.
The word-center method exists because of a measured behaviour: on a page rotated by 4°, Vision's accurate level returned
rotated line quadrilaterals, but the fast level returned axis-aligned line boxes, giving an angle of 0. The word boxes follow
the tilted baseline at both levels. DocumentProcessor.deskew uses a fast probe at 1600 px and applies a correction only
between 0.3° and 15°.
Table extraction
TableExtractor.extract(from:) works from geometry only:
- Tokens: the word boxes of each line (or the line box when the line was edited or has no words).
- Rows: tokens whose vertical centers are within 0.6 of the smaller height are grouped.
- Segments: in a row, tokens closer than about 1.25 line heights are merged into one cell candidate.
- Candidates: runs of at least two consecutive rows holding two or more segments, with no vertical gap larger than three line heights.
- Columns: an occupancy histogram along x; gutters are intervals no segment crosses (a tolerance of one tenth of the rows applies from eight rows up). Segments are assigned to the column containing their center.
- Uncertainty: a cell is flagged when its segment spans a gutter, when two segments landed in the same cell, or when
recognition confidence is below 0.5.
confidenceis the share of rows that matched the grid exactly multiplied by the mean recognition confidence.
ExtractedTable.csv is RFC 4180 (CRLF, quoting when a field contains a comma, a quote or a line break).
Limits, stated in the interface as well:
- Merged cells, nested headers and cell text wrapped on several lines are not reconstructed faithfully.
- Ruling lines are ignored; only text positions are used.
- Measured: in a synthetic table, Vision did not return cells made of a single isolated character (a lone digit such as
3or7), at any level and with or without language correction. Those cells come out empty. Two-character cells in the same table were recognized. - The Tables screen lets the user correct any cell before exporting; those corrections are applied to that export only.
Text correction
The Review screen (App/Sources/Document/DocumentTextView.swift) edits OCRLine.text; recognizedText keeps Vision's
output so a correction can be reverted (isEdited). Corrections are saved to the page's JSON and affect copied text, the
text export, the search index and the PDF text layer. An edited line is drawn in the PDF as one run across the line box,
because its word boxes no longer match its words.
Geometry key
Recognized boxes are valid only for the geometry they were measured on. PageEdit.geometryKey hashes the crop quad, the
quarter turns and the straightening angle, and ignores colour settings. DocumentProcessor.updateEdits deletes a page's
recognition when the geometry changes, and every consumer (text(of:in:), tables(of:), exportPDF) compares
OCRPage.geometryKey with the page's current key, so a stale text layer cannot be exported.
Imported PDF pages that already contain text are not recognized again; their text is read with PDFKit
(PDFTools.embeddedText).
Handwriting
ORIX Scan does not offer a handwriting feature. Vision's accurate level may read clear handwriting, but this has not been validated on real handwritten documents, so the app makes no claim about it.