SPB Git

Repository Public License

offline-check.sh 27.3 KB 460 lines shellscript

1#!/bin/bash2# ORIX Convert — Scripts/offline-check.sh3# Description: Offline proof for the command-line tool and everything it bundles. (a) Static: lists the networking4#              frameworks and symbols present in every Mach-O (tool, libraries, helpers, LibreOffice). (b) Dynamic: runs5#              one conversion per engine under a sandbox profile that denies IP networking, runs conversions of6#              documents that reference a local canary server and checks the server saw nothing, and records which7#              helper executables were actually started. Writes docs/reports/offline.md.8# Author: Simon-Pierre Boucher — contact@spboucher.ai9#10# Usage: Scripts/offline-check.sh        (ORIX_SKIP_BUILD=1 reuses the existing build)11set -uo pipefail12export LC_ALL=C13ROOT="$(cd "$(dirname "$0")/.." && pwd)"14source "$ROOT/Scripts/cli-build.sh"15orix_build_cli || exit 11617TOOLS="$ROOT/EmbeddedEngines/tools"18REPORT="$ROOT/docs/reports/offline.md"19WORK="$(mktemp -d "${TMPDIR:-/tmp}/orix-offline.XXXXXX")"20trap 'rm -rf "$WORK"' EXIT21mkdir -p "$ROOT/docs/reports" "$WORK/out" "$WORK/canary"22# Everything below runs the tool with a minimal environment: no Homebrew directory in PATH, nothing inherited.23CLEAN=(env -i "PATH=/usr/bin:/bin" "HOME=$HOME" "TMPDIR=${TMPDIR:-/tmp}")24FAILURES=02526md() { printf '%s\n' "$*" >> "$REPORT"; }27: > "$REPORT"28md "# ORIX Convert — offline proof"29md ""30md "> Generated file — do not edit. Written by \`Scripts/offline-check.sh\` on $(date -u +%Y-%m-%dT%H:%M:%SZ) ($(sysctl -n machdep.cpu.brand_string), macOS $(sw_vers -productVersion))."31md ""32md "What is shown here: which networking code exists in the tool and in every bundled binary (stated as found, component by component),"33md "why that code is not reached, and three dynamic checks: conversions under a sandbox that denies IP networking, conversions of documents"34md "that point at a local canary server, and the list of helper executables that were really started."35md ""3637# ---------------------------------------------------------------------------------------------------------------38# (a) Static scan39# ---------------------------------------------------------------------------------------------------------------40NET_LIBS='CFNetwork|Network\.framework|libcurl|libresolv|libnetwork|WebKit\.framework|libssl|libneon|libserf'41NET_SYMS='^_(getaddrinfo|gethostbyname|gethostbyname2|getnameinfo|connect|socket|bind|listen|accept|sendto|recvfrom|res_query|res_search|curl_[a-z_]+|CFSocket[A-Za-z]*|CFStreamCreatePairWithSocket[A-Za-z]*|CFHTTP[A-Za-z]*|CFHost[A-Za-z]*|CFReadStreamCreateForHTTPRequest|nw_[a-z_]+|SSL[A-Z][A-Za-z]*|SecTrust[A-Za-z]*)$|NSURLSession|NSURLConnection|NSURLDownload|NWConnection'4243scan() { # scan <file> → "libs<TAB>symbols" (comma-separated, empty when none)44  local libs syms45  libs=$(otool -L "$1" 2>/dev/null | tail -n +2 | awk '{print $1}' | grep -E "$NET_LIBS" | sed -E 's#.*/##' | sort -u | paste -sd, -)46  syms=$(nm -u "$1" 2>/dev/null | awk '{print $NF}' | grep -E "$NET_SYMS" | sed -E 's/^_//; s/^OBJC_CLASS_\$_//' | sort -u | paste -sd, -)47  printf '%s\t%s' "$libs" "$syms"48}4950md "## 1. Static scan (\`otool -L\`, \`nm -u\`)"51md ""52md "For each Mach-O: the networking libraries it links and the networking symbols it imports (resolver, BSD sockets, CFNetwork,"53md "Network.framework, URLSession, curl, TLS). \"—\" means none were found. A binary that imports \`socket\`/\`connect\` can open a network"54md "connection; the last column says what stands in the way."55md ""56md "| Binary | Size | Networking libraries linked | Networking symbols imported | Why it is not reached |"57md "|---|---:|---|---|---|"5859row() { # row <label> <file> <why>60  local out libs syms size61  [ -e "$2" ] || { md "| $1 | — | not present | not present | — |"; return; }62  out=$(scan "$2"); libs=${out%%$'\t'*}; syms=${out#*$'\t'}63  size=$(stat -f %z "$2" | awk '{printf "%.1f MB", $1/1048576}')64  md "| $1 | $size | ${libs:-—} | ${syms:-—} | $3 |"65}6667row "\`orix-convert\`" "$ORIX_CLI" "No engine and no core source file calls a networking API (source scan below). WebKit is linked for HTML/SVG rendering: see section 2."68row "\`libduckdb.dylib\` (data engine)" "$ORIX_CLI_DIR/libduckdb.dylib" "The data engine opens every database with \`enable_external_access = false\`, \`autoinstall_known_extensions = false\` and \`autoload_known_extensions = false\`; no \`httpfs\` extension is loaded."69row "\`orix-model3d-helper\`" "$ORIX_CLI_DIR/orix-model3d-helper" "Reads one model file and writes a blob in a private folder; no networking call in its sources."70row "\`orix-hdf5-probe\`" "$ORIX_CLI_DIR/orix-hdf5-probe" "Opens one HDF5/NetCDF file in its own process to contain crashes of the HDF5 library; no networking call in its sources."71row "\`ffmpeg\`" "$TOOLS/ffmpeg" "Every input is opened with \`-protocol_whitelist file,pipe\`; the protocols compiled in are listed below."72row "\`ffprobe\`" "$TOOLS/ffprobe" "Same whitelist as ffmpeg."73row "\`pandoc\`" "$TOOLS/pandoc" "Pandoc contains an HTTP client (it can fetch remote images). The engine runs every conversion with a Lua filter that replaces remote images by their alternative text and blanks remote URLs in raw HTML, and with \`http_proxy\`/\`https_proxy\`/\`all_proxy\` pointing at a closed local port (127.0.0.1:9)."74md ""7576# ffmpeg protocols actually compiled in.77if [ -x "$TOOLS/ffmpeg" ]; then78  PROTOCOLS=$("$TOOLS/ffmpeg" -hide_banner -protocols 2>/dev/null | awk '/^Input:/{m=1;next} /^Output:/{m=2;next} m==1{printf "%s ", $1}')79  md "ffmpeg input protocols compiled into the bundled build: \`${PROTOCOLS% }\`."80  NETPROTO=$(printf '%s\n' $PROTOCOLS | grep -E '^(http|https|tcp|udp|rtmp|rtp|rtsp|ftp|tls|srt|sctp|hls)$' | paste -sd, -)81  md "Of these, network protocols: ${NETPROTO:-none}."82  md ""83fi8485# DuckDB: is the HTTP file system compiled in?86if [ -f "$ORIX_CLI_DIR/libduckdb.dylib" ]; then87  HTTPFS=$(strings -a "$ORIX_CLI_DIR/libduckdb.dylib" | grep -c -i 'httpfs' || true)88  md "\`libduckdb.dylib\` mentions \`httpfs\` in $HTTPFS string(s): the name of the optional extension is known to DuckDB, but the extension itself is not"89  md "part of the library and cannot be installed or loaded (auto-install and auto-load are off, external access is off)."90  md ""91fi9293# LibreOffice: every Mach-O in the bundle.94LO="$TOOLS/LibreOffice.app"95if [ -d "$LO" ]; then96  : > "$WORK/lo.tsv"97  while IFS= read -r file; do98    head -c 4 "$file" 2>/dev/null | xxd -p | grep -qE '^(cffaedfe|cafebabe|feedfacf)$' || continue99    printf '%s\t%s\n' "${file#$LO/}" "$(scan "$file")" >> "$WORK/lo.tsv"100  done < <(find "$LO/Contents" -type f \( -name '*.dylib' -o -name '*.so' -o -name '*.jnilib' -o -perm +111 \) 2>/dev/null)101  LO_TOTAL=$(wc -l < "$WORK/lo.tsv" | tr -d ' ')102  LO_NET=$(awk -F'\t' '$2 != "" || $3 != ""' "$WORK/lo.tsv" | wc -l | tr -d ' ')103  md "### LibreOffice.app ($(du -sh "$LO" | awk '{print $1}'), $LO_TOTAL Mach-O files)"104  md ""105  md "**LibreOffice contains networking code**: $LO_NET of its $LO_TOTAL Mach-O files link a networking library or import networking symbols."106  md "It ships its own HTTP stack (curl), TLS (NSS), a Firebird database client, GnuPG helpers and its UNO socket layer:"107  md ""108  md "| File (inside LibreOffice.app) | Networking libraries linked | Networking symbols imported |"109  md "|---|---|---|"110  awk -F'\t' '$2 != "" || $3 != "" { s=$3; if (length(s) > 160) s=substr(s,1,160) "…"; printf "| `%s` | %s | %s |\n", $1, ($2==""?"—":$2), (s==""?"—":s) }' "$WORK/lo.tsv" | sort >> "$REPORT"111  md ""112  CURL=$(find "$LO/Contents" -name 'libcurl*' | sed "s#$LO/##" | paste -sd, -)113  md "Bundled curl library: ${CURL:-none found by name}."114  md ""115  md "Why it is not reached: the engines start \`soffice\` headless for one document, with a fresh private profile whose configuration sets"116  md "an HTTP and HTTPS proxy on a closed local port (\`127.0.0.1:9\`, proxy type \"manual\", empty no-proxy list),"117  md "and never refreshes linked content (images, OLE links, sections, DDE); macros are disabled. The update check is not switched off by a"118  md "setting of its own; if a headless run attempted it, the request would meet the same closed proxy. A request LibreOffice would still attempt"119  md "ends at the closed port. Sections 2 and 3 check this by running it."120  md ""121fi122123# Source scan: networking APIs named in first-party sources.124md "### Source scan"125md ""126SRC_HITS=$(grep -rnE 'URLSession|URLRequest|NWConnection|NWListener|import Network|CFSocket|CFStreamCreatePairWithSocket|NSURLConnection|SocketPort|getaddrinfo|gethostbyname' \127  "$ROOT"/Packages/*/Sources --include='*.swift' --include='*.c' --include='*.cpp' --include='*.h' --include='*.m' 2>/dev/null | grep -v '^\s*//' || true)128if [ -z "$SRC_HITS" ]; then129  md "No first-party source file under \`Packages/*/Sources\` names \`URLSession\`, \`URLRequest\`, \`Network.framework\` types, CFNetwork sockets or the resolver."130else131  md "First-party source lines naming a networking API:"132  md ""133  md '```'134  printf '%s\n' "$SRC_HITS" | sed "s#$ROOT/##" | cut -c1-220 >> "$REPORT"135  md '```'136  md ""137  md "(\`WebPDF.swift\` builds a \`URLRequest\` for the document's own private scheme, served from memory by a \`WKURLSchemeHandler\`; it is how"138  md "WebKit is told to load the local document, not a network request.)"139fi140WEBKIT_FILES=$(grep -rlE 'WKWebView' "$ROOT"/Packages/*/Sources 2>/dev/null | sed "s#$ROOT/##" | paste -sd, - || true)141md ""142md "WebKit (\`WKWebView\`) is used by: ${WEBKIT_FILES:-no file}. WebKit is a system framework with its own networking process. The engines load"143md "the document from memory or through a private URL scheme, with JavaScript disabled, a non-persistent data store, a content-rule list that"144md "blocks every URL except the document's own scheme, \`data:\` and \`about:\`, and a navigation delegate that cancels any other navigation."145md "The SVG renderer additionally wraps the drawing in a page whose Content-Security-Policy is \`default-src 'none'; img-src data:\`."146if grep -q '\^(about|data):' "$ROOT/Packages/EngineVector/Sources/EngineVector/SVGWebRenderer.swift" 2>/dev/null; then147  md ""148  md "**Finding:** the content-rule list of the SVG renderer (\`EngineVector/SVGWebRenderer.swift\`) uses the filter \`^(about|data):\`. WebKit's rule"149  md "compiler rejects disjunctions (\"Disjunctions are not supported yet\", printed on stderr at run time), the compilation result is discarded with"150  md "\`try?\`, and the renderer then runs **without** its rule list. The CSP, the disabled JavaScript and the navigation delegate remain; the canary"151  md "test in section 3 shows whether that is enough. Fix: two rules, \`^about:\` and \`^data:\`, as the documents engine does."152fi153md ""154155# ---------------------------------------------------------------------------------------------------------------156# Fixtures (generated in code and through the engines)157# ---------------------------------------------------------------------------------------------------------------158"${CLEAN[@]}" "$ORIX_CLI" dev fixtures "$WORK/fixtures" > "$WORK/fixtures.tsv" 2> "$WORK/fixtures.err" || true159fixture() { awk -F'\t' -v f="$1" '$1 == f { print $2; exit }' "$WORK/fixtures.tsv"; }160161# One representative conversion per engine: "engine|source format|destination|extra arguments".162CASES=(163  "image|png|jpeg|"164  "vector|png|svg|--operation image.vectorize"165  "vector (WebKit)|svg|png|"166  "pdf|pdf|png|"167  "video (ffmpeg)|mp4|mov|"168  "audio (ffmpeg)|wav|mp3|"169  "documents (Pandoc)|markdown|docx|"170  "documents (LibreOffice)|docx|pdf|"171  "documents (WebKit)|html|pdf|"172  "ebooks|epub|html|"173  "presentations (LibreOffice)|pptx|pdf|"174  "data (DuckDB)|csv|parquet|"175  "data (LibreOffice)|xlsx|xls|"176  "archive|zip|folder|--operation archive.extract"177  "font|ttf|woff2|"178  "subtitle|srt|vtt|"179  "model3d|obj|glb|"180  "geo|geojson|kml|"181  "scientific|hdf5|csv|"182)183184run_cases() { # run_cases <label> <output tsv> <command prefix…>185  local label="$1" out="$2"; shift 2186  : > "$out"187  local i=0188  for entry in "${CASES[@]}"; do189    i=$((i + 1))190    IFS='|' read -r engine source destination extra <<< "$entry"191    local file; file=$(fixture "$source")192    if [ -z "$file" ]; then printf '%s\t%s → %s\tno fixture\t—\n' "$engine" "$source" "$destination" >> "$out"; continue; fi193    local start end status194    start=$(perl -MTime::HiRes=time -e 'printf "%.3f", time')195    # shellcheck disable=SC2086196    "$@" "$ORIX_CLI" convert "$file" --to "$destination" $extra --output-dir "$WORK/out/$label-$i" --no-history --quiet > "$WORK/out/$label-$i.log" 2>&1197    status=$?198    end=$(perl -MTime::HiRes=time -e 'printf "%.3f", time')199    [ "$status" -eq 0 ] || FAILURES=$((FAILURES + 1))200    printf '%s\t%s → %s%s\t%s\t%s\n' "$engine" "$source" "$destination" "${extra:+ ($extra)}" "$status" "$(echo "$end - $start" | bc)" >> "$out"201  done202}203204# ---------------------------------------------------------------------------------------------------------------205# (b1) Conversions under a sandbox that denies IP networking206# ---------------------------------------------------------------------------------------------------------------207md "## 2. Conversions under a sandbox that denies networking"208md ""209PROFILE="$WORK/no-network.sb"210cat > "$PROFILE" <<'SB'211(version 1)212(allow default)213(deny network*)214; LibreOffice talks to itself through a UNIX-domain socket in its private profile folder; nothing else is allowed.215(allow network* (local unix-socket))216(allow network* (remote unix-socket))217SB218METHOD="sandbox-exec"219if ! sandbox-exec -f "$PROFILE" /usr/bin/true 2>/dev/null; then METHOD="unavailable"; fi220if [ "$METHOD" = "sandbox-exec" ]; then221  # The profile must really block: an outbound TCP connection to the loopback interface has to fail inside it.222  /usr/bin/nc -l 127.0.0.1 47999 >/dev/null 2>&1 & NCPID=$!223  sleep 0.5224  if sandbox-exec -f "$PROFILE" /usr/bin/nc -z -w 2 127.0.0.1 47999 >/dev/null 2>&1; then BLOCKED="NO — the profile did not block a TCP connection"; FAILURES=$((FAILURES + 1)); else BLOCKED="yes"; fi225  /usr/bin/nc -z -w 2 127.0.0.1 47999 >/dev/null 2>&1 && CONTROL="yes" || CONTROL="no"226  kill "$NCPID" 2>/dev/null; wait "$NCPID" 2>/dev/null227  md "Method: \`sandbox-exec -f no-network.sb\` (deprecated by Apple but present and working on this system). Profile:"228  md ""229  md '```'230  cat "$PROFILE" >> "$REPORT"231  md '```'232  md ""233  md "Control: a TCP connection to a listener on 127.0.0.1 succeeds outside the sandbox ($CONTROL) and is refused inside it ($BLOCKED)."234  md "The profile applies to \`orix-convert\` and to every process it starts (ffmpeg, ffprobe, pandoc, soffice, the 3D helper)."235  md "It does **not** apply to WebKit's own helper processes, which the system starts outside the tool's process tree; section 3 covers those."236  md "With \`(deny network*)\` alone LibreOffice does not start (it needs its local UNIX-domain socket), hence the two \`allow\` lines for UNIX sockets only."237  md ""238  run_cases sandbox "$WORK/sandbox.tsv" "${CLEAN[@]}" sandbox-exec -f "$PROFILE"239  md "| Engine | Conversion | Exit status | Seconds |"240  md "|---|---|---:|---:|"241  awk -F'\t' '{ printf "| %s | %s | %s | %s |\n", $1, $2, $3, $4 }' "$WORK/sandbox.tsv" >> "$REPORT"242  OK=$(awk -F'\t' '$3 == "0"' "$WORK/sandbox.tsv" | wc -l | tr -d ' ')243  md ""244  md "$OK of ${#CASES[@]} conversions succeeded with networking denied."245else246  md "\`sandbox-exec\` could not be used on this system; this check was not run. Section 3 (canary server) is the dynamic evidence."247  FAILURES=$((FAILURES + 1))248fi249md ""250251# ---------------------------------------------------------------------------------------------------------------252# (b2) Canary server: documents that reference a local HTTP server253# ---------------------------------------------------------------------------------------------------------------254md "## 3. Documents that reference a server: canary test"255md ""256if [ -x /usr/bin/python3 ]; then257  PORT=$((20000 + RANDOM % 20000))258  cat > "$WORK/canary/server.py" <<'PY'259import http.server, sys260log = open(sys.argv[2], "a", buffering=1)261class Handler(http.server.BaseHTTPRequestHandler):262    def do_GET(self):263        log.write(self.path + "\n")264        self.send_response(200); self.send_header("Content-Type", "image/png"); self.end_headers()265    do_HEAD = do_POST = do_CONNECT = do_GET266    def log_message(self, *args): pass267http.server.ThreadingHTTPServer(("127.0.0.1", int(sys.argv[1])), Handler).serve_forever()268PY269  : > "$WORK/canary/requests.log"270  /usr/bin/python3 -I "$WORK/canary/server.py" "$PORT" "$WORK/canary/requests.log" & SERVER=$!271  sleep 1272  /usr/bin/curl -s -o /dev/null "http://127.0.0.1:$PORT/control" || true273  CONTROL_SEEN=$(grep -c '^/control$' "$WORK/canary/requests.log" || true)274  URL="http://127.0.0.1:$PORT"275  cat > "$WORK/canary/page.html" <<EOF276<!DOCTYPE html><html><head><meta charset="utf-8"><title>Canary</title>277<link rel="stylesheet" href="$URL/html-style.css"><script src="$URL/html-script.js"></script>278<style>@import url("$URL/html-import.css"); body { background: url("$URL/html-background.png"); } @font-face { font-family: C; src: url("$URL/html-font.woff2"); }</style></head>279<body><h1>Canary</h1><p>Text.</p><img src="$URL/html-image.png" alt="remote"><iframe src="$URL/html-frame.html"></iframe>280<object data="$URL/html-object.svg"></object></body></html>281EOF282  printf '# Canary\n\nText and a remote picture: ![remote](%s/markdown-image.png)\n\n<img src="%s/markdown-raw-image.png">\n' "$URL" "$URL" > "$WORK/canary/note.md"283  cat > "$WORK/canary/drawing.svg" <<EOF284<?xml version="1.0" encoding="UTF-8"?>285<?xml-stylesheet href="$URL/svg-style.css" type="text/css"?>286<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="200" height="100">287  <rect width="200" height="100" fill="#ddd"/><image href="$URL/svg-image.png" width="50" height="50"/>288  <image xlink:href="$URL/svg-xlink-image.png" x="60" width="50" height="50"/><use href="$URL/svg-use.svg#a"/>289</svg>290EOF291  printf '\\documentclass{article}\\usepackage{graphicx}\\begin{document}Canary \\includegraphics{%s/latex-image.png}\\end{document}\n' "$URL" > "$WORK/canary/paper.tex"292  printf 'Canary\n======\n\n.. image:: %s/rst-image.png\n' "$URL" > "$WORK/canary/page.rst"293  CANARY=(294    "page.html|pdf|WebKit"295    "page.html|docx|Pandoc"296    "page.html|odt|Pandoc or LibreOffice"297    "page.html|epub|Pandoc"298    "page.html|markdown|Pandoc"299    "page.html|rtf|documents"300    "note.md|pdf|Pandoc + WebKit"301    "note.md|docx|Pandoc"302    "note.md|html|Pandoc"303    "note.md|epub|Pandoc"304    "drawing.svg|png|WebKit"305    "drawing.svg|pdf|WebKit"306    "paper.tex|docx|Pandoc"307    "page.rst|html|Pandoc"308  )309  md "A web server listens on \`127.0.0.1:$PORT\` and logs every request (control request seen: $([ "$CONTROL_SEEN" = "1" ] && echo yes || echo NO))."310  md "The sources below reference it in every way their format allows (images, stylesheets, \`@import\`, fonts, scripts, frames, objects, SVG"311  md "\`<image>\` and \`<use>\`, LaTeX \`\\includegraphics\`, reStructuredText \`image\`). The conversions run **without** the sandbox, so a fetch"312  md "would succeed if any component attempted it — including WebKit's helper processes. The documents produced by the first pass (DOCX, ODT, EPUB,"313  md "which may still carry the remote reference as a link) are then converted again by LibreOffice and the ebook engine."314  md ""315  md "| Source | Destination | Converter | Exit status | Requests received during the conversion |"316  md "|---|---|---|---:|---:|"317  n=0318  canary_one() { # <source path> <destination> <label> <converter>319    n=$((n + 1))320    local before after321    before=$(wc -l < "$WORK/canary/requests.log" | tr -d ' ')322    "${CLEAN[@]}" "$ORIX_CLI" convert "$1" --to "$2" --output-dir "$WORK/canary/out-$n" --no-history --quiet > "$WORK/canary/out-$n.log" 2>&1323    local status=$?324    sleep 0.3325    after=$(wc -l < "$WORK/canary/requests.log" | tr -d ' ')326    md "| $3 | $2 | $4 | $status | $((after - before)) |"327  }328  for entry in "${CANARY[@]}"; do329    IFS='|' read -r file destination converter <<< "$entry"330    canary_one "$WORK/canary/$file" "$destination" "\`$file\`" "$converter"331  done332  # Second pass: what the first pass produced, through LibreOffice and the ebook reader.333  for produced in "$WORK"/canary/out-2/page.docx "$WORK"/canary/out-3/page.odt; do334    [ -f "$produced" ] && canary_one "$produced" pdf "\`$(basename "$produced")\` (made above)" "LibreOffice"335  done336  [ -f "$WORK/canary/out-4/page.epub" ] && canary_one "$WORK/canary/out-4/page.epub" html "\`page.epub\` (made above)" "ebooks"337  [ -f "$WORK/canary/out-4/page.epub" ] && canary_one "$WORK/canary/out-4/page.epub" pdf "\`page.epub\` (made above)" "ebooks + WebKit"338  sleep 1339  kill "$SERVER" 2>/dev/null; wait "$SERVER" 2>/dev/null340  TOTAL_REQUESTS=$(grep -vc '^/control$' "$WORK/canary/requests.log" || true)341  md ""342  if [ "$TOTAL_REQUESTS" = "0" ] && [ "$CONTROL_SEEN" = "1" ]; then343    md "**The canary server received no request from any conversion** (only the control request)."344  else345    FAILURES=$((FAILURES + 1))346    md "**The canary server received $TOTAL_REQUESTS request(s)** (control seen: $CONTROL_SEEN):"347    md ""348    md '```'349    grep -v '^/control$' "$WORK/canary/requests.log" >> "$REPORT"350    md '```'351  fi352  md "A non-zero exit status in the table is not a network event: the conversion ran, the remote reference was blocked (the job log says"353  md "\`remote-content-blocked\`), and the documents engine then failed its own \"Images kept\" verification because the blocked picture is"354  md "missing from the output — a defect of that verification, reported to the engine's author (the logs are quoted below for the first case)."355  md ""356  md '```'357  grep -v -E 'attributedString|CoreText' "$WORK/canary/out-1.log" | cut -c1-240 | head -6 >> "$REPORT"358  md '```'359else360  md "\`/usr/bin/python3\` is not available: the canary server could not be started and this check was not run."361fi362md ""363364# ---------------------------------------------------------------------------------------------------------------365# (c) Tools: where they come from366# ---------------------------------------------------------------------------------------------------------------367md "## 4. No tool from PATH, Homebrew or an installed application"368md ""369md "All runs above use \`env -i PATH=/usr/bin:/bin HOME=… TMPDIR=…\`: no Homebrew directory is in the environment. The end-to-end sweep"370md "(\`Scripts/matrix.sh --verify\`, \`docs/reports/e2e-matrix.md\`) runs with the same environment and records its PATH."371md ""372# Deterministic check: the same conversions with Homebrew, /usr/local and the installed LibreOffice made unreadable and373# unexecutable by a sandbox profile. If anything depended on them, it would fail here.374ISOLATION="$WORK/no-installed-software.sb"375cat > "$ISOLATION" <<'SB'376(version 1)377(allow default)378(deny file-read* process-exec* (subpath "/opt/homebrew") (subpath "/usr/local") (subpath "/opt/local")379      (subpath "/Applications/LibreOffice.app") (subpath "/Library/Frameworks/Python.framework"))380SB381if sandbox-exec -f "$ISOLATION" /usr/bin/true 2>/dev/null; then382  if [ -e /opt/homebrew/bin ] && sandbox-exec -f "$ISOLATION" /bin/ls /opt/homebrew/bin >/dev/null 2>&1; then HIDDEN="NO"; FAILURES=$((FAILURES + 1)); else HIDDEN="yes"; fi383  before=$FAILURES384  run_cases isolated "$WORK/isolated.tsv" "${CLEAN[@]}" sandbox-exec -f "$ISOLATION"385  OK=$(awk -F'\t' '$3 == "0"' "$WORK/isolated.tsv" | wc -l | tr -d ' ')386  md "**Installed software hidden.** The ${#CASES[@]} representative conversions were run under a sandbox profile that denies reading and executing"387  md "anything under \`/opt/homebrew\`, \`/usr/local\`, \`/opt/local\`, \`/Applications/LibreOffice.app\` and the Python framework"388  md "(control: \`ls /opt/homebrew/bin\` is refused inside the profile: $HIDDEN). Result: **$OK of ${#CASES[@]} succeeded**$([ "$FAILURES" -eq "$before" ] && echo "" || echo " — failures: $(awk -F'\t' '$3 != "0" {printf "%s (%s); ", $2, $3}' "$WORK/isolated.tsv")")."389  md ""390else391  md "The isolation profile could not be loaded by \`sandbox-exec\`; that check was not run."392  md ""393fi394395# Helper executables really started: sample the process table while the representative set runs once more.396: > "$WORK/ps.log"397( while :; do ps -axo pid=,ppid=,comm= >> "$WORK/ps.log" 2>/dev/null; echo "--" >> "$WORK/ps.log"; sleep 0.05; done ) & SAMPLER=$!398run_cases clean "$WORK/clean.tsv" "${CLEAN[@]}"399kill "$SAMPLER" 2>/dev/null; wait "$SAMPLER" 2>/dev/null400# Descendants of any orix-convert process, per snapshot.401awk -v cli="$ORIX_CLI" '402  $0 == "--" {403    changed = 1404    while (changed) { changed = 0; for (p in ppid) if (!(p in mine) && (ppid[p] in mine)) { mine[p] = 1; changed = 1 } }405    for (p in mine) if (comm[p] != cli && comm[p] !~ /^\(/) seen[comm[p]] = 1406    delete ppid; delete comm; delete mine; next407  }408  { pid = $1; par = $2; $1 = ""; $2 = ""; sub(/^ +/, ""); ppid[pid] = par; comm[pid] = $0; if ($0 == cli) mine[pid] = 1 }409  END { for (c in seen) print c }' "$WORK/ps.log" | sort > "$WORK/helpers.txt"410md "Helper executables observed in the process tree of \`orix-convert\` while the ${#CASES[@]} representative conversions ran again (the process"411md "table is sampled about every 50 ms, so a helper that lives for a few milliseconds can be missed; the sandbox run above is the complete check):"412md ""413md '```'414sed "s#$ROOT/##" "$WORK/helpers.txt" >> "$REPORT"415md '```'416OUTSIDE=$(grep -v "^$ROOT/" "$WORK/helpers.txt" | grep -vE '^/(System|usr/libexec|usr/bin|bin|usr/sbin|sbin)/' || true)417SYSTEM_BIN=$(grep -E '^/(usr/bin|bin|usr/sbin|sbin|usr/libexec|System)/' "$WORK/helpers.txt" || true)418if [ -z "$OUTSIDE" ]; then419  md ""420  md "Every helper comes from the repository (\`EmbeddedEngines/tools\` or the build folder)$([ -n "$SYSTEM_BIN" ] && echo ", apart from the macOS system executables listed above (started by a bundled tool, not looked up by an engine)")."421else422  FAILURES=$((FAILURES + 1))423  md ""424  md "**Executables from outside the repository were started:**"425  md '```'426  printf '%s\n' "$OUTSIDE" >> "$REPORT"427  md '```'428fi429OK=$(awk -F'\t' '$3 == "0"' "$WORK/clean.tsv" | wc -l | tr -d ' ')430md ""431md "$OK of ${#CASES[@]} conversions succeeded in that run (minimal environment, no sandbox)."432md ""433md "Software installed on this Mac that the tool could have picked up by mistake, and did not:"434md ""435md "| Candidate | Present on this Mac | Used |"436md "|---|---|---|"437for candidate in /opt/homebrew/bin/ffmpeg /opt/homebrew/bin/ffprobe /opt/homebrew/bin/pandoc /opt/homebrew/bin/soffice /usr/local/bin/ffmpeg /usr/local/bin/pandoc /Applications/LibreOffice.app /opt/homebrew/bin/duckdb /opt/homebrew/bin/7z; do438  if [ -e "$candidate" ]; then present="yes"; else present="no"; fi439  if grep -q "^$candidate" "$WORK/helpers.txt"; then used="YES"; FAILURES=$((FAILURES + 1)); else used="no"; fi440  md "| \`$candidate\` | $present | $used |"441done442md ""443md "\`orix-convert doctor\` in the same minimal environment (tool search directories and resolved paths):"444md ""445md '```'446"${CLEAN[@]}" "$ORIX_CLI" doctor 2>&1 | sed "s#$ROOT#<repo>#g" >> "$REPORT"447DOCTOR=${PIPESTATUS[0]}448md '```'449[ "$DOCTOR" -eq 0 ] || FAILURES=$((FAILURES + 1))450md ""451md "Caveat that applies to an installed copy: \`BundledTools\` (core) also searches \`EmbeddedEngines/tools\` found by walking up from the path of its own"452md "source file as recorded at build time. On a user's Mac that path does not exist; on the build machine it does, so a missing tool in an"453md "app bundle would silently be taken from the repository there. \`doctor\` prints the search directories for that reason."454md ""455md "## Result"456md ""457if [ "$FAILURES" -eq 0 ]; then md "All checks passed."; else md "**$FAILURES check(s) failed — see above.**"; fi458echo "offline-check.sh: wrote $REPORT ($FAILURES failed check(s))"459[ "$FAILURES" -eq 0 ]460
Files main
Menu

Theme