SPB Git forge

spb/hfmarketdata

Public

Open high-frequency market data platform — FirstRate full-history downloader, DuckDB/Parquet lake, open REST API and React docs platform (www.hfmarketdata.io)

127commits 1branches 0releases
24.7 MBsize
maindefault branch
11 days agolast push
JavaScript 53.7% Python 38.3% CSS 4.6% TypeScript 3.1%

ops: run.sh — --no-access-log, --proxy-headers, keep-alive/graceful, variables HFMD_DUCK_*/HFMD_THREADPOOL

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed 18 days ago (Sep 6, 2026) parent 58e5d1d

1 changed file +15 −4

modified hfmarketdata/run.sh +15 −4
@@ -1,5 +1,16 @@
1 1 #!/bin/zsh
2 −export HFMD_DATA_ROOT=/Volumes/ssd/firstratedata
3 −export HFMD_WEB_DIST=$HOME/hfmarketdata/dist
4 −export HFMD_PORT=8090
5 −exec $HOME/hfmarketdata/venv/bin/uvicorn --app-dir $HOME/hfmarketdata/api main:app --host 0.0.0.0 --port 8090 --workers 2
2 +# Production launcher (PM2 `hfmarketdata-api` on M3U96b uses its own args — keep them in sync with this file).
3 +export HFMD_DATA_ROOT=${HFMD_DATA_ROOT:-/Volumes/ssd/firstratedata}
4 +export HFMD_WEB_DIST=${HFMD_WEB_DIST:-$HOME/hfmarketdata/dist}
5 +export HFMD_PORT=${HFMD_PORT:-8090}
6 +# DuckDB: one database per worker process, bounded memory / threads (see core/duck.py)
7 +export HFMD_DUCK_MEMORY=${HFMD_DUCK_MEMORY:-4GB}
8 +export HFMD_DUCK_THREADS=${HFMD_DUCK_THREADS:-4}
9 +# anyio worker threads per process for sync endpoints (each holds a DuckDB cursor)
10 +export HFMD_THREADPOOL=${HFMD_THREADPOOL:-8}
11 +# --no-access-log: the API writes its own JSON access log (logger hfmarketdata.access, one line per request)
12 +# --proxy-headers / --forwarded-allow-ips: ngrok fronts the API, client IPs come from X-Forwarded-For
13 +exec $HOME/hfmarketdata/venv/bin/uvicorn --app-dir $HOME/hfmarketdata/api main:app \
14 + --host 0.0.0.0 --port $HFMD_PORT --workers ${HFMD_WORKERS:-2} \
15 + --no-access-log --proxy-headers --forwarded-allow-ips='*' \
16 + --timeout-keep-alive 15 --timeout-graceful-shutdown 20 --log-level info
6 17