SPB Git

spb/wp40_uqo

Public MIT
4.3 KB · 13 lines latex
1\section{Experimental setup}2\label{sec:setup}34\paragraph{Data.} All market data come from FinData~OS, the author's private lakehouse (\url{https://findata.spboucher.ai/api/v1}) built on FirstRate Data US equity bars under a private-use licence; raw bars are not redistributed and only aggregates appear in this paper. Bars are 1-minute OHLCV, stamped at the bar \emph{open} in UTC, with the extended session 04:00--20:00~ET and the regular session 09:30--16:00~ET (the 16:00 bar is the closing print); zero-volume minutes are absent at the source. The universe (experiment E0) is a seeded random sample of 1\,000 symbols among the 6\,046 of the \texttt{market\_bars.stock.1m.raw} dataset that were active through all of 2024 (first session on or before 2024-01-02, last on or after 2024-12-31, delisted suffixes excluded); nested sub-samples of 1, 10 and 100 symbols are its first elements. Twenty asynchronous Parquet exports (50 symbols each, 43.3 million bars, 0.63\,GB, dataset version 17\,375, 437\,s including polling) are cached with their SHA-256 and request identifiers. Of the 1\,000 symbols, 888 have at least 200 regular sessions with positive realized variance and 926 at least 200 sessions of returns; the experiments report the counts they use. For E3 we add ten years of daily bars (2015--2024, 1.82 million rows) for the same symbols. All experiment inputs are listed with hashes in \texttt{experiments/manifests/}.56The provider's own conventions were verified live before use (opt-in integration tests): the per-session expected bar counts (391; 211 on early closes) equal \raid{}'s calendar for every session queried; page downloads equal exports bar for bar; and \raid{}'s calendar-aware 1-minute to 5-minute aggregation in extended-session mode reproduces the provider's derived 5-minute dataset exactly (280 of 280 regular-window bars, OHLCV), while in regular-session mode the only difference is the closing bin, where the provider's 16:00 bar also contains the 16:01--16:04 post-close minutes.78\paragraph{Hardware and software.} The development and benchmark machine is an Apple M5~Max laptop: 18 CPU cores, 40 GPU cores, 48\,GB unified memory, macOS~27.0, Xcode~27.2, Apple clang~21, Metal~4 (GPU families Apple1--Apple10). Dependencies: Eigen~5.0.1 with Accelerate BLAS/LAPACK, Apache Arrow/Parquet~26.0.0, Catch2~3.16.1, nanobind~3.1.0, Python~3.12 with NumPy~2.5.4, pandas, Polars~2.1.0, DuckDB~2.0.0, statsmodels~0.15.0, arch~8.0.0, SciPy~1.18.1; R~4.6.1 with \texttt{highfrequency}~1.0.3 and \texttt{rugarch}~1.5.6. For E7, eleven other Apple Silicon machines of the author's cluster (M2 to M6, 8 to 96\,GB, 10 to 60 GPU cores) run a self-contained bundle built with \texttt{-mcpu=apple-m1} and a macOS~26 deployment target; nothing is installed on them, and the hosts that serve production services were excluded.910\paragraph{Protocol.} Each benchmark case is run twice as warm-up and then $R$ times ($R = 5$ on the laptop, 3 on cluster nodes); case order is shuffled with a fixed seed; the reported statistic is the median of wall-clock seconds with a 2\,000-draw bootstrap 95\,\% confidence interval of the median. A backend ``wins'' when the intervals do not overlap; otherwise the case is a tie. Wall-clock for Metal includes the FP64$\to$FP32 conversion into shared buffers, encoding, commit and completion; GPU device time is reported separately. Shader compilation is measured apart (cold, cache-busted with a salted source; warm; and prebuilt library). Each run writes JSON records and a manifest with host, OS, compiler, library versions, load average, thermal state, seed and input hashes; peak resident set size is read from \texttt{getrusage}. The laptop was shared with other sessions during the runs (load average 2.5--4 on 18 cores), which the manifests record; cluster nodes were idle. Energy was not measured (no instrument), and we make no energy claim.1112\paragraph{Experiments.} E1: batched realized measures versus panel width and segment length. E2: HAR rolling forecasts and batched estimation. E3: GARCH-family panel estimation and a GPU likelihood grid. E4: realized covariance and rolling connectedness. E5: Monte Carlo inference and reproducibility (Section~\ref{sec:correct}). E6: the Arrow/Parquet-to-estimator pipeline against pandas, Polars and DuckDB. E7: the generation sweep. Micro-benchmarks of the seven primitives precede them.13
Files · ae5a4e6ff6df81cef716eacbb5d51143a008d45b
Menu

Theme