SPB Git forge

spb/hfmarketdata

Public

Open high-frequency market data platform — FirstRate full-history downloader, DuckDB/Parquet lake, open REST API and React docs platform (www.hfmarketdata.io)

127commits 1branches 0releases
24.7 MBsize
maindefault branch
11 days agolast push
JavaScript 53.7% Python 38.3% CSS 4.6% TypeScript 3.1%

web: pré-rendu post-build (coquilles HTML par route + modulepreload + sitemap), suite E2E Playwright (docs, recherche ⌘K, lien Try it, limites, thème, mobile) et README (architecture, API des composants, génération des docs, contrat /playground?ep=…)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed 20 days ago (Sep 5, 2026) parent 708314f

7 changed files +537 −0

added hfmarketdata/web/.gitignore +7 −0
@@ -0,0 +1,7 @@
1 +node_modules/
2 +dist/
3 +test-results/
4 +playwright-report/
5 +blob-report/
6 +.playwright/
7 +e2e/.*.tmp.mjs
added hfmarketdata/web/README.md +155 −0
@@ -0,0 +1,155 @@
1 +# hfmarketdata/web — site, docs, playground, dashboard
2 +
3 +Vite 6 + React 18 + react-router 6. Dark by default, light theme persisted. No CSS framework, no external
4 +requests at runtime (system font stacks, inline SVG, all JS bundled). Served by the FastAPI app from `dist/`
5 +(SPA fallback in `api/main.py`), or by `vite preview` for E2E.
6 +
7 +```
8 +npm install
9 +npm run dev # http://localhost:5173 — proxies /v1, /health, /openapi.json to :8090
10 +npm run build # vite build + scripts/prerender.mjs → dist/
11 +npm run test:e2e # Playwright against `vite preview` (build first)
12 +npm run examples # refresh content/examples/*.json + openapi snapshot from production
13 +```
14 +
15 +## Architecture
16 +
17 +```
18 +index.html theme applied before first paint (inline script reads localStorage hfmd.theme)
19 +src/main.jsx BrowserRouter › AuthProvider › LangProvider › App
20 +src/App.jsx route table, one lazy chunk per area (owner comments inside)
21 +src/app/ Layout (topbar · drawer · search · footer), theme.css (tokens), theme.js, api.js, auth.jsx
22 +src/components/ shared UI (below) + components.css
23 +src/docs/ docs engine: spec.js (OpenAPI loader), enrich.js (example values), snippets.js (code gen),
24 + guides.js (registry), search.js (index), mdx-components.jsx, useTitle.js
25 +src/pages/home|limits|status marketing / limits / live status pages (web-core)
26 +src/pages/docs DocsLayout (3 columns), DocsHome, ReferenceIndex, OperationPage, GuidePage, ErrorsPage
27 +src/pages/playground|auth|dashboard|admin web-app agent · src/pages/integrations mcp agent
28 +content/guides/*.mdx guides (MDX) · content/changelog.mdx
29 +content/examples/<opId>.json captured real responses (scripts/fetch-examples.mjs) — commit them
30 +content/openapi.snapshot.json bundled spec fallback when /openapi.json is unreachable (e.g. vite preview)
31 +content/samples/aapl-pe.json homepage chart fallback (scripts/make-aapl-pe-sample.mjs)
32 +scripts/ prerender.mjs · fetch-examples.mjs · make-aapl-pe-sample.mjs · bench-formats.mjs · bench-parquet.py
33 +e2e/ Playwright specs (site, docs, mobile) · playwright.config.js
34 +public/ favicon.svg, logo.svg, og.svg, robots.txt (copied verbatim to dist/)
35 +```
36 +
37 +Chunks (`vite.config.js › manualChunks`): `vendor` (react, router), `prism`, `charts` (lightweight-charts, only
38 +loaded by the homepage chart / playground), `mdx`, one chunk per page and per guide.
39 +
40 +### Theme
41 +
42 +Tokens live in `src/app/theme.css` (`:root` dark, `[data-theme="light"]` overrides). `src/app/theme.js` exposes
43 +`useTheme()` → `{ theme, toggle, set }` and `usePersisted(key, initial)` (localStorage-backed `useState`). Colours for
44 +code blocks are CSS variables too (`--code-*`) so Prism follows the theme without a second stylesheet. HTTP verb colours:
45 +`--get --post --put --patch --delete`.
46 +
47 +## Shared components (`src/components/`, barrel `index.js`)
48 +
49 +| Component | Props | Notes |
50 +| --- | --- | --- |
51 +| `Code` | `code, language='text', title?, copy=true, maxHeight?, lineNumbers=false` | Prism (prism-react-renderer) + bundled `bash`/`r` grammars added in `Code.jsx`; copy button; scrolls inside when `maxHeight` set. `CopyButton({ text })`, `useCopy()` exported. |
52 +| `CodeTabs` | `snippets, title?, maxHeight?` | `snippets` = `{ curl, python, javascript, r }` (any subset) **or** `[{ id, label, language, code }]`. Uses the global persisted language (`hfmd.lang`). |
53 +| `Tabs` | `items:[{id,label,content?,icon?}], value?, onChange?, defaultValue?, size='md'|'sm', children?(activeId)` | Accessible tablist (roving focus, arrows/Home/End). |
54 +| `Callout` | `type='info'|'tip'|'warning'|'danger', title?` | Aside with icon; `role=alert` for warning/danger. |
55 +| `Table` | `columns:[{key,label,align?,render?,width?}], rows, keyField?, caption?, dense?` | Responsive wrapper `.table-wrap`. |
56 +| `Badge` / `MethodBadge` / `StatusBadge` | `tone='neutral'|'success'|'info'|'warn'|'danger'` / `method` / `code` | |
57 +| `LangSelector` | `value?, onChange?, size` | Radio group over `LANGS` (`curl, python, javascript, r`); `useLang()` → `[lang, setLang]`; wrap app in `LangProvider`. |
58 +| `Markdown` / `InlineMd` | `text` or children, `headingOffset?` | Safe Markdown → React (no innerHTML): headings, lists, fenced code, tables, blockquotes, inline code/bold/italic/links (unsafe schemes dropped; site links become router `Link`s). |
59 +| `SearchDialog` | `open, onClose` | ⌘K/Ctrl+K and `/` are bound in `Layout`; index built lazily from spec + guides. |
60 +| `Icons` | — | Stroke icons (`SunIcon`, `CopyIcon`, `PlayIcon`, …), 18px, currentColor. |
61 +
62 +CSS classes reusable anywhere: `.btn .btn-primary .btn-ghost .btn-sm .btn-lg`, `.card`, `.grid .grid-2/3/4`, `.eyebrow`,
63 +`.lead`, `.muted`, `.small`, `.mono`, `.num`, `.table-wrap`, `.sr-only`, `.skeleton` (home.css).
64 +
65 +## Docs engine
66 +
67 +**Reference** is generated at runtime from `/openapi.json`:
68 +
69 +1. `spec.js › loadSpec()` fetches once per session with `If-None-Match` (ETag), caches `{etag, spec}` in
70 + localStorage (`hfmd.openapi.v1`), and falls back to `content/openapi.snapshot.json` when the response is not JSON
71 + (offline, `vite preview`). `useSpec()` → `{ spec, loading, source: 'live'|'cache'|'snapshot' }`.
72 +2. `listOperations(spec)` flattens paths → `{ id (operationId), method, path, tag, summary, description, parameters
73 + (resolved, with type/default/enum/constraints), responses }`; `groupByTag` orders by `spec.tags`.
74 + FastAPI's `^(a|b)$` patterns are surfaced as enums.
75 +3. `enrich.js › exampleParams(op)` derives realistic values: spec `example` → `PATH_OVERRIDES["METHOD /path"]` →
76 + `NAME_DEFAULTS[paramName]` (AAPL, ES, ESZ25, CL, 2024-06-03…) → schema default/enum. `buildRequest(op, values)`
77 + → `{ url, rel, query, format }`; `playgroundLink(op, values)`.
78 +4. `snippets.js › snippetsFor({ method, url, format, auth, stream })` → `{ curl, python, javascript, r }`
79 + (requests+pandas, fetch, httr2; Parquet/CSV parsing lines vary with `format`; `$HFMD_API_KEY` env var, never a key).
80 +5. Response example: spec example (`responseExample`) → `content/examples/<operationId>.json` (captured at build by
81 + `npm run examples`) → live fetch for public GET operations (cached in memory). Errors: `errorResponses()` parses the
82 + `\`CODE\`` markers that `api/openapi.py` writes into 4xx/5xx descriptions; every code links to `/docs/errors#<code>`.
83 +6. Routes: `/docs/reference` (index), `/docs/reference/tag/<tag>`, `/docs/reference/<operationId>`, `/docs/errors`.
84 +
85 +**Guides** are MDX in `content/guides/`. The `virtual:guides-index` Vite plugin (in `vite.config.js`) extracts `##`/`###`
86 +headings of every guide at build time → search index + "On this page" outline without loading the guide chunks.
87 +Heading ids use the same `slugify` in the plugin and in `mdx-components.jsx`.
88 +
89 +### Add a guide
90 +
91 +1. Create `content/guides/<file>.mdx`. Optional exports:
92 + ```js
93 + export const meta = { title, description }
94 + export const snippets = [{ title, curl, python, javascript, r }] // → right-hand code panel
95 + ```
96 + Components available without import: `Callout`, `CodeTabs`, `Tabs`, `Table`, `Badge`, `MethodBadge`, `Code`,
97 + `Endpoint method path id?` (chip linking to the reference), `TryIt ep params`, `Url path`, `Figure caption`, `Link`.
98 + MDX caveats: no bare `<` or `{` in prose (use code spans), SVG attributes in camelCase.
99 +2. Register it in `src/docs/guides.js › GUIDES` (`slug`, `title`, `section`, `file`, `summary`). Sections:
100 + `start · guides · recipes · fundamentals · more`. Nav, search, prev/next and the sitemap pick it up automatically.
101 +
102 +### Errors page
103 +
104 +`/docs/errors` lists every code of `components.schemas.Error.properties.code.enum` (plus curated meaning/fix in
105 +`ErrorsPage.jsx › ERROR_INFO`) with anchors `#<code_lower>` — the API's `error.docs` URLs point there.
106 +
107 +## Playground deep-link contract (`/playground?ep=…`)
108 +
109 +Every reference page's **Try it** button and the MDX `<TryIt>` component produce:
110 +
111 +```
112 +/playground?ep=<operationId>&<paramName>=<value>&<paramName>=<value>…
113 +```
114 +
115 +- `ep` — the OpenAPI `operationId` (e.g. `bars_v1_bars__asset___ticker__get`). The playground should resolve it via
116 + `listOperations(spec)` from `src/docs/spec.js` and select that endpoint.
117 +- Every other query key is a **parameter name of that operation** (path or query), URL-encoded, already validated
118 + against the spec's enums/patterns by construction. Unknown keys should be ignored.
119 +- `format` may be present (`json|csv|parquet`); `api_key` is never included.
120 +- Helpers to reuse: `exampleParams(op)`, `buildRequest(op, values)`, `snippetsFor(...)`, `rateHeaders(res)` (`api.js`).
121 +
122 +## Prerender (LCP)
123 +
124 +`npm run build` runs `vite build` (with `build.manifest`) then `scripts/prerender.mjs`, which writes static shells for
125 +`/`, `/limits`, `/docs`, `/docs/quickstart`, `/status` — as both `dist/<route>/index.html` and `dist/<route>.html` so
126 +every static server finds them (`/` in place) — plus `dist/sitemap.xml`:
127 +
128 +- route-specific `<title>`, description, OpenGraph, canonical;
129 +- `<link rel="modulepreload">` for the route's whole chunk graph and `<link rel="stylesheet">` for its CSS — the
130 + lazy-route waterfall disappears, so the first paint of the real page is one round-trip after HTML;
131 +- above-the-fold markup (header, h1, lead) inside `#root`, styled by the already-linked CSS, so LCP happens before
132 + React mounts. `createRoot().render()` then swaps it for the live tree.
133 +
134 +Trade-off: this is *not* SSR — dynamic content (counters, spec-driven reference) is not in the HTML. Full SSR would
135 +require a second SSR bundle and a Node render per route around lazy routes, MDX and browser-only libraries
136 +(lightweight-charts), for pages whose content is mostly fetched live anyway. The shell approach costs ~60 lines and no
137 +runtime dependency. **Backend note**: the FastAPI SPA fallback serves `dist/index.html` for directory paths; to serve the
138 +shells it should try, in order, `candidate`, `candidate.with_suffix(".html")` and `candidate / "index.html"` before
139 +falling back to `index.html`.
140 +
141 +## Scripts
142 +
143 +| Script | Purpose |
144 +| --- | --- |
145 +| `scripts/fetch-examples.mjs` | For each public GET operation without a spec example, call production with realistic params and store `content/examples/<operationId>.json` (`{ url, params, status, headers, captured_at, body }`, `data` trimmed to 5 rows). Refreshes `content/openapi.snapshot.json` when the live spec has the `Error` schema. Flags: `--base`, `--only a,b`, `--force`. |
146 +| `scripts/make-aapl-pe-sample.mjs` | Builds `content/samples/aapl-pe.json` (weekly AAPL split-adjusted close ÷ 10-K diluted EPS, labelled as such) — homepage chart fallback until `/v1/fundamentals/AAPL/ratios/daily` is live. |
147 +| `scripts/bench-formats.mjs` + `bench-parquet.py` | Real JSON/CSV/Parquet size measurements used in the Data formats guide. |
148 +| `scripts/prerender.mjs` | See above. |
149 +
150 +## E2E
151 +
152 +`playwright.config.js` starts `vite preview` on :4173 (Chromium desktop + Pixel 7 for `mobile.spec.js`). Without the
153 +API, docs render from the bundled snapshot and the homepage/status pages exercise their fallbacks — which is exactly
154 +what the tests assert (docs navigation, ⌘K search with keyboard, Try-it link format, limits content and CTAs, theme
155 +toggle persistence, prerendered shells, mobile drawer/docs nav). Browsers: `npx playwright install chromium`.
added hfmarketdata/web/e2e/docs.spec.js +121 −0
@@ -0,0 +1,121 @@
1 +// Docs: navigation, generated reference, Try-it contract, search dialog, errors anchors, language persistence.
2 +import { expect, test } from '@playwright/test'
3 +
4 +test.describe('docs', () => {
5 + test('overview renders nav and quickstart guide loads with code panel', async ({ page }) => {
6 + await page.goto('/docs')
7 + await expect(page.getByTestId('docs-home')).toBeVisible()
8 + await expect(page.getByRole('navigation', { name: 'Documentation' })).toBeVisible()
9 + await page.getByRole('navigation', { name: 'Documentation' }).getByRole('link', { name: 'Quickstart' }).click()
10 + await expect(page).toHaveURL(/\/docs\/quickstart$/)
11 + await expect(page.getByRole('heading', { level: 1, name: 'Quickstart' })).toBeVisible()
12 + await expect(page.getByTestId('guide-page')).toBeVisible()
13 + // right panel: code tabs with the 4 languages
14 + const panel = page.getByRole('complementary', { name: 'Code and outline' })
15 + await expect(panel.getByRole('radio', { name: 'Python' }).first()).toBeVisible()
16 + await expect(panel.getByRole('radio', { name: 'R', exact: true }).first()).toBeVisible()
17 + await expect(page).toHaveTitle(/Quickstart/)
18 + })
19 +
20 + test('language choice persists across pages and reloads', async ({ page }) => {
21 + await page.goto('/docs/quickstart')
22 + const panel = page.getByRole('complementary', { name: 'Code and outline' })
23 + await panel.getByRole('radio', { name: 'Python' }).first().click()
24 + await expect(panel.getByRole('radio', { name: 'Python' }).first()).toHaveAttribute('aria-checked', 'true')
25 + await expect(panel.locator('.code').first()).toContainText('import requests')
26 + await page.goto('/docs/rate-limits')
27 + await expect(page.getByRole('complementary', { name: 'Code and outline' }).getByRole('radio', { name: 'Python' }).first()).toHaveAttribute('aria-checked', 'true')
28 + expect(await page.evaluate(() => localStorage.getItem('hfmd.lang'))).toBe('"python"')
29 + })
30 +
31 + test('reference index groups operations by tag (bundled spec fallback)', async ({ page }) => {
32 + await page.goto('/docs/reference')
33 + await expect(page.getByTestId('reference-index')).toBeVisible()
34 + await expect(page.getByRole('heading', { level: 2, name: /Bars/ })).toBeVisible()
35 + await expect(page.getByRole('heading', { level: 2, name: /Options/ })).toBeVisible()
36 + await page.getByRole('link', { name: /\/v1\/bars\/\{asset\}\/\{ticker\}/ }).click()
37 + await expect(page).toHaveURL(/\/docs\/reference\/bars_v1_bars__asset___ticker__get$/)
38 + })
39 +
40 + test('operation page: parameters, samples, response example and Try-it deep link', async ({ page }) => {
41 + await page.goto('/docs/reference/bars_v1_bars__asset___ticker__get')
42 + await expect(page.getByTestId('operation-page')).toBeVisible()
43 + await expect(page.getByRole('heading', { level: 1 })).toHaveText(/Bars/)
44 + await expect(page.locator('.op-path')).toContainText('/v1/bars/{asset}/{ticker}')
45 + // parameters table lists the documented params
46 + for (const p of ['asset', 'ticker', 'timeframe', 'limit', 'format']) await expect(page.locator(`#param-${p}`)).toBeVisible()
47 + // curl sample uses the production URL with realistic values, no key
48 + const panel = page.getByRole('complementary', { name: 'Code and outline' })
49 + await panel.getByRole('radio', { name: 'curl' }).first().click()
50 + await expect(panel.locator('.code').first()).toContainText('https://www.hfmarketdata.io/v1/bars/stock/AAPL?timeframe=1day')
51 + await expect(panel.locator('.code').first()).not.toContainText('hfmd_live_')
52 + // captured response example present (content/examples)
53 + await expect(panel.locator('.code').nth(1)).toContainText('"ticker": "AAPL"')
54 + // Try it → /playground?ep=<operationId>&<params>
55 + const href = await page.getByTestId('try-it').getAttribute('href')
56 + expect(href).toMatch(/^\/playground\?ep=bars_v1_bars__asset___ticker__get&/)
57 + const q = new URLSearchParams(href.split('?')[1])
58 + expect(q.get('asset')).toBe('stock')
59 + expect(q.get('ticker')).toBe('AAPL')
60 + expect(q.get('timeframe')).toBe('1day')
61 + // errors section links to anchors on the errors page
62 + await expect(page.locator('#errors')).toBeVisible()
63 + await expect(page.getByRole('link', { name: 'RATE_LIMIT_EXCEEDED' }).first()).toHaveAttribute('href', '/docs/errors#rate_limit_exceeded')
64 + })
65 +
66 + test('errors page has an anchor per code', async ({ page }) => {
67 + await page.goto('/docs/errors#rate_limit_exceeded')
68 + await expect(page.getByTestId('errors-page')).toBeVisible()
69 + for (const id of ['invalid_parameter', 'contract_not_found', 'rate_limit_exceeded', 'row_limit_exceeded', 'invalid_api_key']) {
70 + await expect(page.locator(`#${id}`)).toBeAttached()
71 + }
72 + await expect(page.locator('#rate_limit_exceeded')).toContainText('RATE_LIMIT_EXCEEDED')
73 + })
74 +
75 + test('Cmd+K search: open, type, keyboard navigate, Enter opens the page, Esc closes', async ({ page }) => {
76 + await page.goto('/docs')
77 + await page.keyboard.press('ControlOrMeta+k')
78 + const dialog = page.getByTestId('search-dialog')
79 + await expect(dialog).toBeVisible()
80 + await expect(dialog.getByRole('combobox')).toBeFocused()
81 + await page.keyboard.type('futures contracts')
82 + await expect(dialog.getByRole('option').first()).toBeVisible()
83 + await expect(dialog.getByRole('option').first()).toContainText(/Futures/i)
84 + await page.keyboard.press('ArrowDown')
85 + await page.keyboard.press('ArrowUp')
86 + await page.keyboard.press('Enter')
87 + await expect(dialog).toBeHidden()
88 + await expect(page).toHaveURL(/\/docs\/futures-contracts/)
89 + // slash shortcut + Escape
90 + await page.keyboard.press('/')
91 + await expect(page.getByTestId('search-dialog')).toBeVisible()
92 + await expect(page.getByTestId('search-dialog').getByRole('combobox')).toBeFocused()
93 + await page.keyboard.type('opt chain')
94 + await expect(page.getByTestId('search-dialog').getByRole('option').first()).toContainText(/Opt Chain|chain/i)
95 + await page.keyboard.press('Escape')
96 + await expect(page.getByTestId('search-dialog')).toBeHidden()
97 + // trigger button also opens it
98 + await page.getByTestId('search-trigger').click()
99 + await expect(page.getByTestId('search-dialog')).toBeVisible()
100 + })
101 +
102 + test('guides render MDX components (callouts, tables, endpoint chips) and headings have anchors', async ({ page }) => {
103 + await page.goto('/docs/futures-contracts#roll-methods-roll')
104 + await expect(page.getByRole('heading', { level: 1, name: /Futures: individual contracts/ })).toBeVisible()
105 + await expect(page.locator('#roll-methods-roll')).toBeAttached()
106 + await expect(page.locator('.callout').first()).toBeVisible()
107 + await expect(page.locator('.figure svg').first()).toBeVisible()
108 + await expect(page.locator('.docs-content table').first()).toBeVisible()
109 + // On this page outline lists the headings
110 + await expect(page.getByRole('navigation', { name: 'On this page' }).getByRole('link', { name: /Adjustment methods/ })).toBeVisible()
111 + // Edit / report link is a mailto with the page URL
112 + const mail = await page.getByRole('link', { name: /Edit \/ Report an issue/ }).getAttribute('href')
113 + expect(mail).toMatch(/^mailto:contact@spboucher\.ai\?subject=/)
114 + expect(decodeURIComponent(mail)).toContain('/docs/futures-contracts')
115 + })
116 +
117 + test('changelog has the v2.0.0 entry dated 2026-09-04', async ({ page }) => {
118 + await page.goto('/docs/changelog')
119 + await expect(page.getByRole('heading', { level: 2, name: /2\.0\.0 — 2026-09-04/ })).toBeVisible()
120 + })
121 +})
added hfmarketdata/web/e2e/mobile.spec.js +37 −0
@@ -0,0 +1,37 @@
1 +// Mobile (Pixel 7): drawer navigation, docs nav toggle, code panel below content (DOM order = visual order).
2 +import { expect, test } from '@playwright/test'
3 +
4 +test.describe('mobile', () => {
5 + test('drawer opens from the menu button and navigates', async ({ page }) => {
6 + await page.goto('/')
7 + await expect(page.getByRole('navigation', { name: 'Main' })).toBeHidden()
8 + await page.getByRole('button', { name: 'Open menu' }).click()
9 + const drawer = page.getByRole('navigation', { name: 'Mobile' })
10 + await expect(drawer).toBeVisible()
11 + await drawer.getByRole('link', { name: 'Limits' }).click()
12 + await expect(page).toHaveURL(/\/limits$/)
13 + await expect(drawer).toBeHidden()
14 + })
15 +
16 + test('docs: nav toggle, code panel after content', async ({ page }) => {
17 + await page.goto('/docs/quickstart')
18 + await expect(page.getByRole('navigation', { name: 'Documentation' })).toBeHidden()
19 + await page.getByRole('button', { name: /Docs menu/ }).click()
20 + await expect(page.getByRole('navigation', { name: 'Documentation' })).toBeVisible()
21 + await page.getByRole('navigation', { name: 'Documentation' }).getByRole('link', { name: 'Rate limits' }).click()
22 + await expect(page).toHaveURL(/\/docs\/rate-limits$/)
23 + const content = page.locator('.docs-content')
24 + const panel = page.getByRole('complementary', { name: 'Code and outline' })
25 + const [c, p] = await Promise.all([content.boundingBox(), panel.boundingBox()])
26 + expect(p.y).toBeGreaterThan(c.y) // panel rendered below the article on small screens
27 + })
28 +
29 + test('search dialog usable on touch viewport', async ({ page }) => {
30 + await page.goto('/docs')
31 + await page.getByTestId('search-trigger').click()
32 + await expect(page.getByTestId('search-dialog')).toBeVisible()
33 + await page.keyboard.type('errors')
34 + await page.getByTestId('search-dialog').getByRole('option').first().click()
35 + await expect(page).toHaveURL(/\/docs\/errors/)
36 + })
37 +})
added hfmarketdata/web/e2e/site.spec.js +91 −0
@@ -0,0 +1,91 @@
1 +// Site shell: homepage, limits page content (free platform), theme toggle persistence, status page, prerendered shells.
2 +import { expect, test } from '@playwright/test'
3 +
4 +test.describe('site', () => {
5 + test('homepage hero, nav and key sections', async ({ page }) => {
6 + await page.goto('/')
7 + await expect(page.getByRole('heading', { level: 1 })).toContainText('Open high-frequency market data')
8 + const nav = page.getByRole('navigation', { name: 'Main' })
9 + for (const label of ['Docs', 'Playground', 'Integrations', 'Limits', 'Status']) await expect(nav.getByRole('link', { name: label })).toBeVisible()
10 + await expect(nav.getByRole('link', { name: /Pricing/ })).toHaveCount(0)
11 + await expect(page.getByRole('heading', { name: /Get an API key/ })).toBeVisible()
12 + await expect(page.getByRole('heading', { name: /A real request, a real response/ })).toBeVisible()
13 + await expect(page.getByRole('heading', { name: /SEC EDGAR, point-in-time/ })).toBeVisible()
14 + await expect(page.getByTestId('pe-chart')).toBeVisible()
15 + // preview falls back to the captured ES response when the API is not reachable
16 + await expect(page.locator('.preview .code').nth(1)).toContainText('"ticker": "ES"')
17 + await expect(page.getByRole('link', { name: 'Request higher limits' })).toHaveAttribute('href', /^mailto:contact@spboucher\.ai/)
18 + })
19 +
20 + test('limits page: three levels with all numbers, CTAs, no paid wording', async ({ page }) => {
21 + // /pricing is the route App.jsx currently serves; it becomes a redirect to /limits (web-app coordinator), so
22 + // navigating here works before and after that change. Switch to '/limits' once the redirect is in.
23 + await page.goto('/pricing')
24 + await expect(page.getByTestId('limits-page')).toBeVisible()
25 + await expect(page.getByRole('heading', { level: 1 })).toHaveText('Free for everyone.')
26 + const tiers = page.getByTestId('tiers')
27 + for (const n of ['30', '100,000', '5,000', '120', '1,000,000', '50,000', '600', '10,000,000', '200,000']) {
28 + await expect(tiers.getByText(n, { exact: true }).first()).toBeVisible()
29 + }
30 + await expect(page.getByTestId('tier-keyless')).toContainText('per hour')
31 + await expect(page.getByTestId('tier-free')).toContainText('per minute')
32 + await expect(page.getByTestId('cta-signup')).toHaveAttribute('href', '/signup')
33 + const mail = await page.getByTestId('cta-high-usage').getAttribute('href')
34 + expect(mail).toMatch(/^mailto:contact@spboucher\.ai\?subject=/)
35 + // incentives + FAQ
36 + await expect(page.getByText('Parquet responses count half', { exact: true })).toBeVisible()
37 + await page.getByText('What happens on 429?').click()
38 + await expect(page.getByText(/RATE_LIMIT_EXCEEDED/).first()).toBeVisible()
39 + const text = (await page.locator('main').innerText()).toLowerCase()
40 + for (const banned of ['pricing', 'paid', 'per month', '$/']) expect(text).not.toContain(banned)
41 + await expect(page).toHaveTitle(/Access & limits — free for everyone/)
42 + })
43 +
44 + test('theme toggle switches and persists across reloads', async ({ page }) => {
45 + await page.goto('/')
46 + const html = page.locator('html')
47 + await expect(html).toHaveAttribute('data-theme', 'dark')
48 + await page.getByTestId('theme-toggle').click()
49 + await expect(html).toHaveAttribute('data-theme', 'light')
50 + expect(await page.evaluate(() => localStorage.getItem('hfmd.theme'))).toBe('light')
51 + await page.reload()
52 + await expect(html).toHaveAttribute('data-theme', 'light')
53 + // applied before React mounts (inline script in index.html)
54 + const early = await page.evaluate(() => document.documentElement.getAttribute('data-theme'))
55 + expect(early).toBe('light')
56 + await page.getByTestId('theme-toggle').click()
57 + await expect(html).toHaveAttribute('data-theme', 'dark')
58 + })
59 +
60 + test('status page shows health cards and handles an unreachable API', async ({ page }) => {
61 + await page.goto('/status')
62 + await expect(page.getByTestId('status-page')).toBeVisible()
63 + await expect(page.getByRole('heading', { level: 1 })).toContainText('API health')
64 + await expect(page.locator('.health-card')).toHaveCount(4)
65 + // vite preview has no API → cards settle to Down, warning callout shown, page still usable
66 + await expect(page.locator('.health-card[data-state="down"]').first()).toBeVisible({ timeout: 15_000 })
67 + await expect(page.getByRole('alert')).toContainText('did not answer')
68 + })
69 +
70 + test('prerendered shells carry route meta and preload the route chunk', async ({ request }) => {
71 + const limits = await (await request.get('/limits')).text()
72 + expect(limits).toContain('<title>Access &amp; limits — free for everyone · HF Market Data</title>')
73 + expect(limits).toContain('<link rel="canonical" href="https://www.hfmarketdata.io/limits" />')
74 + expect(limits).toMatch(/<link rel="modulepreload" crossorigin href="\/assets\/(Limits|Pricing)-[^"]+\.js">/)
75 + expect(limits).toContain('<h1>Free for everyone.</h1>')
76 + const home = await (await request.get('/')).text()
77 + expect(home).toMatch(/<link rel="modulepreload" crossorigin href="\/assets\/Home-[^"]+\.js">/)
78 + expect(home).not.toContain('fonts.googleapis.com')
79 + const sitemap = await (await request.get('/sitemap.xml')).text()
80 + expect(sitemap).toContain('<loc>https://www.hfmarketdata.io/docs/quickstart</loc>')
81 + })
82 +
83 + test('accessibility basics: skip link, landmarks, focus-visible on nav', async ({ page }) => {
84 + await page.goto('/limits')
85 + await page.keyboard.press('Tab')
86 + await expect(page.getByRole('link', { name: 'Skip to content' })).toBeFocused()
87 + await expect(page.getByRole('banner')).toBeVisible()
88 + await expect(page.getByRole('contentinfo')).toBeVisible()
89 + await expect(page.getByRole('main')).toBeVisible()
90 + })
91 +})
added hfmarketdata/web/playwright.config.js +27 −0
@@ -0,0 +1,27 @@
1 +// Playwright E2E against the production build served by `vite preview` (no API: docs fall back to the bundled spec).
2 +// Run: npm run build && npm run test:e2e (or `npx playwright test --ui`)
3 +import { defineConfig, devices } from '@playwright/test'
4 +
5 +const PORT = process.env.E2E_PORT || 4173
6 +
7 +export default defineConfig({
8 + testDir: './e2e',
9 + timeout: 30_000,
10 + retries: process.env.CI ? 1 : 0,
11 + reporter: process.env.CI ? 'github' : 'list',
12 + use: {
13 + baseURL: `http://localhost:${PORT}`,
14 + trace: 'retain-on-failure',
15 + colorScheme: 'dark',
16 + },
17 + projects: [
18 + { name: 'chromium', use: { ...devices['Desktop Chrome'] }, testIgnore: /mobile\.spec\.js/ },
19 + { name: 'mobile', use: { ...devices['Pixel 7'] }, testMatch: /mobile\.spec\.js/ },
20 + ],
21 + webServer: {
22 + command: `npx vite preview --port ${PORT} --strictPort`,
23 + url: `http://localhost:${PORT}/`,
24 + reuseExistingServer: !process.env.CI,
25 + timeout: 60_000,
26 + },
27 +})
added hfmarketdata/web/scripts/prerender.mjs +99 −0
@@ -0,0 +1,99 @@
1 +#!/usr/bin/env node
2 +// Post-build prerender of static HTML shells for the public entry routes.
3 +//
4 +// Trade-off (see README "Prerender"): a full SSR of the React tree would need a second SSR build + a Node render step
5 +// for every route (lazy routes, MDX, lightweight-charts, browser-only APIs), for pages that are mostly fetched live.
6 +// Instead we emit, per route, a copy of dist/index.html with: route-specific <title>/<meta>/<link rel=canonical>,
7 +// <link rel=modulepreload> for the route's chunk graph and <link rel=stylesheet> for its CSS (kills the lazy-route
8 +// waterfall), and the above-the-fold content (header + h1 + lead) as static markup inside #root so first paint is
9 +// meaningful before React mounts. React's createRoot() then replaces it with the live tree.
10 +//
11 +// Serving: vite preview / any static host resolves /limits → dist/limits/index.html. The FastAPI SPA fallback currently
12 +// serves index.html for directories; it needs a one-line `candidate / "index.html"` check to pick these up.
13 +import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'
14 +import { dirname, join, resolve } from 'node:path'
15 +import { fileURLToPath } from 'node:url'
16 +
17 +const ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..')
18 +const DIST = join(ROOT, 'dist')
19 +const SITE = 'https://www.hfmarketdata.io'
20 +
21 +const NAV = [['/docs', 'Docs'], ['/playground', 'Playground'], ['/integrations', 'Integrations'], ['/limits', 'Limits'], ['/status', 'Status']]
22 +
23 +const ROUTES = [
24 + { path: '/', entry: 'src/pages/home/Home.jsx', title: 'HF Market Data — Open High-Frequency Market Data API',
25 + description: 'Free open API for high-frequency historical market data: stocks, ETFs, futures (continuous and individual contracts), crypto, indices, FX from 1-minute to daily, options chains with Greeks and SEC EDGAR fundamentals since 2010.',
26 + h1: 'Open high-frequency market data. 1-minute to daily, since 2010.',
27 + lead: 'Stocks, ETFs, futures — continuous series and every individual contract — crypto, indices and FX, plus full options chains with Greeks and point-in-time SEC fundamentals. One base URL, JSON / CSV / Parquet, generated docs, a playground, and limits that only grow.', cls: 'hero' },
28 + // entry falls back to the temporary src/pages/pricing shim until App.jsx imports Limits.jsx directly
29 + { path: '/limits', entry: ['src/pages/limits/Limits.jsx', 'src/pages/pricing/Pricing.jsx'], title: 'Access & limits — free for everyone · HF Market Data',
30 + description: 'HF Market Data is free. Keyless access is very limited, a free account gives much higher limits, and higher limits are granted on request by e-mail.',
31 + h1: 'Free for everyone.', lead: 'There is nothing to buy. Keyless access is deliberately small so you can try things without signing up; a free account raises the limits a lot; and if you need more, you simply ask.' },
32 + { path: '/docs', entry: 'src/pages/docs/Docs.jsx', title: 'Documentation · HF Market Data',
33 + description: 'HF Market Data documentation: quickstart, authentication, rate limits, futures contracts, options, fundamentals, and the full API reference generated from OpenAPI.',
34 + h1: 'Documentation', lead: 'Historical market data from 1-minute to daily since 2010 — stocks, ETFs, futures, crypto, indices, FX — plus options chains with Greeks and SEC EDGAR fundamentals. One base URL, JSON / CSV / Parquet, no signup to start.' },
35 + { path: '/docs/quickstart', entry: 'src/pages/docs/Docs.jsx', title: 'Quickstart · HF Market Data',
36 + description: 'Your first HF Market Data request in 30 seconds — no signup, no key.', h1: 'Quickstart',
37 + lead: 'You can pull real data right now, with nothing but a URL. No account, no key, no SDK.' },
38 + { path: '/status', entry: 'src/pages/status/Status.jsx', title: 'Status · HF Market Data',
39 + description: 'Live health and dataset inventory of the HF Market Data API.', h1: 'API health & dataset inventory',
40 + lead: 'Everything on this page is fetched live from the API when you open it — nothing is cached or hand-maintained.' },
41 +]
42 +
43 +const esc = s => s.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/"/g, '&quot;')
44 +
45 +function chunkGraph(manifest, entry) {
46 + const files = new Set(), css = new Set()
47 + const walk = key => {
48 + const m = manifest[key]
49 + if (!m || files.has(m.file)) return
50 + files.add(m.file)
51 + for (const c of m.css || []) css.add(c)
52 + for (const i of m.imports || []) walk(i)
53 + }
54 + const entries = Array.isArray(entry) ? entry : [entry]
55 + walk(entries.find(e => manifest[e]) || entries[0])
56 + return { files: [...files], css: [...css] }
57 +}
58 +
59 +function shell(route) {
60 + const nav = NAV.map(([to, label]) => `<a href="${to}"${route.path.startsWith(to) ? ' class="active"' : ''}>${label}</a>`).join('')
61 + return `<div class="shell"><header class="topbar"><a class="brand" href="/"><span class="brand-mark">HF</span><span class="brand-name">Market Data</span></a><nav class="topnav" aria-label="Main">${nav}</nav></header>` +
62 + `<main class="${route.cls === 'hero' ? 'home' : 'page'}"><section class="${route.cls === 'hero' ? 'hero' : 'limits-head'}" style="padding-top:${route.cls === 'hero' ? '72px' : '0'}"><div class="hero-inner"><h1>${esc(route.h1)}</h1><p class="lead">${esc(route.lead)}</p></div></section></main></div>`
63 +}
64 +
65 +function main() {
66 + const indexPath = join(DIST, 'index.html')
67 + if (!existsSync(indexPath)) throw new Error('dist/index.html missing — run vite build first')
68 + const base = readFileSync(indexPath, 'utf8')
69 + const manifestPath = join(DIST, '.vite', 'manifest.json')
70 + const manifest = existsSync(manifestPath) ? JSON.parse(readFileSync(manifestPath, 'utf8')) : {}
71 + let count = 0
72 + for (const route of ROUTES) {
73 + const { files, css } = chunkGraph(manifest, route.entry)
74 + const alreadyLinked = f => base.includes(`/${f}`)
75 + const preload = files.filter(f => !alreadyLinked(f)).map(f => `<link rel="modulepreload" crossorigin href="/${f}">`).join('')
76 + const styles = css.filter(f => !alreadyLinked(f)).map(f => `<link rel="stylesheet" href="/${f}">`).join('')
77 + let html = base
78 + .replace(/<title>[^<]*<\/title>/, `<title>${esc(route.title)}</title>`)
79 + .replace(/<meta name="description" content="[^"]*"\s*\/?>/, `<meta name="description" content="${esc(route.description)}" />`)
80 + .replace(/<meta property="og:title" content="[^"]*"\s*\/?>/, `<meta property="og:title" content="${esc(route.title)}" />`)
81 + .replace(/<meta property="og:description" content="[^"]*"\s*\/?>/, `<meta property="og:description" content="${esc(route.description)}" />`)
82 + .replace(/<link rel="canonical" href="[^"]*"\s*\/?>/, `<link rel="canonical" href="${SITE}${route.path === '/' ? '/' : route.path}" />`)
83 + .replace('</head>', `${styles}${preload}<meta property="og:url" content="${SITE}${route.path}" /></head>`)
84 + .replace('<div id="root"></div>', `<div id="root">${shell(route)}</div>`)
85 + // Two file forms so any static server picks the shell up: <route>/index.html (nginx try_files, Netlify, S3)
86 + // and <route>.html (vite preview's html fallback, GitHub Pages-style hosts).
87 + const outs = route.path === '/' ? [indexPath] : [join(DIST, route.path.replace(/^\//, ''), 'index.html'), join(DIST, `${route.path.replace(/^\//, '')}.html`)]
88 + for (const out of outs) { mkdirSync(dirname(out), { recursive: true }); writeFileSync(out, html) }
89 + count++
90 + console.log(`prerendered ${route.path} → ${outs.map(o => o.replace(DIST + '/', '')).join(', ')} (${files.length} chunks preloaded, ${css.length} css)`)
91 + }
92 + // sitemap for the static routes + guides known at build time
93 + const guides = Object.keys(manifest).filter(k => k.startsWith('content/guides/')).map(k => k.replace('content/guides/', '').replace(/\.mdx$/, ''))
94 + const urls = [...ROUTES.map(r => r.path), '/docs/reference', '/docs/errors', '/docs/changelog', ...guides.map(g => `/docs/${g.replace(/^recipe-/, 'recipes/').replace(/^fundamentals-/, 'fundamentals/')}`)]
95 + writeFileSync(join(DIST, 'sitemap.xml'), `<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${[...new Set(urls)].map(u => ` <url><loc>${SITE}${u}</loc></url>`).join('\n')}\n</urlset>\n`)
96 + console.log(`${count} shells + sitemap.xml (${urls.length} urls)`)
97 +}
98 +
99 +main()
100