SPB Git forge
spboucher.ai ↗

spb/hfmarketdata

Public

Open high-frequency market data platform — FirstRate full-history downloader, DuckDB/Parquet lake, open REST API and React docs platform (www.hfmarketdata.io)

127commits 1branches 0releases
24.7 MBsize
maindefault branch
11 days agolast push
JavaScript 53.7% Python 38.3% CSS 4.6% TypeScript 3.1%

ratelimit: principal `site` — les requêtes de données du site (/charts) sont hors quota (3 critères cumulés X-HFMD-Client + Sec-Fetch-Site + Origin), max_rows 200 000, garde anti-abus 1 200 req/min/IP, stats sous `site`

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Simon-Pierre Boucher committed 18 days ago (Sep 7, 2026) parent 6525eb7

4 changed files +336 −7

modified docs/accounts-ratelimit.md +32 −0
@@ -59,6 +59,38 @@ Other quota-related errors: `400 ROW_LIMIT_EXCEEDED` when `limit` exceeds the ti
59 59 (`details.max_rows`), `401 INVALID_API_KEY` (unknown/revoked key — the request is NOT downgraded to keyless),
60 60 `403 ACCOUNT_DISABLED`, `401 AUTH_REQUIRED`.
61 61
62 +### The site's own requests (`/charts`) have no quota — principal `site`
63 +
64 +The charts page must work without an account and **without any limit**. The middleware recognises the
65 +site's own data requests and takes them out of the quota system entirely (`ratelimit/middleware.py`,
66 +`is_site_request`). A request is the site's own when **all three** hold, on a **GET** of a charts data path
67 +(`/v1/bars/*`, `/v1/futures/*`, `/v1/{asset}/tickers`, `/v1/options/*`):
68 +
69 +1. `X-HFMD-Client: charts` (or `web`) — set by the page's data layer (`web/src/charts/data/*.js`);
70 +2. `Sec-Fetch-Site: same-origin` (or `same-site`) — set by the browser, never by a `fetch()` from another origin;
71 +3. `Origin` (or `Referer`) whose host is the host of `HFMD_PUBLIC_URL` (with/without `www.`); `localhost`,
72 + `127.0.0.1`, `::1` are also accepted when `HFMD_ENV != production`.
73 +
74 +A Bearer key / `?api_key=` always wins (the caller asked for its own principal). Anything short of the three
75 +criteria — a `curl`, a third-party site, a missing header — is a regular keyless / key / session request.
76 +
77 +What changes for a `site` request:
78 +
79 +* **not counted, not row-charged, never 429 for quota reasons**; no `X-RateLimit-*` header at all;
80 +* `request.state.principal = "site"`, `principal_kind = "site"`, `max_rows = 200 000`
81 + (`tiers.SITE_MAX_ROWS`; the page asks for 5 000–20 000 bars at once; the JSON ceiling of the legacy bars
82 + endpoint, 50 000, still applies through `bound_limit`);
83 +* usage is recorded under the `site` principal for internal statistics (requests, rows, bytes) — **never under
84 + the visitor's IP**, whose keyless counters stay untouched;
85 +* the only guard is a **DoS protection**, not a product limit: `tiers.SITE_BURST_PER_MINUTE_PER_IP` = 1 200
86 + requests per minute per IP on this path (Redis sliding window, fail-open). Above it the API answers
87 + `429 RATE_LIMIT_EXCEEDED` with a dedicated message ("an abuse guard, not a quota: the charts have no data
88 + limit"), `Retry-After`, `details.guard = "site_burst"` and no `X-RateLimit-*` header. A human on the page
89 + cannot reach it (a full 1-minute history load is ~10 requests).
90 +
91 +`site` is not a tier: it is absent from `/v1/limits` (which, not being a data path, still answers as keyless /
92 +key / session). Tests: `tests/test_ratelimit_site.py`.
93 +
62 94 ## 3. Authentication model
63 95
64 96 * **API key**: `Authorization: Bearer hfmd_live_…` or `?api_key=…` — for data endpoints and programmatic
modified hfmarketdata/api/ratelimit/middleware.py +103 −7
@@ -21,23 +21,33 @@ value is the N-th hop from the END (`HFMD_TRUSTED_PROXY_HOPS`, default 1), never
21 21 controls. The header is ignored altogether when the direct peer is not a loopback / private address.
22 22
23 23 A signed-in browser (session cookie, no Bearer) is the `user:<id>` principal with the account's tier — the
24 −site (charts, playground) gets account limits without pasting a key. Account endpoints (`/v1/limits`, `/v1/me/*`,
24 +site (playground) gets account limits without pasting a key. Account endpoints (`/v1/limits`, `/v1/me/*`,
25 25 `/v1/admin/*`) are not charged. `/health`, `/openapi.json`
26 26 and the SPA are exempt. Redis down → data endpoints fail open (no headers), the API never goes down because of
27 27 quotas.
28 28
29 +Site principal (`site`, kind "site") — the /charts page has NO quota. A data GET (`/v1/bars/*`, `/v1/futures/*`,
30 +`/v1/{asset}/tickers`, `/v1/options/*`) is the site's own when ALL of: `X-HFMD-Client: charts|web`,
31 +`Sec-Fetch-Site: same-origin|same-site`, and an `Origin` (or `Referer`) whose host is `settings.public_url`'s
32 +(localhost outside production). Such requests are not counted, not row-charged, never 429 for quota reasons and
33 +carry no `X-RateLimit-*` header; `request.state.max_rows` is `SITE_MAX_ROWS`; usage is recorded under the
34 +`site` principal (internal stats), never under the visitor's IP. The one guard is `SITE_BURST_PER_MINUTE_PER_IP`
35 +(DoS protection, fail-open, dedicated 429 message). Anything short of the three criteria is a regular request.
36 +
29 37 Author: Simon-Pierre Boucher <contact@spboucher.ai>
30 38 """
31 39 from __future__ import annotations
32 40
33 41 import ipaddress
34 42 import logging
43 +import re
35 44 import threading
36 45 import time
37 46 from collections import OrderedDict
38 47 from dataclasses import dataclass
39 48 from datetime import datetime, timezone
40 49 from typing import Any
50 +from urllib.parse import urlsplit
41 51
42 52 from fastapi import FastAPI
43 53 from starlette.datastructures import Headers, MutableHeaders, QueryParams
@@ -48,9 +58,11 @@ from core.config import settings
48 58 from core.errors import ApiError
49 59
50 60 from . import redis_limiter as rl
61 +from . import tiers as _tiers
51 62 from . import usage
52 63 from .tiers import (AUTH_DEFAULT_LIMIT, AUTH_FAIL_CLOSED, AUTH_FAIL_CLOSED_RETRY_S, AUTH_LIMITS, AUTH_WINDOW_S,
53 − KEY_LOOKUPS_PER_MINUTE_PER_IP, KEY_REQUIRED_TAGS, TIERS, Tier, row_cost, tier_for, upgrade_hint)
64 + KEY_LOOKUPS_PER_MINUTE_PER_IP, KEY_REQUIRED_TAGS, SITE_TIER, TIERS, Tier, row_cost, tier_for,
65 + upgrade_hint)
54 66
55 67 log = logging.getLogger("hfmarketdata.ratelimit")
56 68
@@ -62,6 +74,15 @@ UNCHARGED_PREFIXES = ("/v1/limits", "/v1/me", "/v1/admin")
62 74 AUTH_PREFIX = "/v1/auth/"
63 75 AUTH_UNTHROTTLED = ("/v1/auth/logout",)
64 76
77 +# Site principal (charts): the three cumulative criteria + the data paths they apply to.
78 +SITE_PRINCIPAL = "site"
79 +SITE_CLIENT_HEADER = "x-hfmd-client"
80 +SITE_CLIENTS = frozenset({"charts", "web"})
81 +SITE_FETCH_SITES = frozenset({"same-origin", "same-site"})
82 +SITE_DATA_PREFIXES = ("/v1/bars/", "/v1/futures/", "/v1/options/")
83 +SITE_TICKERS_RE = re.compile(r"^/v1/[a-z]+/tickers$")
84 +SITE_LOCAL_HOSTS = frozenset({"localhost", "127.0.0.1", "::1", "0.0.0.0"})
85 +
65 86
66 87 # ----------------------------------------------------------------------------------------------------
67 88 # Principal resolution
@@ -289,6 +310,62 @@ def resolve_principal(scope: dict) -> Principal:
289 310 return Principal(f"ip:{hash_ip(ip)}", "keyless", TIERS["keyless"])
290 311
291 312
313 +# ----------------------------------------------------------------------------------------------------
314 +# Site principal (the /charts page has no quota)
315 +# ----------------------------------------------------------------------------------------------------
316 +
317 +def _site_hosts() -> frozenset[str]:
318 + """Hosts accepted in Origin / Referer: the public host with and without `www.`, plus localhost outside production."""
319 + host = (urlsplit(settings.public_url).hostname or "").lower()
320 + hosts = {host} if host else set()
321 + if host.startswith("www."):
322 + hosts.add(host[4:])
323 + elif host:
324 + hosts.add("www." + host)
325 + if settings.env != "production":
326 + hosts |= SITE_LOCAL_HOSTS
327 + return frozenset(hosts)
328 +
329 +
330 +def is_site_data_path(path: str) -> bool:
331 + return path.startswith(SITE_DATA_PREFIXES) or bool(SITE_TICKERS_RE.match(path))
332 +
333 +
334 +def is_site_request(scope: dict, headers: Headers) -> bool:
335 + """True when a data GET carries the three cumulative proofs of being issued by the site itself.
336 +
337 + A Bearer key (or `?api_key=`) always wins: the caller explicitly asked for its own principal."""
338 + if scope.get("method", "GET").upper() != "GET" or not is_site_data_path(scope.get("path", "")):
339 + return False
340 + if headers.get(SITE_CLIENT_HEADER, "").strip().lower() not in SITE_CLIENTS:
341 + return False
342 + if headers.get("sec-fetch-site", "").strip().lower() not in SITE_FETCH_SITES:
343 + return False
344 + if extract_key(headers, QueryParams(scope.get("query_string", b""))):
345 + return False
346 + origin = headers.get("origin") or headers.get("referer") or ""
347 + try:
348 + host = (urlsplit(origin).hostname or "").lower()
349 + except ValueError:
350 + return False
351 + return bool(host) and host in _site_hosts()
352 +
353 +
354 +def site_guard(ip: str, *, now_s: int) -> ApiError | None:
355 + """Per-IP burst guard on the site path (DoS protection, not a product limit). Fail-open when Redis is down."""
356 + limit = _tiers.SITE_BURST_PER_MINUTE_PER_IP
357 + d = rl.throttle(f"site:ip:{hash_ip(ip)}", limit=limit, window_s=60)
358 + if d is None or d.allowed_requests:
359 + return None
360 + retry = max(1, d.reset_requests - now_s)
361 + return ApiError(429, "RATE_LIMIT_EXCEEDED",
362 + f"Too many chart requests from this address ({limit} per minute — an abuse guard, not a quota: "
363 + f"the charts have no data limit). Retry in {retry} s.",
364 + type="requests_per_minute", headers={"Retry-After": str(retry)},
365 + details={"limit": limit, "window_seconds": 60, "reset": d.reset_requests, "principal": SITE_PRINCIPAL,
366 + "guard": "site_burst"})
367 +
368 +
292 369 # ----------------------------------------------------------------------------------------------------
293 370 # Helpers
294 371 # ----------------------------------------------------------------------------------------------------
@@ -406,6 +483,21 @@ class RateLimitMiddleware:
406 483 ip = client_ip(scope, headers)
407 484 state["client_ip_hash"] = hash_ip(ip)
408 485
486 + # the site's own data requests (charts): no quota, no headers, burst guard only, accounted under `site`
487 + if is_site_request(scope, headers):
488 + err = site_guard(ip, now_s=now_s)
489 + if err is not None:
490 + usage.record(SITE_PRINCIPAL, requests=0, status=429)
491 + return await err.response()(scope, receive, send)
492 + principal = Principal(SITE_PRINCIPAL, "site", SITE_TIER)
493 + state.update(principal=principal.id, principal_kind=principal.kind, tier=SITE_TIER.name,
494 + max_rows=SITE_TIER.max_rows_per_request, api_key_id=None, user_id=None)
495 + if route_tags(scope) & KEY_REQUIRED_TAGS:
496 + return await _auth_required_error().response()(scope, receive, send)
497 + state["ratelimit"] = {"principal": principal.id, "kind": principal.kind, "tier": SITE_TIER.name, "exempt": True,
498 + "max_rows_per_request": SITE_TIER.max_rows_per_request}
499 + return await self._forward(scope, receive, send, principal, SITE_TIER, None, charged=False, account_as=SITE_PRINCIPAL)
500 +
409 501 try:
410 502 principal = resolve_principal(scope)
411 503 except ApiError as exc:
@@ -439,7 +531,8 @@ class RateLimitMiddleware:
439 531 return await self._forward(scope, receive, send, principal, tier, decision, charged=charged)
440 532
441 533 async def _forward(self, scope, receive, send, principal: Principal, tier: Tier, decision: rl.Decision | None,
442 − *, charged: bool, extra_headers: dict[str, str] | None = None):
534 + *, charged: bool, extra_headers: dict[str, str] | None = None, account_as: str | None = None):
535 + """`account_as`: record usage under that principal even though the request is not charged (site stats)."""
443 536 state: dict = scope["state"]
444 537 started = False
445 538 replaced = False
@@ -449,7 +542,7 @@ class RateLimitMiddleware:
449 542 nonlocal started, replaced, final
450 543 if message["type"] == "http.response.start":
451 544 started = True
452 − if principal.keyless and state.get("requires_key"):
545 + if (principal.keyless or principal.kind == "site") and state.get("requires_key"):
453 546 replaced = True
454 547 err = _auth_required_error()
455 548 for k, v in (final.headers() if final else {}).items():
@@ -475,13 +568,16 @@ class RateLimitMiddleware:
475 568 headers[k] = v
476 569 for k, v in (extra_headers or {}).items():
477 570 headers[k] = v
478 − if charged:
571 + if charged or account_as:
479 572 try:
480 573 size = int(headers.get("content-length") or 0)
481 574 except ValueError:
482 575 size = 0
483 − usage.record(principal.id, requests=1 + extra_req, rows=cost, rows_parquet=cost if is_parquet else 0,
484 − bytes_=size, status=status)
576 + if charged:
577 + usage.record(principal.id, requests=1 + extra_req, rows=cost, rows_parquet=cost if is_parquet else 0,
578 + bytes_=size, status=status)
579 + else: # site: internal statistics only (rows as served, nothing is charged)
580 + usage.record(account_as, requests=1, rows=rows if 200 <= status < 300 else 0, bytes_=size, status=status)
485 581 elif replaced:
486 582 return
487 583 await send(message)
modified hfmarketdata/api/ratelimit/tiers.py +10 −0
@@ -13,6 +13,11 @@ Cost rules:
13 13 * keyless principals cannot use endpoints tagged `stream` or `screener`, nor endpoints that set
14 14 `request.state.requires_key = True` (401 AUTH_REQUIRED).
15 15
16 +Site principal (`site`): data GET requests issued by the web site itself (the /charts page — see
17 +`middleware.is_site_request`) are NOT a tier: no request / row quota, no `X-RateLimit-*` headers, up to
18 +`SITE_MAX_ROWS` rows per request. The only guard is `SITE_BURST_PER_MINUTE_PER_IP`, a DoS protection invisible to a
19 +human user, not a product limit.
20 +
16 21 Author: Simon-Pierre Boucher <contact@spboucher.ai>
17 22 """
18 23 from __future__ import annotations
@@ -76,6 +81,11 @@ ACCOUNT_TIERS = ("free", "high_usage", "unlimited")
76 81 INTERNAL_TIERS = ("unlimited",)
77 82 DEFAULT_ACCOUNT_TIER = "free"
78 83
84 +# The site's own data requests (charts): not in TIERS on purpose — never listed by /v1/limits, never charged.
85 +SITE_MAX_ROWS = 200_000
86 +SITE_BURST_PER_MINUTE_PER_IP = 1200 # DoS guard only; a human on /charts stays far below (fail-open)
87 +SITE_TIER = Tier("site", "site", 60, 1_000_000_000, 1_000_000_000_000, SITE_MAX_ROWS)
88 +
79 89
80 90 def tier_for(name: str | None) -> Tier:
81 91 return TIERS.get(name or "", TIERS["free"])
added tests/test_ratelimit_site.py +191 −0
@@ -0,0 +1,191 @@
1 +"""Site principal: the /charts page has no quota (ratelimit/middleware.py `is_site_request`).
2 +
3 +Three cumulative criteria (`X-HFMD-Client`, `Sec-Fetch-Site`, same-host `Origin`/`Referer`) on data GETs →
4 +principal `site`: no X-RateLimit-* headers, no 429 for quota reasons, max_rows 200 000, usage under `site`
5 +(never under the visitor's IP), and a per-IP burst guard that is a DoS protection, not a product limit.
6 +"""
7 +from __future__ import annotations
8 +
9 +from dataclasses import replace
10 +
11 +import pytest
12 +from fastapi import FastAPI, Request # module level: `from __future__ import annotations` + a fixture-local import
13 + # would leave `Request` unresolvable in the echo handler (→ query param, 422)
14 +
15 +SITE = {"X-HFMD-Client": "charts", "Sec-Fetch-Site": "same-origin", "Origin": "https://www.hfmarketdata.io"}
16 +RL_HDRS = ("X-RateLimit-Limit-Requests", "X-RateLimit-Remaining-Requests", "X-RateLimit-Limit-Rows",
17 + "X-RateLimit-Remaining-Rows", "X-RateLimit-Reset")
18 +
19 +
20 +def ip(n: int) -> dict:
21 + return {"X-Forwarded-For": f"172.16.0.1, 10.0.0.{n}"}
22 +
23 +
24 +def site(n: int, **over) -> dict:
25 + return {**ip(n), **SITE, **over}
26 +
27 +
28 +def no_rl_headers(r) -> bool:
29 + return not any(h in r.headers for h in RL_HDRS)
30 +
31 +
32 +# ----------------------------------------------------------------------------------------------------
33 +# Full app (synthetic lake)
34 +# ----------------------------------------------------------------------------------------------------
35 +
36 +def test_site_requests_are_exempt_and_leave_the_ip_counters_untouched(client):
37 + for _ in range(31): # keyless would 429 after 30
38 + r = client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=5", headers=site(101))
39 + assert r.status_code == 200, r.text
40 + assert no_rl_headers(r)
41 + assert r.headers["X-Row-Count"] == "5"
42 + # the same IP, as a plain keyless client, still has its full hourly budget
43 + r = client.get("/v1/status", headers=ip(101))
44 + assert r.status_code == 200 and r.headers["X-RateLimit-Remaining-Requests"] == "29"
45 + assert r.headers["X-RateLimit-Remaining-Rows"] == "100000"
46 +
47 +
48 +@pytest.mark.parametrize("path", ["/v1/stock/tickers?limit=3", "/v1/futures/roots", "/v1/options/quarters",
49 + "/v1/bars/stock/AAPL?timeframe=1min&limit=10"])
50 +def test_every_charts_data_path_is_covered(client, path):
51 + r = client.get(path, headers=site(102))
52 + assert r.status_code == 200, r.text
53 + assert no_rl_headers(r)
54 +
55 +
56 +def test_web_client_label_and_referer_fallback_are_accepted(client):
57 + r = client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=2", headers=site(103, **{"X-HFMD-Client": "web"}))
58 + assert r.status_code == 200 and no_rl_headers(r)
59 + hdrs = {**ip(103), "X-HFMD-Client": "charts", "Sec-Fetch-Site": "same-site", "Referer": "https://hfmarketdata.io/charts?s=AAPL"}
60 + r = client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=2", headers=hdrs)
61 + assert r.status_code == 200 and no_rl_headers(r)
62 +
63 +
64 +@pytest.mark.parametrize("missing", [
65 + {"Sec-Fetch-Site": None}, # no Sec-Fetch-Site → keyless
66 + {"Sec-Fetch-Site": "cross-site"},
67 + {"Origin": "https://evil.example"}, # wrong host
68 + {"Origin": None}, # no Origin, no Referer
69 + {"X-HFMD-Client": None},
70 + {"X-HFMD-Client": "curl"},
71 +])
72 +def test_any_missing_criterion_falls_back_to_keyless(client, missing):
73 + hdrs = {**ip(104), **SITE}
74 + for k, v in missing.items():
75 + if v is None:
76 + hdrs.pop(k)
77 + else:
78 + hdrs[k] = v
79 + r = client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=2", headers=hdrs)
80 + assert r.status_code == 200
81 + assert r.headers["X-RateLimit-Limit-Requests"] == "30", dict(r.headers)
82 +
83 +
84 +def test_site_headers_on_a_non_data_path_or_a_non_get_are_ignored(client):
85 + r = client.get("/v1/status", headers=site(105))
86 + assert r.headers["X-RateLimit-Limit-Requests"] == "30" # /v1/status is not a charts data path
87 + r = client.get("/v1/limits", headers=site(105))
88 + assert r.json()["data"]["principal"]["kind"] == "keyless"
89 +
90 +
91 +def test_bearer_key_wins_over_the_site_criteria(client, make_user):
92 + _, key, _, _ = make_user()
93 + r = client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=2", headers=site(106, Authorization=f"Bearer {key}"))
94 + assert r.status_code == 200 and r.headers["X-RateLimit-Limit-Requests"] == "120"
95 +
96 +
97 +def test_usage_is_recorded_under_site_not_under_the_ip(client):
98 + from ratelimit import middleware, usage
99 + client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=7", headers=site(107))
100 + live = usage._live_minutes()
101 + by_principal: dict[str, dict] = {}
102 + for (principal, _minute), fields in live.items():
103 + acc = by_principal.setdefault(principal, {})
104 + for k, v in fields.items():
105 + acc[k] = acc.get(k, 0) + v
106 + assert by_principal["site"]["requests"] >= 1 and by_principal["site"]["rows"] >= 7
107 + assert f"ip:{middleware.hash_ip('10.0.0.107')}" not in by_principal
108 +
109 +
110 +def test_burst_guard_is_a_dedicated_429_without_ratelimit_headers(client, monkeypatch):
111 + from ratelimit import tiers
112 + monkeypatch.setattr(tiers, "SITE_BURST_PER_MINUTE_PER_IP", 3)
113 + for _ in range(3):
114 + assert client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=1", headers=site(108)).status_code == 200
115 + r = client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=1", headers=site(108))
116 + assert r.status_code == 429
117 + body = r.json()["error"]
118 + assert body["code"] == "RATE_LIMIT_EXCEEDED" and body["details"]["guard"] == "site_burst"
119 + assert "not a quota" in body["message"] and "free account" not in body["message"]
120 + assert int(r.headers["Retry-After"]) >= 1 and no_rl_headers(r)
121 + # another IP is unaffected, and the guarded IP keeps its keyless budget intact
122 + assert client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=1", headers=site(109)).status_code == 200
123 + assert client.get("/v1/status", headers=ip(108)).headers["X-RateLimit-Remaining-Requests"] == "29"
124 +
125 +
126 +def test_burst_guard_fails_open_when_redis_is_down(client, monkeypatch):
127 + from ratelimit import redis_limiter as rl
128 + from ratelimit import tiers
129 + monkeypatch.setattr(tiers, "SITE_BURST_PER_MINUTE_PER_IP", 1)
130 + monkeypatch.setattr(rl, "throttle", lambda *a, **k: None)
131 + for _ in range(3):
132 + r = client.get("/v1/bars/stock/AAPL?timeframe=1day&limit=1", headers=site(110))
133 + assert r.status_code == 200 and no_rl_headers(r)
134 +
135 +
136 +# ----------------------------------------------------------------------------------------------------
137 +# Minimal app: what the handler sees (max_rows, principal), localhost rule
138 +# ----------------------------------------------------------------------------------------------------
139 +
140 +@pytest.fixture
141 +def mini(app):
142 + """Tiny FastAPI app with only the rate-limit middleware, echoing request.state on a charts data path."""
143 + from fastapi.testclient import TestClient
144 +
145 + from core.responses import json_response
146 + from ratelimit import middleware
147 +
148 + m = FastAPI()
149 +
150 + def echo(request: Request, asset: str, ticker: str):
151 + st = request.state
152 + return json_response({"principal": st.principal, "kind": st.principal_kind, "max_rows": st.max_rows,
153 + "ratelimit": st.ratelimit})
154 +
155 + m.add_api_route("/v1/bars/{asset}/{ticker}", echo, methods=["GET"])
156 + middleware.install(m)
157 + with TestClient(m) as c:
158 + yield c
159 +
160 +
161 +def test_site_state_max_rows_and_principal(mini):
162 + r = mini.get("/v1/bars/stock/AAPL", headers=site(111))
163 + assert r.status_code == 200
164 + body = r.json()["data"]
165 + assert body == {"principal": "site", "kind": "site", "max_rows": 200_000,
166 + "ratelimit": {"principal": "site", "kind": "site", "tier": "site", "exempt": True, "max_rows_per_request": 200_000}}
167 + r = mini.get("/v1/bars/stock/AAPL", headers=ip(111)).json()["data"]
168 + assert r["kind"] == "keyless" and r["max_rows"] == 5000
169 +
170 +
171 +def test_localhost_origin_only_outside_production(mini, monkeypatch):
172 + from ratelimit import middleware
173 + dev = site(112, Origin="http://localhost:5173")
174 + assert mini.get("/v1/bars/stock/AAPL", headers=dev).json()["data"]["kind"] == "site" # HFMD_ENV=test
175 + monkeypatch.setattr(middleware, "settings", replace(middleware.settings, env="production"))
176 + assert mini.get("/v1/bars/stock/AAPL", headers=dev).json()["data"]["kind"] == "keyless"
177 + assert mini.get("/v1/bars/stock/AAPL", headers=site(112)).json()["data"]["kind"] == "site" # public host still fine
178 +
179 +
180 +def test_is_site_request_unit():
181 + from starlette.datastructures import Headers
182 +
183 + from ratelimit.middleware import is_site_data_path, is_site_request
184 + assert is_site_data_path("/v1/bars/stock/AAPL") and is_site_data_path("/v1/fx/tickers") and is_site_data_path("/v1/futures/ES/contracts")
185 + assert not is_site_data_path("/v1/status") and not is_site_data_path("/v1/fundamentals/screen") and not is_site_data_path("/v1/stock/tickers/x")
186 + scope = {"method": "GET", "path": "/v1/bars/stock/AAPL", "query_string": b""}
187 + good = Headers({"x-hfmd-client": "charts", "sec-fetch-site": "same-origin", "origin": "https://www.hfmarketdata.io"})
188 + assert is_site_request(scope, good)
189 + assert not is_site_request({**scope, "method": "POST"}, good)
190 + assert not is_site_request({**scope, "query_string": b"api_key=hfmd_live_x"}, good)
191 + assert not is_site_request(scope, Headers({"x-hfmd-client": "charts", "sec-fetch-site": "same-origin", "origin": "https://www.hfmarketdata.io.evil.example"}))
192