coinexplorer v4 — self-hosted stablecoin & crypto explorer
Full platform: 24-chain indexer (EVM/Tron/Solana/Bitcoin) on free public RPCs only, FastAPI + WebSocket API, responsive dashboard with live charts, interactive API docs, price-aware whale/flow/supply analytics. Author: Simon-Pierre Boucher <contact@spboucher.ai> Live: https://www.coinexplorer.io Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Showing 47 changed files with +6,211 and −0
added
.env.example
+24 −0
@@ -0,0 +1,24 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +# --- indexer --------------------------------------------------------- | |
| 4 | +# comma-separated subset of chains to index; empty = all configured | |
| 5 | +EXPLORER_CHAINS= | |
| 6 | +# SQLite path (dev). Docker compose overrides this to /data/explorer.db | |
| 7 | +EXPLORER_DB=./explorer.db | |
| 8 | +# grow history backwards N days (0 = off); per-chain backfill_days in | |
| 9 | +# chains.yaml overrides. Backfill runs at lower priority than head tailing. | |
| 10 | +EXPLORER_BACKFILL_DAYS=0 | |
| 11 | + | |
| 12 | +# --- api ------------------------------------------------------------- | |
| 13 | +API_PORT=8080 | |
| 14 | + | |
| 15 | +# --- postgres --------------------------------------------------------- | |
| 16 | +POSTGRES_USER=explorer | |
| 17 | +POSTGRES_PASSWORD=explorer | |
| 18 | +POSTGRES_DB=explorer | |
| 19 | +POSTGRES_PORT=5433 # host port (containers use 5432 internally) | |
| 20 | +# Uncomment to store in postgres instead of SQLite. | |
| 21 | +# Inside docker compose: | |
| 22 | +# DATABASE_URL=postgresql://explorer:explorer@postgres:5432/explorer | |
| 23 | +# From the host (local dev): | |
| 24 | +# DATABASE_URL=postgresql://explorer:explorer@127.0.0.1:5433/explorer | |
added
.gitignore
+15 −0
@@ -0,0 +1,15 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +.venv/ | |
| 4 | +__pycache__/ | |
| 5 | +*.pyc | |
| 6 | +explorer.db* | |
| 7 | +indexer.log | |
| 8 | +.env | |
| 9 | +api.log | |
| 10 | +api.pid | |
| 11 | +*.log | |
| 12 | +.claude/ | |
| 13 | +pg_test.log | |
| 14 | +.venv/ | |
| 15 | +.DS_Store | |
added
CLAUDE.md
+136 −0
@@ -0,0 +1,136 @@ | ||
| 1 | +# CLAUDE.md — Stablecoin Blockchains & Building a Free Block Explorer API From Scratch | |
| 2 | + | |
| 3 | +## Role | |
| 4 | + | |
| 5 | +You are an expert in blockchain infrastructure, stablecoins, and indexer/explorer architecture. When the user asks questions in this project, answer in English, in a technical and hands-on way, following the directives below. The end goal of this project is to **build a free, self-hosted block explorer API** covering every chain where major stablecoins live — using only free public RPC endpoints and open-source tooling, no paid API keys. | |
| 6 | + | |
| 7 | +--- | |
| 8 | + | |
| 9 | +## Mission 1 — Map every major stablecoin to its blockchains | |
| 10 | + | |
| 11 | +For each major stablecoin, always detail: | |
| 12 | + | |
| 13 | +1. **The issuer** (Tether, Circle, Ethena, Sky/Maker, First Digital, Blast, PayPal/Paxos, Ripple, Global Dollar Network/Paxos, Techteryx…) | |
| 14 | +2. **Chains where it is natively issued** vs **bridged versions** (explain the difference and bridge counterparty risk) | |
| 15 | +3. **The token standard on each network** (ERC-20, TRC-20, SPL, BEP-20, Jetton on TON, Move coin on Aptos/Sui, ASA on Algorand, issued asset on XRPL/Stellar, IBC denom on Cosmos/Noble, etc.) | |
| 16 | +4. **The verified contract/mint address** on major networks when known — always verify via web search, never guess addresses | |
| 17 | + | |
| 18 | +### Reference table (keep up to date via web search) | |
| 19 | + | |
| 20 | +| Stablecoin | Main blockchains | | |
| 21 | +|---|---| | |
| 22 | +| USDT | Ethereum, Tron, Solana, BNB Chain, Avalanche, Arbitrum, Optimism, Polygon, TON, Aptos, Celo, EOS, Near, Tezos, Algorand, Cosmos (via Noble), Kaia, Ink | | |
| 23 | +| USDC | Ethereum, Solana, Base, Arbitrum, Optimism, Polygon, Avalanche, Stellar, Aptos, Sui, Hedera, Algorand, Celo, Near, ZKsync, Starknet, Linea, Monad, XRP Ledger, World Chain, Unichain, XDC, Sei, Sonic, Polkadot, Morph, Ink, HyperEVM, Noble (30+ networks) | | |
| 24 | +| USDe | Ethereum, Solana, BNB Chain, Base, Arbitrum, Optimism, Mantle, Scroll | | |
| 25 | +| DAI / USDS | Ethereum, Arbitrum, Optimism, Base, Polygon, Avalanche, BNB Chain, Gnosis, Linea, Scroll, Unichain | | |
| 26 | +| FDUSD | Ethereum, BNB Chain | | |
| 27 | +| USDB | Blast (mainly), Ethereum | | |
| 28 | +| PYUSD | Ethereum, Solana | | |
| 29 | +| RLUSD | XRP Ledger (XRPL), Ethereum | | |
| 30 | +| USDG | Ethereum, Solana, Ink | | |
| 31 | +| TUSD | Ethereum, Tron, BNB Chain, Avalanche | | |
| 32 | + | |
| 33 | +Key facts to always recall: Ethereum hosts nearly every major stablecoin; Tron dominates USDT transfers (international payments, low fees); Solana is popular for USDC, USDT, PYUSD, USDG thanks to very low fees; Base is growing fast. Stablecoin deployments change quickly (e.g., Circle's CCTP expansions) — **verify with web search before presenting a list as exhaustive**. | |
| 34 | + | |
| 35 | +--- | |
| 36 | + | |
| 37 | +## Mission 2 — Explain how each chain works (only what an explorer builder needs) | |
| 38 | + | |
| 39 | +When explaining a network, focus on what matters for indexing it: | |
| 40 | + | |
| 41 | +- **Consensus & finality** (PoW, PoS, DPoS on Tron, PoH+PoS on Solana, Tendermint/CometBFT on Cosmos chains) — finality determines how many blocks to wait before treating data as final and how to handle **reorgs** | |
| 42 | +- **Account model**: account/balance (EVM, Tron), UTXO (Bitcoin), object model (Sui), resource model (Aptos/Move) | |
| 43 | +- **Block time & throughput** — determines polling frequency and ingestion load (Solana ≈ 400ms slots vs Ethereum ≈ 12s) | |
| 44 | +- **How a stablecoin transfer actually appears on-chain**: on EVM it's a smart-contract `transfer()` call emitting a `Transfer` event log — NOT a native value transfer; on Solana it's an SPL Token program instruction; on XRPL/Stellar it's a native issued-asset payment; on Cosmos it's a bank/IBC message | |
| 45 | + | |
| 46 | +--- | |
| 47 | + | |
| 48 | +## Mission 3 — Explore every chain from scratch, the hard way (no paid APIs) | |
| 49 | + | |
| 50 | +This is the core mission. Teach raw RPC access per chain family, then how to turn it into an indexer + API. Always show real, runnable code (curl first, then Python or Node.js). Never rely on Etherscan-style APIs with keys — the point is to read chains directly. | |
| 51 | + | |
| 52 | +### 3.1 Free public RPC endpoints | |
| 53 | + | |
| 54 | +Maintain a config of free endpoints (verify availability via web search; suggest fallbacks and rotation since free RPCs rate-limit): | |
| 55 | + | |
| 56 | +- **EVM chains** (Ethereum, BNB, Polygon, Avalanche C-Chain, Arbitrum, Optimism, Base, Blast, Scroll, Linea, Mantle, Celo, Gnosis, ZKsync, Unichain, World Chain, Sei-EVM, Sonic, Ink, Kaia, XDC, HyperEVM…): use public endpoints from Chainlist, LlamaNodes (`eth.llamarpc.com`), PublicNode (`ethereum-rpc.publicnode.com`), Ankr public tier, 1RPC, dRPC free tier, plus each chain's official public RPC | |
| 57 | +- **Solana**: `api.mainnet-beta.solana.com` (heavily rate-limited; explain limits and alternatives) | |
| 58 | +- **Tron**: TronGrid free tier, public fullnode HTTP API | |
| 59 | +- **TON**: toncenter free tier, public liteservers | |
| 60 | +- **XRPL**: `s1.ripple.com:51234` public JSON-RPC / WebSocket | |
| 61 | +- **Stellar**: Horizon at `horizon.stellar.org` (free) | |
| 62 | +- **Cosmos/Noble**: public REST (LCD) and RPC endpoints from the chain registry | |
| 63 | +- **Aptos / Sui**: official public fullnode REST/JSON-RPC endpoints | |
| 64 | +- **Algorand**: AlgoNode/Nodely free endpoints (algod + indexer) | |
| 65 | +- **Near, Tezos, Hedera (mirror node REST), Starknet, Polkadot (Sidecar/RPC), EOS**: list public endpoints per chain | |
| 66 | + | |
| 67 | +### 3.2 Chain family playbooks — raw block & transfer reading | |
| 68 | + | |
| 69 | +For each family, show: (a) get latest block/height, (b) fetch a full block, (c) fetch a transaction + receipt/result, (d) **extract stablecoin transfers**. | |
| 70 | + | |
| 71 | +**EVM (covers ~70% of the table — one codebase, many chains):** | |
| 72 | +```bash | |
| 73 | +# latest block | |
| 74 | +curl -s -X POST $RPC -H 'Content-Type: application/json' \ | |
| 75 | + -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' | |
| 76 | +# full block with transactions | |
| 77 | +curl -s -X POST $RPC -H 'Content-Type: application/json' \ | |
| 78 | + -d '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest",true],"id":1}' | |
| 79 | +``` | |
| 80 | +Key methods: `eth_blockNumber`, `eth_getBlockByNumber`, `eth_getTransactionReceipt`, `eth_getLogs`, `eth_call` (for `balanceOf`, `decimals`, `symbol`). To capture stablecoin transfers, filter logs by the ERC-20 Transfer topic: | |
| 81 | +`topic0 = keccak256("Transfer(address,address,uint256)") = 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef` | |
| 82 | +with `address` = the stablecoin contract. Teach ABI decoding by hand (topics = indexed from/to addresses left-padded to 32 bytes; data = uint256 amount; divide by 10^decimals — USDT/USDC use 6, DAI uses 18). | |
| 83 | + | |
| 84 | +**Tron:** HTTP API (`/wallet/getnowblock`, `/wallet/getblockbynum`, `/wallet/gettransactioninfobyid`). TRC-20 transfers appear as `TriggerSmartContract` calls; decode the same Transfer event from `log` entries in transaction info. Explain Base58Check vs hex address conversion (41-prefixed hex ↔ T-addresses). | |
| 85 | + | |
| 86 | +**Solana:** JSON-RPC `getSlot`, `getBlock` (with `maxSupportedTransactionVersion: 0`), `getTransaction`, `getSignaturesForAddress`, `getTokenAccountsByOwner`. SPL transfers are `spl-token` program instructions (`transfer` / `transferChecked`); parse `preTokenBalances` / `postTokenBalances` diffs as the robust method. Explain mint addresses, associated token accounts (ATAs), and why owner ≠ token account. | |
| 87 | + | |
| 88 | +**XRP Ledger:** JSON-RPC/WebSocket `ledger`, `tx`, `account_tx`, `account_lines`. RLUSD/USDC are issued currencies — a transfer is a native `Payment` with `currency` + `issuer`; also explain trust lines. | |
| 89 | + | |
| 90 | +**Stellar:** Horizon REST (`/ledgers`, `/transactions`, `/operations`, `/payments?asset_code=USDC&asset_issuer=...`) with streaming via SSE. | |
| 91 | + | |
| 92 | +**Cosmos / Noble:** CometBFT RPC (`/block?height=`) + LCD REST (`/cosmos/tx/v1beta1/txs?events=...`). USDC on Noble is a bank denom (`uusdc`); transfers are `MsgSend` and IBC `MsgTransfer`. | |
| 93 | + | |
| 94 | +**Aptos:** REST `/v1/blocks/by_height/{h}?with_transactions=true`; token transfers are `0x1::coin` / fungible-asset events. | |
| 95 | +**Sui:** JSON-RPC `sui_getCheckpoint`, `suix_queryTransactionBlocks`; parse `balanceChanges` by coin type. | |
| 96 | +**Algorand:** algod `/v2/blocks/{round}`; ASA transfers are `axfer` transactions filtered by asset-id. | |
| 97 | +**TON:** toncenter `/getTransactions`; USDT is a Jetton — explain jetton wallets and internal message parsing. | |
| 98 | +**Near, Tezos, Hedera, Starknet, Polkadot, EOS:** give the equivalent minimal playbook (block by height → tx → token event) using each chain's native RPC/REST, using web search to confirm current endpoints and formats. | |
| 99 | + | |
| 100 | +### 3.3 Build the indexer (the hard way) | |
| 101 | + | |
| 102 | +Guide the user to build, step by step: | |
| 103 | + | |
| 104 | +1. **Ingestion loop** per chain: poll head height → backfill from a checkpoint → fetch blocks in batches (JSON-RPC batching on EVM) → respect rate limits (token bucket) → rotate across multiple free RPCs on 429s | |
| 105 | +2. **Reorg handling**: store block hash + parent hash; on mismatch, roll back N blocks (chain-specific finality depth; near-zero for Tendermint/Solana finalized commitment) | |
| 106 | +3. **Decoding layer**: per chain-family adapters that normalize everything into one canonical event: `{chain, block, tx_hash, timestamp, token, from, to, amount, decimals}` | |
| 107 | +4. **Storage**: start with SQLite, graduate to PostgreSQL; schema: `blocks`, `transactions`, `transfers`, `tokens`, `cursors`; index on `(token, block)` and `(from)`, `(to)` | |
| 108 | +5. **API layer**: FastAPI (Python) or Express/Fastify (Node): endpoints like | |
| 109 | + - `GET /v1/{chain}/block/latest` · `GET /v1/{chain}/block/{number}` | |
| 110 | + - `GET /v1/{chain}/tx/{hash}` | |
| 111 | + - `GET /v1/{chain}/token/{address}/transfers?from=&to=&min_amount=` | |
| 112 | + - `GET /v1/address/{addr}/transfers` (cross-chain) | |
| 113 | + - `GET /v1/stablecoins/volume?token=USDT&window=24h` | |
| 114 | +6. **Free deployment**: run locally or on free tiers (Fly.io, Railway/Render free plans, a cheap VPS, or a home machine + Cloudflare Tunnel); caching with in-process LRU or Redis; explain that full historical backfill of Ethereum on free RPCs is impractical — start from recent blocks and grow | |
| 115 | +7. **Optional level-up**: run your own node where feasible (Reth/Geth + consensus client, Tron java-tron, Solana RPC node hardware caveats) to remove rate limits entirely | |
| 116 | + | |
| 117 | +### 3.4 Teaching style for this mission | |
| 118 | + | |
| 119 | +- Always start with a single `curl` the user can run immediately, then scale up to scripts (`web3.py`/`ethers` optional — prefer raw JSON-RPC first so the user learns the protocol, not the library) | |
| 120 | +- Explain every hex/encoding detail (0x quantities, RLP existence, topic padding, base58, bech32) rather than hiding it | |
| 121 | +- Prefer one **EVM-generic module** reused across all EVM chains + one adapter per non-EVM family | |
| 122 | +- When an endpoint or address may have changed, verify via web search before answering | |
| 123 | + | |
| 124 | +--- | |
| 125 | + | |
| 126 | +## Mission 4 — Guided tour format | |
| 127 | + | |
| 128 | +When the user asks for a tour, structure it as: (1) stablecoin market overview (verify market caps via web search); (2) stablecoin → chain map (table above); (3) how the dominant networks work; (4) live demo: fetch a real recent block, find a USDT/USDC transfer in it, decode it by hand, and show how it would flow through the indexer into the API. | |
| 129 | + | |
| 130 | +## Response style | |
| 131 | + | |
| 132 | +- Answer in **English**, clear prose, with runnable code blocks | |
| 133 | +- Use **web search** for market caps, endpoints, contract addresses, and anything that may have changed | |
| 134 | +- Never fabricate contract addresses or RPC URLs — verify or say so | |
| 135 | +- Always distinguish **native issuance** vs **bridged** versions (bridge counterparty risk) | |
| 136 | +- Remind that none of this is financial advice | |
added
Dockerfile
+16 −0
@@ -0,0 +1,16 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +FROM python:3.12-slim | |
| 4 | + | |
| 5 | +WORKDIR /app | |
| 6 | +COPY requirements.txt . | |
| 7 | +RUN pip install --no-cache-dir -r requirements.txt | |
| 8 | + | |
| 9 | +COPY config/ config/ | |
| 10 | +COPY indexer/ indexer/ | |
| 11 | +COPY api/ api/ | |
| 12 | +COPY scripts/ scripts/ | |
| 13 | +COPY run_indexer.py . | |
| 14 | + | |
| 15 | +# same image runs both roles; compose picks the command | |
| 16 | +CMD ["python", "run_indexer.py"] | |
added
LICENSE
+9 −0
@@ -0,0 +1,9 @@ | ||
| 1 | +MIT License | |
| 2 | + | |
| 3 | +Copyright (c) 2026 Simon-Pierre Boucher <contact@spboucher.ai> | |
| 4 | + | |
| 5 | +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: | |
| 6 | + | |
| 7 | +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. | |
| 8 | + | |
| 9 | +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | |
added
README.md
+198 −0
@@ -0,0 +1,198 @@ | ||
| 1 | +<div align="center"> | |
| 2 | + | |
| 3 | +# 🪙 coinexplorer | |
| 4 | + | |
| 5 | +### The self-hosted, zero-API-key explorer for stablecoins & major crypto | |
| 6 | + | |
| 7 | +**Live at [www.coinexplorer.io](https://www.coinexplorer.io)** | |
| 8 | + | |
| 9 | +[](https://www.coinexplorer.io) | |
| 10 | +[](https://www.coinexplorer.io/v1/prices) | |
| 11 | +[](https://www.coinexplorer.io/v1/prices) | |
| 12 | +[](https://www.coinexplorer.io/status.html) | |
| 13 | +[](https://www.coinexplorer.io/v1/tokens) | |
| 14 | +[](#-free-rpc-philosophy) | |
| 15 | +[](https://www.python.org) | |
| 16 | +[](https://www.coinexplorer.io/api.html) | |
| 17 | +[](tests/) | |
| 18 | +[](LICENSE) | |
| 19 | + | |
| 20 | +*The BTC/ETH badges above are rendered from this platform's own live API.* | |
| 21 | + | |
| 22 | +<img src="docs/screenshot-overview.png" alt="coinexplorer market overview" width="920"> | |
| 23 | + | |
| 24 | +</div> | |
| 25 | + | |
| 26 | +--- | |
| 27 | + | |
| 28 | +## What is this? | |
| 29 | + | |
| 30 | +A **complete blockchain explorer platform built from scratch** — indexer, API, and dashboard — | |
| 31 | +that reads **raw JSON-RPC/REST from free public endpoints only**. No Etherscan keys, no Infura, | |
| 32 | +no paid data vendors. Point it at the internet and it indexes: | |
| 33 | + | |
| 34 | +- 💵 **11 stablecoins** — USDT, USDC, DAI/USDS, USDe, FDUSD, PYUSD, RLUSD, USDG, TUSD, USDB — across every chain they live on | |
| 35 | +- 🚀 **Major crypto** — WETH, WBTC, LINK, UNI, AAVE, SHIB, PEPE, ARB, OP, BONK, JUP, WIF and more, valued at live prices | |
| 36 | +- 🐋 **Native-coin whales** — large BTC, ETH, BNB, AVAX, POL, TRX and SOL transfers, straight from full blocks | |
| 37 | + | |
| 38 | +**Four chain families, one canonical event:** | |
| 39 | + | |
| 40 | +| Family | Chains | How transfers are read | | |
| 41 | +|---|---|---| | |
| 42 | +| **EVM** | Ethereum, Base, Arbitrum, Optimism, Polygon, BSC, Avalanche, Celo, Kaia, Ink, Scroll, Linea, Mantle, Gnosis, ZKsync, Blast, Unichain, World Chain, Sonic, Sei, HyperEVM | `eth_getLogs` + hand-rolled ABI decoding | | |
| 43 | +| **Tron** | Tron | block tx-infos + Base58Check codec written from scratch | | |
| 44 | +| **Solana** | Solana | pre/post token-balance diffs (covers Token-2022 & CPIs) | | |
| 45 | +| **Bitcoin** | Bitcoin | esplora REST, UTXO whale outputs | | |
| 46 | + | |
| 47 | +Everything lands in one schema: `{chain, block, tx_hash, timestamp, token, from, to, amount, decimals}` — | |
| 48 | +so the API, WebSocket stream, and dashboard are chain-agnostic. | |
| 49 | + | |
| 50 | +## 📊 The numbers | |
| 51 | + | |
| 52 | +| Metric | Value | | |
| 53 | +|---|---| | |
| 54 | +| Chains indexing live | **24** (13 more configured & verified, awaiting adapters) | | |
| 55 | +| Verified asset deployments | **94** — every EVM address checked on-chain (`symbol()` + `decimals()`) before indexing | | |
| 56 | +| Transfers indexed (production instance) | **20M+ and growing** | | |
| 57 | +| Hot API endpoints | **< 200 ms** at any index size (precomputed aggregates) | | |
| 58 | +| Whale detection floor | $100K, incremental — a $72M BTC move was caught minutes after boot | | |
| 59 | +| Price refresh | 40 symbols / 5 min, one keyless CoinGecko call | | |
| 60 | +| Paid API keys required | **0** | | |
| 61 | + | |
| 62 | +## ✨ The platform | |
| 63 | + | |
| 64 | +<div align="center"><img src="docs/screenshot-api.png" alt="API documentation" width="920"></div> | |
| 65 | + | |
| 66 | +- **8-page responsive app** (desktop → smartphone): market overview, per-token analytics, per-chain | |
| 67 | + explorer, transfer browser, whale watch, issuance flows, address & tx views, infra status | |
| 68 | +- **Real charts** — hand-rolled SVG line charts with crosshair tooltips, diverging issuance columns, | |
| 69 | + magnitude bars; colorblind-validated palette, native dark mode | |
| 70 | +- **Crypto icons everywhere** — real logos with brand-colored generated fallbacks (no broken images, no tracking) | |
| 71 | +- **Live WebSocket feed** — `wss://…/v1/stream/transfers` pushes every transfer above your USD floor | |
| 72 | +- **[Interactive API docs](https://www.coinexplorer.io/api.html)** — 22 endpoints with in-page | |
| 73 | + *Try it* consoles that execute against the live instance, plus OpenAPI at `/docs` | |
| 74 | +- **Signals stablecoin holders actually want**: net issuance (mints − burns) per token, on-chain | |
| 75 | + supply history per chain, native vs bridged tagging, discontinued-asset flags, whale rankings | |
| 76 | + | |
| 77 | +## 🏗 Architecture | |
| 78 | + | |
| 79 | +```mermaid | |
| 80 | +flowchart LR | |
| 81 | + subgraph rpcs [Free public RPCs — no keys] | |
| 82 | + EVM[21 EVM chains] | |
| 83 | + TRON[TronGrid] | |
| 84 | + SOL[Solana] | |
| 85 | + BTC[esplora REST] | |
| 86 | + end | |
| 87 | + subgraph indexer [Indexer — one watchdogged thread per chain] | |
| 88 | + POOL[RpcPool / RestPool<br/>token bucket · health EMA · failover] | |
| 89 | + AD[4 family adapters<br/>canonical event out] | |
| 90 | + WORK[Workers: prices · supply<br/>whale extraction · rolling aggregates] | |
| 91 | + end | |
| 92 | + DB[(SQLite WAL / PostgreSQL<br/>one codebase, both engines)] | |
| 93 | + subgraph api [FastAPI] | |
| 94 | + REST[REST /v1] | |
| 95 | + WS[WebSocket stream] | |
| 96 | + MET[Prometheus /metrics] | |
| 97 | + UI[dashboard + API docs] | |
| 98 | + end | |
| 99 | + rpcs --> POOL --> AD --> DB | |
| 100 | + POOL --> WORK --> DB | |
| 101 | + DB --> REST & WS & MET & UI | |
| 102 | +``` | |
| 103 | + | |
| 104 | +### Built for hostile (free) infrastructure | |
| 105 | + | |
| 106 | +- **Per-endpoint token buckets** — respect documented limits (toncenter 1 rps, Solana ~10 rps/IP…) *before* getting 429'd | |
| 107 | +- **Health-scored failover** — success-rate EMA + exponential cooldowns; rate limits disguised as JSON-RPC errors are detected and routed around; *"query too big"* errors trigger adaptive range halving instead | |
| 108 | +- **Reorg safety** — parent-hash chain-linking on every cursor advance + chain-specific confirmation depths, with automatic rollback | |
| 109 | +- **Two-cursor design** — head-tailing always has priority; history grows *backwards* one slice per cycle (`EXPLORER_BACKFILL_DAYS`) | |
| 110 | +- **Watchdog supervisor** — any dead worker thread is rebuilt within 30 s | |
| 111 | +- **Scale-proof reads** — whale extraction and volume/series aggregates are maintained incrementally by background workers, so no UI query ever rescans the transfers table (26 s → 2 ms, measured) | |
| 112 | + | |
| 113 | +## 🚀 Quickstart | |
| 114 | + | |
| 115 | +```bash | |
| 116 | +git clone https://github.com/spboucher-ai/coinexplorer && cd coinexplorer | |
| 117 | + | |
| 118 | +# Docker — one command | |
| 119 | +cp .env.example .env && docker compose up -d --build | |
| 120 | +open http://localhost:8080 | |
| 121 | + | |
| 122 | +# Or bare metal | |
| 123 | +python3 -m venv .venv && .venv/bin/pip install -r requirements.txt | |
| 124 | +.venv/bin/python scripts/verify_tokens.py # never index an unverified address | |
| 125 | +.venv/bin/python run_indexer.py # Ctrl-C safe, resumes from cursors | |
| 126 | +.venv/bin/uvicorn api.main:app --port 8080 | |
| 127 | +``` | |
| 128 | + | |
| 129 | +## 🔌 API in 30 seconds | |
| 130 | + | |
| 131 | +Full interactive reference: **[coinexplorer.io/api.html](https://www.coinexplorer.io/api.html)** | |
| 132 | + | |
| 133 | +```bash | |
| 134 | +# largest transfers of the last 24h, any asset, any chain | |
| 135 | +curl "https://www.coinexplorer.io/v1/stablecoins/whales?window=24h&min_usd=10000000" | |
| 136 | + | |
| 137 | +# USDC net issuance (mints − burns), bucketed | |
| 138 | +curl "https://www.coinexplorer.io/v1/stablecoins/flows?token=USDC&window=7d" | |
| 139 | + | |
| 140 | +# resolve anything: tx hash, address, symbol, chain | |
| 141 | +curl "https://www.coinexplorer.io/v1/search?q=TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t" | |
| 142 | + | |
| 143 | +# live feed | |
| 144 | +websocat "wss://www.coinexplorer.io/v1/stream/transfers?min_usd=100000" | |
| 145 | +``` | |
| 146 | + | |
| 147 | +## ➕ Add a chain in < 30 lines (EVM: config only) | |
| 148 | + | |
| 149 | +```yaml | |
| 150 | +# config/chains.yaml | |
| 151 | + mynewchain: | |
| 152 | + family: evm | |
| 153 | + chain_id: 12345 | |
| 154 | + block_time: 2 | |
| 155 | + confirmations: 10 | |
| 156 | + max_range: 150 | |
| 157 | + start_offset: 1800 | |
| 158 | + rpcs: [https://rpc.mynewchain.org] | |
| 159 | +``` | |
| 160 | + | |
| 161 | +Add its tokens to `config/tokens.yaml`, run `scripts/verify_tokens.py` | |
| 162 | +(it refuses wrong addresses and chain-ids), restart. Non-EVM families are ~150-line | |
| 163 | +adapters — see `indexer/adapters/tron.py` as the template. **13 more chains** | |
| 164 | +(XRPL, Stellar, Noble, TON, Aptos, Sui, Algorand, Near, Tezos, Hedera, Starknet, | |
| 165 | +Polkadot, EOS) are already configured with verified identifiers, waiting for their adapters. | |
| 166 | + | |
| 167 | +## 🧪 Tests | |
| 168 | + | |
| 169 | +```bash | |
| 170 | +for t in tests/test_*.py; do .venv/bin/python "$t"; done | |
| 171 | +``` | |
| 172 | + | |
| 173 | +Four suites, 19 checks: token-bucket pacing, failover classification, getLogs range-halving | |
| 174 | +contiguity, reorg rollback, Base58Check vectors, TRC-20/SPL decoding, Bitcoin UTXO whale | |
| 175 | +semantics, EVM native extraction, price upserts. | |
| 176 | + | |
| 177 | +## ⚖️ Data honesty | |
| 178 | + | |
| 179 | +Supply figures use on-chain `totalSupply()` (bridged wrappers double-count their locked | |
| 180 | +collateral — the `native` flag lets you de-duplicate). Solana multi-sender transfers keep exact | |
| 181 | +amounts with `from: null`. Volume uses face value × live price over the indexed window only. | |
| 182 | +Aggregates use float casts — dashboards, not accounting. **Nothing here is financial advice.** | |
| 183 | + | |
| 184 | +--- | |
| 185 | + | |
| 186 | +<div align="center"> | |
| 187 | + | |
| 188 | +## Author | |
| 189 | + | |
| 190 | +**Simon-Pierre Boucher** | |
| 191 | +📧 [contact@spboucher.ai](mailto:contact@spboucher.ai) | |
| 192 | + | |
| 193 | +Built from scratch — every decoder, every adapter, every chart. | |
| 194 | +🌐 **[www.coinexplorer.io](https://www.coinexplorer.io)** | |
| 195 | + | |
| 196 | +[MIT License](LICENSE) | |
| 197 | + | |
| 198 | +</div> | |
added
api/__init__.py
+2 −0
@@ -0,0 +1,2 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
added
api/main.py
+832 −0
@@ -0,0 +1,832 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Stablecoin explorer API — FastAPI over the indexer's SQLite DB, | |
| 4 | +plus live pass-through reads (blocks, txs) straight from free RPCs. | |
| 5 | + | |
| 6 | + uvicorn api.main:app --port 8080 | |
| 7 | +""" | |
| 8 | + | |
| 9 | +import asyncio | |
| 10 | +import functools | |
| 11 | +import os | |
| 12 | +import time | |
| 13 | + | |
| 14 | +import pathlib | |
| 15 | + | |
| 16 | +from fastapi import FastAPI, HTTPException, Query, WebSocket, WebSocketDisconnect | |
| 17 | +from fastapi.staticfiles import StaticFiles | |
| 18 | + | |
| 19 | +from indexer import config, db | |
| 20 | +from indexer.decode import TRANSFER_TOPIC, decode_transfer, format_amount | |
| 21 | +from indexer.enrich import ZERO_ADDRESSES | |
| 22 | +from indexer.rpc import RpcPool | |
| 23 | + | |
| 24 | +DB_PATH = config.db_path() | |
| 25 | +CHAINS = config.load_chains() | |
| 26 | +TOKENS = config.load_tokens() | |
| 27 | + | |
| 28 | +app = FastAPI( | |
| 29 | + title="coinexplorer API", | |
| 30 | + description="Self-hosted, free-RPC-only explorer for stablecoins and major crypto.", | |
| 31 | + version="0.3.0", | |
| 32 | +) | |
| 33 | + | |
| 34 | + | |
| 35 | +@app.exception_handler(Exception) | |
| 36 | +async def unhandled_exception(request, exc): | |
| 37 | + # never leak a stack trace; always answer JSON | |
| 38 | + from fastapi.responses import JSONResponse | |
| 39 | + return JSONResponse(status_code=500, content={"error": str(exc)[:200]}) | |
| 40 | + | |
| 41 | +_pools = {} | |
| 42 | + | |
| 43 | + | |
| 44 | +def pool(chain): | |
| 45 | + if chain not in CHAINS: | |
| 46 | + raise HTTPException(404, f"unknown chain '{chain}'") | |
| 47 | + if CHAINS[chain].get("family", "evm") != "evm": | |
| 48 | + raise HTTPException(501, f"live reads for '{chain}' land with its adapter (Deliverable 4)") | |
| 49 | + if chain not in _pools: | |
| 50 | + _pools[chain] = RpcPool(CHAINS[chain]["rpcs"]) | |
| 51 | + return _pools[chain] | |
| 52 | + | |
| 53 | + | |
| 54 | +def dbc(): | |
| 55 | + return db.connect(DB_PATH, readonly=True) | |
| 56 | + | |
| 57 | + | |
| 58 | +# stablecoins trade ~$1, so face value ≈ USD; CASE avoids relying on POW() | |
| 59 | +# (not compiled into every SQLite build) | |
| 60 | +USD_EXPR = ( | |
| 61 | + "CAST(amount AS DOUBLE PRECISION) / (CASE decimals " | |
| 62 | + + " ".join(f"WHEN {d} THEN 1e{d}" for d in range(19)) | |
| 63 | + + " ELSE 1e6 END)" | |
| 64 | +) | |
| 65 | + | |
| 66 | +# price-aware USD: stablecoins are seeded at 1.0 by the PriceWorker; crypto | |
| 67 | +# assets get their CoinGecko price; unknown symbols fall back to face value | |
| 68 | +PRICE_JOIN = "LEFT JOIN prices ON prices.symbol = transfers.symbol" | |
| 69 | +USD_PRICED = f"(({USD_EXPR}) * COALESCE(prices.usd, 1.0))" | |
| 70 | +SELECT_T = f"SELECT transfers.*, COALESCE(prices.usd, 1.0) AS usd_rate FROM transfers {PRICE_JOIN}" | |
| 71 | + | |
| 72 | +DEFAULT_WHALE_USD = float(os.environ.get("WHALE_MIN_USD", 1_000_000)) | |
| 73 | + | |
| 74 | + | |
| 75 | +def price_of(conn, symbol): | |
| 76 | + r = conn.execute("SELECT usd FROM prices WHERE symbol = ?", (symbol,)).fetchone() | |
| 77 | + return r["usd"] if r and r["usd"] is not None else 1.0 | |
| 78 | + | |
| 79 | +_cache = {} | |
| 80 | + | |
| 81 | + | |
| 82 | +def cached(ttl): | |
| 83 | + """Tiny in-process TTL cache for aggregate endpoints.""" | |
| 84 | + def deco(fn): | |
| 85 | + @functools.wraps(fn) | |
| 86 | + def wrap(*a, **k): | |
| 87 | + key = (fn.__name__, a, tuple(sorted(k.items()))) | |
| 88 | + hit = _cache.get(key) | |
| 89 | + now = time.time() | |
| 90 | + if hit and now - hit[0] < ttl: | |
| 91 | + return hit[1] | |
| 92 | + val = fn(*a, **k) | |
| 93 | + _cache[key] = (now, val) | |
| 94 | + return val | |
| 95 | + return wrap | |
| 96 | + return deco | |
| 97 | + | |
| 98 | + | |
| 99 | +def parse_window(window): | |
| 100 | + units = {"m": 60, "h": 3600, "d": 86400} | |
| 101 | + try: | |
| 102 | + return int(window[:-1]) * units[window[-1]] | |
| 103 | + except (KeyError, ValueError, IndexError): | |
| 104 | + raise HTTPException(400, "window must look like 15m, 24h or 7d") | |
| 105 | + | |
| 106 | + | |
| 107 | +def with_value(row): | |
| 108 | + d = dict(row) | |
| 109 | + rate = d.pop("usd_rate", None) | |
| 110 | + if d.get("amount") is not None and d.get("decimals") is not None: | |
| 111 | + d["value"] = format_amount(d["amount"], d["decimals"]) | |
| 112 | + if rate is not None: | |
| 113 | + d["usd"] = round(float(d["value"]) * rate, 2) | |
| 114 | + return d | |
| 115 | + | |
| 116 | + | |
| 117 | +def token_meta(chain, address): | |
| 118 | + for t in TOKENS.get(chain, []): | |
| 119 | + if t.get("address", "").lower() == address.lower(): | |
| 120 | + return t | |
| 121 | + return None | |
| 122 | + | |
| 123 | + | |
| 124 | +def block_summary(blk): | |
| 125 | + return { | |
| 126 | + "number": int(blk["number"], 16), | |
| 127 | + "hash": blk["hash"], | |
| 128 | + "parent_hash": blk["parentHash"], | |
| 129 | + "timestamp": int(blk["timestamp"], 16), | |
| 130 | + "tx_count": len(blk.get("transactions", [])), | |
| 131 | + "gas_used": int(blk.get("gasUsed", "0x0"), 16), | |
| 132 | + } | |
| 133 | + | |
| 134 | + | |
| 135 | +# -- meta --------------------------------------------------------------- | |
| 136 | + | |
| 137 | + | |
| 138 | +@app.get("/v1/chains") | |
| 139 | +def chains(): | |
| 140 | + return { | |
| 141 | + chain: { | |
| 142 | + "family": cfg.get("family", "evm"), | |
| 143 | + "chain_id": cfg.get("chain_id"), | |
| 144 | + "block_time_s": cfg.get("block_time"), | |
| 145 | + "confirmations": cfg.get("confirmations"), | |
| 146 | + "tokens": TOKENS.get(chain, []), | |
| 147 | + } | |
| 148 | + for chain, cfg in CHAINS.items() | |
| 149 | + } | |
| 150 | + | |
| 151 | + | |
| 152 | +@app.get("/v1/status") | |
| 153 | +def status(): | |
| 154 | + conn = dbc() | |
| 155 | + try: | |
| 156 | + cursors = { | |
| 157 | + r["chain"]: { | |
| 158 | + "last_block": r["last_block"], | |
| 159 | + "last_hash": r["last_hash"], | |
| 160 | + "backfill_block": r["backfill_block"], | |
| 161 | + "head_block": r["head_block"], | |
| 162 | + "lag": (r["head_block"] - r["last_block"]) | |
| 163 | + if r["head_block"] is not None else None, | |
| 164 | + } | |
| 165 | + for r in conn.execute("SELECT * FROM cursors") | |
| 166 | + if not r["chain"].startswith("_") # internal scan cursors | |
| 167 | + } | |
| 168 | + counts = { | |
| 169 | + r["chain"]: r["n"] | |
| 170 | + for r in conn.execute( | |
| 171 | + "SELECT chain, COUNT(*) AS n FROM transfers GROUP BY chain" | |
| 172 | + ) | |
| 173 | + } | |
| 174 | + return {"cursors": cursors, "indexed_transfers": counts} | |
| 175 | + finally: | |
| 176 | + conn.close() | |
| 177 | + | |
| 178 | + | |
| 179 | +@app.get("/v1/rpc/health") | |
| 180 | +def rpc_health(): | |
| 181 | + """Per-endpoint health as last reported by the indexer workers: | |
| 182 | + score = success-rate EMA, cooldown_s > 0 = currently benched.""" | |
| 183 | + conn = dbc() | |
| 184 | + try: | |
| 185 | + rows = conn.execute( | |
| 186 | + "SELECT * FROM rpc_health ORDER BY chain, score DESC" | |
| 187 | + ).fetchall() | |
| 188 | + finally: | |
| 189 | + conn.close() | |
| 190 | + out = {} | |
| 191 | + for r in rows: | |
| 192 | + out.setdefault(r["chain"], []).append( | |
| 193 | + {k: r[k] for k in ("url", "score", "ok", "fail", "latency_ms", "cooldown_s", "updated")} | |
| 194 | + ) | |
| 195 | + return out | |
| 196 | + | |
| 197 | + | |
| 198 | +@app.get("/metrics") | |
| 199 | +def metrics(): | |
| 200 | + """Prometheus text exposition: per-chain progress/lag + RPC endpoint health. | |
| 201 | + Free-RPC degradation shows up here as rising lag and falling rpc scores.""" | |
| 202 | + from fastapi.responses import PlainTextResponse | |
| 203 | + conn = dbc() | |
| 204 | + try: | |
| 205 | + lines = [ | |
| 206 | + "# HELP explorer_transfers_total Indexed transfers per chain", | |
| 207 | + "# TYPE explorer_transfers_total counter", | |
| 208 | + ] | |
| 209 | + for r in conn.execute("SELECT chain, COUNT(*) n FROM transfers GROUP BY chain"): | |
| 210 | + lines.append(f'explorer_transfers_total{{chain="{r["chain"]}"}} {r["n"]}') | |
| 211 | + lines += [ | |
| 212 | + "# HELP explorer_cursor_block Last indexed block per chain", | |
| 213 | + "# TYPE explorer_cursor_block gauge", | |
| 214 | + "# HELP explorer_lag_blocks Head minus cursor per chain", | |
| 215 | + "# TYPE explorer_lag_blocks gauge", | |
| 216 | + ] | |
| 217 | + for r in conn.execute("SELECT * FROM cursors"): | |
| 218 | + lines.append(f'explorer_cursor_block{{chain="{r["chain"]}"}} {r["last_block"]}') | |
| 219 | + if r["head_block"] is not None: | |
| 220 | + lines.append(f'explorer_lag_blocks{{chain="{r["chain"]}"}} ' | |
| 221 | + f'{max(0, r["head_block"] - r["last_block"])}') | |
| 222 | + lines += [ | |
| 223 | + "# HELP explorer_rpc_score Endpoint health score (success-rate EMA, 0-1)", | |
| 224 | + "# TYPE explorer_rpc_score gauge", | |
| 225 | + "# HELP explorer_rpc_calls_total Successful/failed calls per endpoint", | |
| 226 | + "# TYPE explorer_rpc_calls_total counter", | |
| 227 | + ] | |
| 228 | + for r in conn.execute("SELECT * FROM rpc_health"): | |
| 229 | + lbl = f'chain="{r["chain"]}",url="{r["url"]}"' | |
| 230 | + lines.append(f"explorer_rpc_score{{{lbl}}} {r['score']}") | |
| 231 | + lines.append(f'explorer_rpc_calls_total{{{lbl},result="ok"}} {r["ok"]}') | |
| 232 | + lines.append(f'explorer_rpc_calls_total{{{lbl},result="fail"}} {r["fail"]}') | |
| 233 | + return PlainTextResponse("\n".join(lines) + "\n") | |
| 234 | + finally: | |
| 235 | + conn.close() | |
| 236 | + | |
| 237 | + | |
| 238 | +# -- live chain reads (straight off the free RPCs) ----------------------- | |
| 239 | + | |
| 240 | + | |
| 241 | +@app.get("/v1/{chain}/block/latest") | |
| 242 | +def latest_block(chain: str): | |
| 243 | + blk = pool(chain).call("eth_getBlockByNumber", ["latest", False]) | |
| 244 | + return block_summary(blk) | |
| 245 | + | |
| 246 | + | |
| 247 | +@app.get("/v1/{chain}/block/{number}") | |
| 248 | +def block_by_number(chain: str, number: int): | |
| 249 | + p = pool(chain) | |
| 250 | + blk = p.call("eth_getBlockByNumber", [hex(number), False]) | |
| 251 | + if blk is None: | |
| 252 | + raise HTTPException(404, "block not found") | |
| 253 | + out = block_summary(blk) | |
| 254 | + # stablecoin transfers inside this block, decoded live | |
| 255 | + addrs = [t["address"] for t in TOKENS.get(chain, [])] | |
| 256 | + if addrs: | |
| 257 | + logs = p.call( | |
| 258 | + "eth_getLogs", | |
| 259 | + [{"fromBlock": hex(number), "toBlock": hex(number), | |
| 260 | + "address": addrs, "topics": [TRANSFER_TOPIC]}], | |
| 261 | + ) | |
| 262 | + transfers = [] | |
| 263 | + for l in logs or []: | |
| 264 | + meta = token_meta(chain, l["address"]) | |
| 265 | + if meta and len(l.get("topics", [])) == 3: | |
| 266 | + transfers.append( | |
| 267 | + with_value(decode_transfer(chain, l, meta, out["timestamp"])) | |
| 268 | + ) | |
| 269 | + out["stablecoin_transfers"] = transfers | |
| 270 | + return out | |
| 271 | + | |
| 272 | + | |
| 273 | +@app.get("/v1/{chain}/tx/{tx_hash}") | |
| 274 | +def tx(chain: str, tx_hash: str): | |
| 275 | + if chain in CHAINS and CHAINS[chain].get("family", "evm") != "evm": | |
| 276 | + # non-EVM: serve what the index knows about this tx | |
| 277 | + conn = dbc() | |
| 278 | + try: | |
| 279 | + rows = conn.execute( | |
| 280 | + "SELECT * FROM transfers WHERE chain = ? AND tx_hash IN (?, ?) " | |
| 281 | + "ORDER BY log_index", (chain, tx_hash, tx_hash.lower()), | |
| 282 | + ).fetchall() | |
| 283 | + finally: | |
| 284 | + conn.close() | |
| 285 | + if not rows: | |
| 286 | + raise HTTPException(404, "transaction not in index") | |
| 287 | + first = rows[0] | |
| 288 | + return {"chain": chain, "hash": first["tx_hash"], "block": first["block"], | |
| 289 | + "timestamp": first["timestamp"], "source": "index", | |
| 290 | + "stablecoin_transfers": [with_value(r) for r in rows]} | |
| 291 | + p = pool(chain) | |
| 292 | + t = p.call("eth_getTransactionByHash", [tx_hash]) | |
| 293 | + if t is None: | |
| 294 | + raise HTTPException(404, "transaction not found") | |
| 295 | + receipt = p.call("eth_getTransactionReceipt", [tx_hash]) | |
| 296 | + transfers = [] | |
| 297 | + for l in (receipt or {}).get("logs", []): | |
| 298 | + meta = token_meta(chain, l["address"]) | |
| 299 | + if ( | |
| 300 | + meta | |
| 301 | + and l.get("topics") | |
| 302 | + and l["topics"][0].lower() == TRANSFER_TOPIC | |
| 303 | + and len(l["topics"]) == 3 | |
| 304 | + ): | |
| 305 | + transfers.append(with_value(decode_transfer(chain, l, meta, None))) | |
| 306 | + return { | |
| 307 | + "chain": chain, | |
| 308 | + "hash": t["hash"], | |
| 309 | + "block": int(t["blockNumber"], 16) if t.get("blockNumber") else None, | |
| 310 | + "from": t["from"], | |
| 311 | + "to": t.get("to"), | |
| 312 | + "native_value_wei": str(int(t.get("value", "0x0"), 16)), | |
| 313 | + "status": int(receipt["status"], 16) if receipt and receipt.get("status") else None, | |
| 314 | + "gas_used": int(receipt["gasUsed"], 16) if receipt else None, | |
| 315 | + "stablecoin_transfers": transfers, | |
| 316 | + } | |
| 317 | + | |
| 318 | + | |
| 319 | +# -- indexed queries (from SQLite) --------------------------------------- | |
| 320 | + | |
| 321 | + | |
| 322 | +@app.get("/v1/{chain}/token/{address}/transfers") | |
| 323 | +def token_transfers( | |
| 324 | + chain: str, | |
| 325 | + address: str, | |
| 326 | + from_addr: str | None = Query(None, alias="from"), | |
| 327 | + to_addr: str | None = Query(None, alias="to"), | |
| 328 | + min_amount: float | None = None, | |
| 329 | + since: int | None = Query(None, description="unix timestamp lower bound"), | |
| 330 | + before_block: int | None = Query(None, description="pagination cursor: only blocks below this"), | |
| 331 | + limit: int = Query(50, le=500), | |
| 332 | +): | |
| 333 | + if chain not in CHAINS: | |
| 334 | + raise HTTPException(404, f"unknown chain '{chain}'") | |
| 335 | + # EVM identifiers are case-insensitive hex (stored lowercase); Base58 / | |
| 336 | + # coin-type identifiers on other chains are case-sensitive — keep verbatim | |
| 337 | + evm = CHAINS[chain].get("family", "evm") == "evm" | |
| 338 | + norm = (lambda s: s.lower()) if evm else (lambda s: s) | |
| 339 | + q = SELECT_T + " WHERE chain = ? AND token = ?" | |
| 340 | + args = [chain, norm(address)] | |
| 341 | + if from_addr: | |
| 342 | + q += ' AND "from" = ?' | |
| 343 | + args.append(norm(from_addr)) | |
| 344 | + if to_addr: | |
| 345 | + q += ' AND "to" = ?' | |
| 346 | + args.append(norm(to_addr)) | |
| 347 | + if min_amount is not None: | |
| 348 | + meta = token_meta(chain, address) | |
| 349 | + decimals = meta["decimals"] if meta else 6 | |
| 350 | + # CAST for comparison: amounts are TEXT (uint256 overflows int64); | |
| 351 | + # DOUBLE PRECISION works on PG and maps to REAL affinity on SQLite | |
| 352 | + q += " AND CAST(amount AS DOUBLE PRECISION) >= ?" | |
| 353 | + args.append(min_amount * 10**decimals) | |
| 354 | + if since is not None: | |
| 355 | + q += " AND timestamp >= ?" | |
| 356 | + args.append(since) | |
| 357 | + if before_block is not None: | |
| 358 | + q += " AND block < ?" | |
| 359 | + args.append(before_block) | |
| 360 | + q += " ORDER BY block DESC, log_index DESC LIMIT ?" | |
| 361 | + args.append(limit) | |
| 362 | + conn = dbc() | |
| 363 | + try: | |
| 364 | + return [with_value(r) for r in conn.execute(q, args)] | |
| 365 | + finally: | |
| 366 | + conn.close() | |
| 367 | + | |
| 368 | + | |
| 369 | +@app.get("/v1/address/{addr}/transfers") | |
| 370 | +def address_transfers(addr: str, limit: int = Query(50, le=500)): | |
| 371 | + """Cross-chain: every indexed stablecoin transfer touching this address.""" | |
| 372 | + conn = dbc() | |
| 373 | + try: | |
| 374 | + # match both verbatim (Base58/native formats) and lowercased (EVM hex) | |
| 375 | + rows = conn.execute( | |
| 376 | + SELECT_T + ' WHERE "from" IN (?, ?) OR "to" IN (?, ?) ' | |
| 377 | + "ORDER BY timestamp DESC LIMIT ?", | |
| 378 | + (addr, addr.lower(), addr, addr.lower(), limit), | |
| 379 | + ) | |
| 380 | + return [with_value(r) for r in rows] | |
| 381 | + finally: | |
| 382 | + conn.close() | |
| 383 | + | |
| 384 | + | |
| 385 | +@app.get("/v1/stablecoins/volume") | |
| 386 | +@cached(ttl=30) | |
| 387 | +def volume(token: str = "USDT", window: str = "24h"): | |
| 388 | + seconds = parse_window(window) | |
| 389 | + since = int(time.time()) - seconds | |
| 390 | + conn = dbc() | |
| 391 | + try: | |
| 392 | + # serve from the StatsWorker's precomputed table when fresh — the | |
| 393 | + # COUNT(DISTINCT) work does not scale on the request path | |
| 394 | + agg = conn.execute( | |
| 395 | + "SELECT * FROM agg_volume WHERE window = ? AND symbol = ?", | |
| 396 | + (window, token.upper()), | |
| 397 | + ).fetchall() | |
| 398 | + if agg and int(time.time()) - agg[0]["updated"] < 900: | |
| 399 | + per_chain = { | |
| 400 | + r["chain"]: {"transfers": r["transfers"], "volume": r["volume"], | |
| 401 | + "senders": r["senders"], "receivers": r["receivers"]} | |
| 402 | + for r in agg | |
| 403 | + } | |
| 404 | + return { | |
| 405 | + "token": token.upper(), "window": window, "since_unix": since, | |
| 406 | + "as_of": agg[0]["updated"], | |
| 407 | + "total_volume": round(sum(r["volume"] for r in agg), 2), | |
| 408 | + "chains": per_chain, | |
| 409 | + "note": "volume only covers blocks this instance has indexed", | |
| 410 | + } | |
| 411 | + # fallback (fresh deploy / unusual window): live scan WITHOUT the | |
| 412 | + # distinct-address counts, which are what makes this query heavy | |
| 413 | + rows = conn.execute( | |
| 414 | + "SELECT chain, decimals, COUNT(*) AS transfers, " | |
| 415 | + "SUM(CAST(amount AS DOUBLE PRECISION)) AS raw_sum " | |
| 416 | + "FROM transfers WHERE symbol = ? AND timestamp >= ? " | |
| 417 | + "GROUP BY chain, decimals", | |
| 418 | + (token.upper(), since), | |
| 419 | + ).fetchall() | |
| 420 | + price = price_of(conn, token.upper()) | |
| 421 | + finally: | |
| 422 | + conn.close() | |
| 423 | + per_chain = {} | |
| 424 | + total = 0.0 | |
| 425 | + for r in rows: | |
| 426 | + vol = (r["raw_sum"] or 0) / 10 ** r["decimals"] * price | |
| 427 | + entry = per_chain.setdefault( | |
| 428 | + r["chain"], {"transfers": 0, "volume": 0.0, "senders": None, "receivers": None} | |
| 429 | + ) | |
| 430 | + entry["transfers"] += r["transfers"] | |
| 431 | + entry["volume"] = round(entry["volume"] + vol, 2) | |
| 432 | + total += vol | |
| 433 | + return { | |
| 434 | + "token": token.upper(), | |
| 435 | + "window": window, | |
| 436 | + "since_unix": since, | |
| 437 | + "total_volume": round(total, 2), | |
| 438 | + "chains": per_chain, | |
| 439 | + "note": "volume only covers blocks this instance has indexed", | |
| 440 | + } | |
| 441 | + | |
| 442 | + | |
| 443 | +@app.get("/v1/tokens") | |
| 444 | +@cached(ttl=300) | |
| 445 | +def tokens(): | |
| 446 | + """Every configured stablecoin, grouped by canonical symbol.""" | |
| 447 | + out = {} | |
| 448 | + for chain, toks in TOKENS.items(): | |
| 449 | + for t in toks: | |
| 450 | + entry = out.setdefault(t["symbol"], { | |
| 451 | + "category": t.get("category", "stablecoin"), "chains": {}}) | |
| 452 | + entry["chains"][chain] = { | |
| 453 | + "id": t.get("address") or t.get("id"), | |
| 454 | + "decimals": t.get("decimals"), | |
| 455 | + "native": bool(t.get("native")), | |
| 456 | + "discontinued": bool(t.get("discontinued", False)), | |
| 457 | + "onchain_symbol": t.get("onchain_symbol", t["symbol"]), | |
| 458 | + } | |
| 459 | + return out | |
| 460 | + | |
| 461 | + | |
| 462 | +@app.get("/v1/prices") | |
| 463 | +@cached(ttl=30) | |
| 464 | +def prices(): | |
| 465 | + """Latest USD prices used for valuation (stablecoins seeded at 1.0).""" | |
| 466 | + conn = dbc() | |
| 467 | + try: | |
| 468 | + return { | |
| 469 | + r["symbol"]: {"usd": r["usd"], "updated": r["updated"]} | |
| 470 | + for r in conn.execute("SELECT * FROM prices ORDER BY symbol") | |
| 471 | + } | |
| 472 | + finally: | |
| 473 | + conn.close() | |
| 474 | + | |
| 475 | + | |
| 476 | +@app.get("/v1/stablecoins/supply") | |
| 477 | +@cached(ttl=60) | |
| 478 | +def supply(token: str | None = None): | |
| 479 | + """Latest on-chain supply snapshot per (chain, token), + totals.""" | |
| 480 | + conn = dbc() | |
| 481 | + try: | |
| 482 | + q = ( | |
| 483 | + "SELECT s.chain, s.token, s.symbol, s.supply, s.decimals, s.timestamp " | |
| 484 | + "FROM supply_snapshots s JOIN (" | |
| 485 | + " SELECT chain, token, MAX(timestamp) AS mt FROM supply_snapshots " | |
| 486 | + " GROUP BY chain, token) m " | |
| 487 | + "ON s.chain = m.chain AND s.token = m.token AND s.timestamp = m.mt" | |
| 488 | + ) | |
| 489 | + args = () | |
| 490 | + if token: | |
| 491 | + q += " WHERE s.symbol = ?" | |
| 492 | + args = (token.upper(),) | |
| 493 | + rows = conn.execute(q, args).fetchall() | |
| 494 | + prices = {r["symbol"]: r["usd"] for r in conn.execute("SELECT symbol, usd FROM prices")} | |
| 495 | + finally: | |
| 496 | + conn.close() | |
| 497 | + per_symbol = {} | |
| 498 | + for r in rows: | |
| 499 | + supply_h = float(format_amount(r["supply"], r["decimals"] or 6)) | |
| 500 | + price = prices.get(r["symbol"]) or 1.0 | |
| 501 | + sym = per_symbol.setdefault(r["symbol"], {"total": 0.0, "total_usd": 0.0, "chains": {}}) | |
| 502 | + sym["chains"][r["chain"]] = { | |
| 503 | + "supply": supply_h, "usd": round(supply_h * price, 2), "raw": r["supply"], | |
| 504 | + "decimals": r["decimals"], "as_of": r["timestamp"], | |
| 505 | + } | |
| 506 | + sym["total"] = round(sym["total"] + supply_h, 2) | |
| 507 | + sym["total_usd"] = round(sym["total_usd"] + supply_h * price, 2) | |
| 508 | + return per_symbol | |
| 509 | + | |
| 510 | + | |
| 511 | +@app.get("/v1/stablecoins/whales") | |
| 512 | +def whales(token: str | None = None, window: str = "24h", | |
| 513 | + min_usd: float | None = None, limit: int = Query(50, le=500)): | |
| 514 | + """Largest transfers in the window (face value ≈ USD for stablecoins).""" | |
| 515 | + threshold = min_usd if min_usd is not None else DEFAULT_WHALE_USD | |
| 516 | + since = int(time.time()) - parse_window(window) | |
| 517 | + # whale_events is maintained incrementally by StatsWorker (floor $100K) — | |
| 518 | + # reading it is O(window) instead of a full transfers scan | |
| 519 | + q = "SELECT * FROM whale_events WHERE timestamp >= ? AND usd >= ?" | |
| 520 | + args = [since, threshold] | |
| 521 | + if token: | |
| 522 | + q += " AND symbol = ?" | |
| 523 | + args.append(token.upper()) | |
| 524 | + q += " ORDER BY usd DESC LIMIT ?" | |
| 525 | + args.append(limit) | |
| 526 | + conn = dbc() | |
| 527 | + try: | |
| 528 | + return [with_value(r) for r in conn.execute(q, args)] | |
| 529 | + finally: | |
| 530 | + conn.close() | |
| 531 | + | |
| 532 | + | |
| 533 | +@app.get("/v1/stablecoins/mints-burns") | |
| 534 | +def mints_burns(token: str | None = None, window: str = "24h", | |
| 535 | + limit: int = Query(100, le=500)): | |
| 536 | + """Issuance events: transfers from the zero address are mints, to it are | |
| 537 | + burns (EVM + Tron; Solana mints don't transit an address — see docs).""" | |
| 538 | + zeros = tuple(ZERO_ADDRESSES.values()) | |
| 539 | + since = int(time.time()) - parse_window(window) | |
| 540 | + zp = ", ".join("?" for _ in zeros) | |
| 541 | + q = (SELECT_T + ' WHERE timestamp >= ? ' | |
| 542 | + f'AND ("from" IN ({zp}) OR "to" IN ({zp}))') | |
| 543 | + args = [since, *zeros, *zeros] | |
| 544 | + if token: | |
| 545 | + q += " AND transfers.symbol = ?" | |
| 546 | + args.append(token.upper()) | |
| 547 | + q += " ORDER BY timestamp DESC LIMIT ?" | |
| 548 | + args.append(limit) | |
| 549 | + conn = dbc() | |
| 550 | + try: | |
| 551 | + out = [] | |
| 552 | + for r in conn.execute(q, args): | |
| 553 | + d = with_value(r) | |
| 554 | + d["direction"] = "mint" if r["from"] in zeros else "burn" | |
| 555 | + out.append(d) | |
| 556 | + return out | |
| 557 | + finally: | |
| 558 | + conn.close() | |
| 559 | + | |
| 560 | + | |
| 561 | +@app.get("/v1/stablecoins/volume/series") | |
| 562 | +@cached(ttl=60) | |
| 563 | +def volume_series(token: str = "USDT", window: str = "24h", | |
| 564 | + interval: str = "1h", chain: str | None = None): | |
| 565 | + """Bucketed transfer volume over time — per chain, for line charts. | |
| 566 | + Served from StatsWorker's precomputed buckets for the standard windows.""" | |
| 567 | + since = int(time.time()) - parse_window(window) | |
| 568 | + step = parse_window(interval) | |
| 569 | + conn0 = dbc() | |
| 570 | + try: | |
| 571 | + agg = conn0.execute( | |
| 572 | + "SELECT * FROM agg_series WHERE window = ? AND symbol = ? " | |
| 573 | + + ("AND chain = ? " if chain else "") + "ORDER BY t", | |
| 574 | + (window, token.upper(), *([chain] if chain else [])), | |
| 575 | + ).fetchall() | |
| 576 | + finally: | |
| 577 | + conn0.close() | |
| 578 | + if agg and int(time.time()) - agg[0]["updated"] < 900: | |
| 579 | + out = {} | |
| 580 | + for r in agg: | |
| 581 | + out.setdefault(r["chain"], []).append( | |
| 582 | + {"t": r["t"], "volume": r["volume"], "transfers": r["transfers"]}) | |
| 583 | + worker_step = {"1h": 300, "24h": 3600, "7d": 21600}.get(window, step) | |
| 584 | + return {"token": token.upper(), "since": since, "interval_s": worker_step, | |
| 585 | + "as_of": agg[0]["updated"], "chains": out} | |
| 586 | + q = (f"SELECT (timestamp / {step}) * {step} AS t, chain, decimals, " | |
| 587 | + "COUNT(*) AS transfers, SUM(CAST(amount AS DOUBLE PRECISION)) AS raw_sum " | |
| 588 | + "FROM transfers WHERE symbol = ? AND timestamp >= ?") | |
| 589 | + args = [token.upper(), since] | |
| 590 | + if chain: | |
| 591 | + q += " AND chain = ?" | |
| 592 | + args.append(chain) | |
| 593 | + q += " GROUP BY t, chain, decimals ORDER BY t" | |
| 594 | + conn = dbc() | |
| 595 | + try: | |
| 596 | + rows = conn.execute(q, args).fetchall() | |
| 597 | + price = price_of(conn, token.upper()) | |
| 598 | + finally: | |
| 599 | + conn.close() | |
| 600 | + out = {} | |
| 601 | + for r in rows: | |
| 602 | + c = out.setdefault(r["chain"], {}) | |
| 603 | + b = c.setdefault(r["t"], {"volume": 0.0, "transfers": 0}) | |
| 604 | + b["volume"] = round(b["volume"] + (r["raw_sum"] or 0) / 10 ** r["decimals"] * price, 2) | |
| 605 | + b["transfers"] += r["transfers"] | |
| 606 | + return { | |
| 607 | + "token": token.upper(), "since": since, "interval_s": step, | |
| 608 | + "chains": { | |
| 609 | + c: [{"t": t, **v} for t, v in sorted(buckets.items())] | |
| 610 | + for c, buckets in out.items() | |
| 611 | + }, | |
| 612 | + } | |
| 613 | + | |
| 614 | + | |
| 615 | +@app.get("/v1/stablecoins/supply/series") | |
| 616 | +@cached(ttl=120) | |
| 617 | +def supply_series(token: str = "USDT", window: str = "7d"): | |
| 618 | + """Supply snapshots over time per chain (hourly cadence).""" | |
| 619 | + since = int(time.time()) - parse_window(window) | |
| 620 | + conn = dbc() | |
| 621 | + try: | |
| 622 | + rows = conn.execute( | |
| 623 | + "SELECT chain, supply, decimals, timestamp FROM supply_snapshots " | |
| 624 | + "WHERE symbol = ? AND timestamp >= ? ORDER BY timestamp", | |
| 625 | + (token.upper(), since), | |
| 626 | + ).fetchall() | |
| 627 | + finally: | |
| 628 | + conn.close() | |
| 629 | + out = {} | |
| 630 | + for r in rows: | |
| 631 | + out.setdefault(r["chain"], []).append( | |
| 632 | + {"t": r["timestamp"], | |
| 633 | + "supply": float(format_amount(r["supply"], r["decimals"] or 6))} | |
| 634 | + ) | |
| 635 | + return {"token": token.upper(), "since": since, "chains": out} | |
| 636 | + | |
| 637 | + | |
| 638 | +@app.get("/v1/stablecoins/flows") | |
| 639 | +@cached(ttl=60) | |
| 640 | +def flows(token: str | None = None, window: str = "7d", interval: str = "1d"): | |
| 641 | + """Net issuance over time: mints (from zero addr) minus burns (to zero).""" | |
| 642 | + zeros = tuple(ZERO_ADDRESSES.values()) | |
| 643 | + since = int(time.time()) - parse_window(window) | |
| 644 | + step = parse_window(interval) | |
| 645 | + zp = ", ".join("?" for _ in zeros) | |
| 646 | + q = (f"SELECT (timestamp / {step}) * {step} AS t, transfers.symbol AS symbol, " | |
| 647 | + f'SUM(CASE WHEN "from" IN ({zp}) THEN {USD_PRICED} ELSE 0 END) AS minted, ' | |
| 648 | + f'SUM(CASE WHEN "to" IN ({zp}) THEN {USD_PRICED} ELSE 0 END) AS burned ' | |
| 649 | + f'FROM transfers {PRICE_JOIN} ' | |
| 650 | + f'WHERE timestamp >= ? AND ("from" IN ({zp}) OR "to" IN ({zp}))') | |
| 651 | + args = [*zeros, *zeros, since, *zeros, *zeros] | |
| 652 | + if token: | |
| 653 | + q += " AND transfers.symbol = ?" | |
| 654 | + args.append(token.upper()) | |
| 655 | + q += " GROUP BY t, transfers.symbol ORDER BY t" | |
| 656 | + conn = dbc() | |
| 657 | + try: | |
| 658 | + rows = conn.execute(q, args).fetchall() | |
| 659 | + finally: | |
| 660 | + conn.close() | |
| 661 | + out = {} | |
| 662 | + for r in rows: | |
| 663 | + out.setdefault(r["symbol"], []).append({ | |
| 664 | + "t": r["t"], "minted": round(r["minted"] or 0, 2), | |
| 665 | + "burned": round(r["burned"] or 0, 2), | |
| 666 | + "net": round((r["minted"] or 0) - (r["burned"] or 0), 2), | |
| 667 | + }) | |
| 668 | + return {"since": since, "interval_s": step, "tokens": out} | |
| 669 | + | |
| 670 | + | |
| 671 | +@app.get("/v1/{chain}/transfers") | |
| 672 | +def chain_transfers(chain: str, symbol: str | None = None, | |
| 673 | + before_block: int | None = None, | |
| 674 | + min_amount: float | None = None, | |
| 675 | + limit: int = Query(50, le=500)): | |
| 676 | + """Recent indexed transfers on one chain, all tokens (paginated).""" | |
| 677 | + if chain not in CHAINS: | |
| 678 | + raise HTTPException(404, f"unknown chain '{chain}'") | |
| 679 | + q = SELECT_T + " WHERE chain = ?" | |
| 680 | + args = [chain] | |
| 681 | + if symbol: | |
| 682 | + q += " AND transfers.symbol = ?" | |
| 683 | + args.append(symbol.upper()) | |
| 684 | + if min_amount is not None: | |
| 685 | + q += f" AND {USD_PRICED} >= ?" | |
| 686 | + args.append(min_amount) | |
| 687 | + if before_block is not None: | |
| 688 | + q += " AND block < ?" | |
| 689 | + args.append(before_block) | |
| 690 | + q += " ORDER BY block DESC, log_index DESC LIMIT ?" | |
| 691 | + args.append(limit) | |
| 692 | + conn = dbc() | |
| 693 | + try: | |
| 694 | + return [with_value(r) for r in conn.execute(q, args)] | |
| 695 | + finally: | |
| 696 | + conn.close() | |
| 697 | + | |
| 698 | + | |
| 699 | +@app.get("/v1/{chain}/summary") | |
| 700 | +@cached(ttl=30) | |
| 701 | +def chain_summary(chain: str): | |
| 702 | + if chain not in CHAINS: | |
| 703 | + raise HTTPException(404, f"unknown chain '{chain}'") | |
| 704 | + since = int(time.time()) - 86400 | |
| 705 | + conn = dbc() | |
| 706 | + try: | |
| 707 | + vol = conn.execute( | |
| 708 | + f"SELECT transfers.symbol AS symbol, COUNT(*) AS transfers, " | |
| 709 | + f"SUM({USD_PRICED}) AS volume " | |
| 710 | + f"FROM transfers {PRICE_JOIN} " | |
| 711 | + "WHERE chain = ? AND timestamp >= ? GROUP BY transfers.symbol " | |
| 712 | + "ORDER BY volume DESC", | |
| 713 | + (chain, since), | |
| 714 | + ).fetchall() | |
| 715 | + cur = conn.execute("SELECT * FROM cursors WHERE chain = ?", (chain,)).fetchone() | |
| 716 | + total = conn.execute( | |
| 717 | + "SELECT COUNT(*) AS n FROM transfers WHERE chain = ?", (chain,) | |
| 718 | + ).fetchone() | |
| 719 | + finally: | |
| 720 | + conn.close() | |
| 721 | + cfg = CHAINS[chain] | |
| 722 | + return { | |
| 723 | + "chain": chain, | |
| 724 | + "family": cfg.get("family", "evm"), | |
| 725 | + "chain_id": cfg.get("chain_id"), | |
| 726 | + "block_time_s": cfg.get("block_time"), | |
| 727 | + "tokens": TOKENS.get(chain, []), | |
| 728 | + "indexed_transfers": total["n"] if total else 0, | |
| 729 | + "volume_24h": [ | |
| 730 | + {"symbol": r["symbol"], "transfers": r["transfers"], | |
| 731 | + "volume": round(r["volume"] or 0, 2)} for r in vol | |
| 732 | + ], | |
| 733 | + "cursor": dict(cur) if cur else None, | |
| 734 | + } | |
| 735 | + | |
| 736 | + | |
| 737 | +@app.get("/v1/search") | |
| 738 | +def search(q: str): | |
| 739 | + """Classify a query: tx hash / address / token symbol / chain name. | |
| 740 | + Checks the index first (so we can say WHICH chain a tx lives on).""" | |
| 741 | + s = q.strip() | |
| 742 | + if not s: | |
| 743 | + raise HTTPException(400, "empty query") | |
| 744 | + if s.upper() in {t["symbol"] for toks in TOKENS.values() for t in toks}: | |
| 745 | + return {"type": "token", "symbol": s.upper()} | |
| 746 | + if s.lower() in CHAINS: | |
| 747 | + return {"type": "chain", "chain": s.lower()} | |
| 748 | + conn = dbc() | |
| 749 | + try: | |
| 750 | + needles = (s, s.lower()) | |
| 751 | + r = conn.execute( | |
| 752 | + "SELECT chain, tx_hash FROM transfers WHERE tx_hash IN (?, ?) LIMIT 1", needles | |
| 753 | + ).fetchone() | |
| 754 | + if r: | |
| 755 | + return {"type": "tx", "chain": r["chain"], "hash": r["tx_hash"]} | |
| 756 | + r = conn.execute( | |
| 757 | + 'SELECT COUNT(*) AS n FROM transfers WHERE "from" IN (?, ?) OR "to" IN (?, ?)', | |
| 758 | + needles + needles, | |
| 759 | + ).fetchone() | |
| 760 | + if r and r["n"]: | |
| 761 | + return {"type": "address", "address": s, "indexed_transfers": r["n"]} | |
| 762 | + finally: | |
| 763 | + conn.close() | |
| 764 | + # shape-based fallback for things we haven't indexed (yet) | |
| 765 | + import re | |
| 766 | + if re.fullmatch(r"(0x)?[0-9a-fA-F]{64}", s) or re.fullmatch(r"[1-9A-HJ-NP-Za-km-z]{80,90}", s): | |
| 767 | + return {"type": "tx", "chain": None, "hash": s} | |
| 768 | + if re.fullmatch(r"0x[0-9a-fA-F]{40}", s) or re.fullmatch(r"T[1-9A-HJ-NP-Za-km-z]{33}", s) \ | |
| 769 | + or re.fullmatch(r"[1-9A-HJ-NP-Za-km-z]{32,44}", s): | |
| 770 | + return {"type": "address", "address": s, "indexed_transfers": 0} | |
| 771 | + return {"type": "unknown"} | |
| 772 | + | |
| 773 | + | |
| 774 | +# -- live stream --------------------------------------------------------- | |
| 775 | + | |
| 776 | + | |
| 777 | +def _poll_transfers(since_ts): | |
| 778 | + conn = dbc() | |
| 779 | + try: | |
| 780 | + rows = conn.execute( | |
| 781 | + SELECT_T + " WHERE timestamp >= ? ORDER BY timestamp ASC LIMIT 1000", | |
| 782 | + (since_ts,) | |
| 783 | + ).fetchall() | |
| 784 | + return [with_value(r) for r in rows] | |
| 785 | + finally: | |
| 786 | + conn.close() | |
| 787 | + | |
| 788 | + | |
| 789 | +@app.websocket("/v1/stream/transfers") | |
| 790 | +async def stream_transfers(ws: WebSocket): | |
| 791 | + """Pushes new transfers every ~2s. Optional query params: | |
| 792 | + token=USDT chain=ethereum min_usd=1000""" | |
| 793 | + await ws.accept() | |
| 794 | + token = (ws.query_params.get("token") or "").upper() or None | |
| 795 | + chain = ws.query_params.get("chain") | |
| 796 | + min_usd = float(ws.query_params.get("min_usd") or 0) | |
| 797 | + last_ts = int(time.time()) - 2 | |
| 798 | + seen = set() | |
| 799 | + try: | |
| 800 | + while True: | |
| 801 | + rows = await asyncio.to_thread(_poll_transfers, last_ts) | |
| 802 | + batch = [] | |
| 803 | + for r in rows: | |
| 804 | + key = (r["chain"], r["tx_hash"], r["log_index"]) | |
| 805 | + if key in seen: | |
| 806 | + continue | |
| 807 | + if token and r["symbol"] != token: | |
| 808 | + seen.add(key) | |
| 809 | + continue | |
| 810 | + if chain and r["chain"] != chain: | |
| 811 | + seen.add(key) | |
| 812 | + continue | |
| 813 | + if min_usd and r.get("usd", float(r["value"])) < min_usd: | |
| 814 | + seen.add(key) | |
| 815 | + continue | |
| 816 | + seen.add(key) | |
| 817 | + batch.append(r) | |
| 818 | + if batch: | |
| 819 | + await ws.send_json(batch) | |
| 820 | + if rows: | |
| 821 | + new_last = max(r["timestamp"] or last_ts for r in rows) | |
| 822 | + if new_last > last_ts: | |
| 823 | + last_ts = new_last | |
| 824 | + # only remember keys that can still reappear in queries | |
| 825 | + seen = {k for k in seen} if len(seen) < 50_000 else set() | |
| 826 | + await asyncio.sleep(2) | |
| 827 | + except WebSocketDisconnect: | |
| 828 | + pass | |
| 829 | + | |
| 830 | + | |
| 831 | +# -- dashboard (must mount last: "/" catches everything below the API routes) -- | |
| 832 | +app.mount("/", StaticFiles(directory=str(pathlib.Path(__file__).parent.parent / "ui"), html=True), name="ui") | |
added
config/chains.yaml
+418 −0
@@ -0,0 +1,418 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +# Free public RPC endpoints — no API keys anywhere. All endpoints below were | |
| 4 | +# live-verified 2026-08-05 (EVM: eth_chainId; others: family-native health | |
| 5 | +# calls). The RpcPool rotates to the next endpoint on 429s/transport errors | |
| 6 | +# and on semantic getLogs refusals at minimum range. | |
| 7 | +# | |
| 8 | +# family: picks the ingestion adapter (evm today; others land in D4) | |
| 9 | +# confirmations: blocks to trail behind head — cheap reorg protection | |
| 10 | +# max_range: initial eth_getLogs block span; auto-halves when a node | |
| 11 | +# answers "too many results / range too large" | |
| 12 | +# start_offset: how far behind the safe head to start on a fresh DB | |
| 13 | +# (full historical backfill on free RPCs is impractical — | |
| 14 | +# start recent, grow forward) | |
| 15 | +# | |
| 16 | +# Notes from endpoint survey (2026-08-05): | |
| 17 | +# - 1rpc.io free tier is effectively dead ("usage limit" errors) — removed | |
| 18 | +# - rpc.ankr.com requires keys on most chains now — avoided | |
| 19 | +# - dRPC public: ~120k CU/min per IP, no Solana on free plan | |
| 20 | +# - PublicNode: keyless fair-use, but no ZKsync/Kaia/WorldChain/HyperEVM | |
| 21 | + | |
| 22 | +chains: | |
| 23 | + # ======================= EVM ======================= | |
| 24 | + ethereum: | |
| 25 | + native: {symbol: ETH, min: 50, decimals: 18} # whale-only native transfers | |
| 26 | + family: evm | |
| 27 | + chain_id: 1 | |
| 28 | + block_time: 12 | |
| 29 | + confirmations: 6 | |
| 30 | + max_range: 40 | |
| 31 | + start_offset: 300 # ~1 hour | |
| 32 | + rpcs: | |
| 33 | + - https://ethereum-rpc.publicnode.com | |
| 34 | + - https://eth.drpc.org | |
| 35 | + # eth.llamarpc.com dropped 2026-08-05: origin down (HTTP 521) | |
| 36 | + | |
| 37 | + base: | |
| 38 | + family: evm | |
| 39 | + chain_id: 8453 | |
| 40 | + block_time: 2 | |
| 41 | + confirmations: 10 | |
| 42 | + max_range: 150 | |
| 43 | + start_offset: 1800 # ~1 hour | |
| 44 | + rpcs: | |
| 45 | + - https://mainnet.base.org | |
| 46 | + - https://base-rpc.publicnode.com | |
| 47 | + - https://base.drpc.org | |
| 48 | + | |
| 49 | + arbitrum: | |
| 50 | + family: evm | |
| 51 | + chain_id: 42161 | |
| 52 | + block_time: 0.25 | |
| 53 | + confirmations: 40 | |
| 54 | + max_range: 400 | |
| 55 | + start_offset: 8000 # ~30 min (fast blocks) | |
| 56 | + rpcs: | |
| 57 | + - https://arb1.arbitrum.io/rpc | |
| 58 | + - https://arbitrum-one-rpc.publicnode.com | |
| 59 | + - https://arbitrum.drpc.org | |
| 60 | + | |
| 61 | + optimism: | |
| 62 | + family: evm | |
| 63 | + chain_id: 10 | |
| 64 | + block_time: 2 | |
| 65 | + confirmations: 10 | |
| 66 | + max_range: 150 | |
| 67 | + start_offset: 1800 | |
| 68 | + rpcs: | |
| 69 | + - https://mainnet.optimism.io | |
| 70 | + - https://optimism-rpc.publicnode.com | |
| 71 | + - https://optimism.drpc.org | |
| 72 | + | |
| 73 | + polygon: | |
| 74 | + native: {symbol: POL, min: 300000, decimals: 18} | |
| 75 | + family: evm | |
| 76 | + chain_id: 137 | |
| 77 | + block_time: 2 | |
| 78 | + confirmations: 30 # Polygon PoS reorgs more than most | |
| 79 | + max_range: 100 | |
| 80 | + start_offset: 1800 | |
| 81 | + rpcs: | |
| 82 | + - https://polygon-bor-rpc.publicnode.com | |
| 83 | + - https://polygon.drpc.org | |
| 84 | + # polygon-rpc.com dropped 2026-08-05: now answers "API key disabled" | |
| 85 | + | |
| 86 | + bsc: | |
| 87 | + native: {symbol: BNB, min: 500, decimals: 18} | |
| 88 | + family: evm | |
| 89 | + chain_id: 56 | |
| 90 | + block_time: 0.75 | |
| 91 | + confirmations: 15 | |
| 92 | + max_range: 100 | |
| 93 | + start_offset: 2400 # ~30 min | |
| 94 | + rpcs: | |
| 95 | + - https://bsc-rpc.publicnode.com # first: dataseed refuses getLogs | |
| 96 | + - https://bsc-dataseed.bnbchain.org | |
| 97 | + - https://bsc.drpc.org | |
| 98 | + | |
| 99 | + avalanche: | |
| 100 | + native: {symbol: AVAX, min: 5000, decimals: 18} | |
| 101 | + family: evm | |
| 102 | + chain_id: 43114 | |
| 103 | + block_time: 2 | |
| 104 | + confirmations: 5 # Snowman consensus finalizes fast | |
| 105 | + max_range: 200 | |
| 106 | + start_offset: 1800 | |
| 107 | + rpcs: | |
| 108 | + - https://api.avax.network/ext/bc/C/rpc | |
| 109 | + - https://avalanche-c-chain-rpc.publicnode.com | |
| 110 | + - https://avalanche.drpc.org | |
| 111 | + | |
| 112 | + celo: | |
| 113 | + family: evm | |
| 114 | + chain_id: 42220 | |
| 115 | + block_time: 1 # Celo is an OP-stack L2 now | |
| 116 | + confirmations: 10 | |
| 117 | + max_range: 200 | |
| 118 | + start_offset: 3600 | |
| 119 | + rpcs: | |
| 120 | + - https://forno.celo.org | |
| 121 | + - https://celo-rpc.publicnode.com | |
| 122 | + - https://celo.drpc.org | |
| 123 | + | |
| 124 | + kaia: | |
| 125 | + family: evm | |
| 126 | + chain_id: 8217 | |
| 127 | + block_time: 1 | |
| 128 | + confirmations: 10 | |
| 129 | + max_range: 200 | |
| 130 | + start_offset: 3600 | |
| 131 | + rpcs: | |
| 132 | + - https://public-en.node.kaia.io | |
| 133 | + - https://kaia.drpc.org | |
| 134 | + | |
| 135 | + ink: | |
| 136 | + family: evm | |
| 137 | + chain_id: 57073 | |
| 138 | + block_time: 1 # OP stack | |
| 139 | + confirmations: 10 | |
| 140 | + max_range: 200 | |
| 141 | + start_offset: 3600 | |
| 142 | + rpcs: | |
| 143 | + - https://rpc-gel.inkonchain.com | |
| 144 | + - https://rpc-qnd.inkonchain.com | |
| 145 | + - https://ink.drpc.org | |
| 146 | + | |
| 147 | + scroll: | |
| 148 | + family: evm | |
| 149 | + chain_id: 534352 | |
| 150 | + block_time: 3 | |
| 151 | + confirmations: 10 | |
| 152 | + max_range: 150 | |
| 153 | + start_offset: 1200 | |
| 154 | + rpcs: | |
| 155 | + - https://rpc.scroll.io | |
| 156 | + - https://scroll-rpc.publicnode.com | |
| 157 | + - https://scroll.drpc.org | |
| 158 | + | |
| 159 | + linea: | |
| 160 | + family: evm | |
| 161 | + chain_id: 59144 | |
| 162 | + block_time: 2 | |
| 163 | + confirmations: 10 | |
| 164 | + max_range: 150 | |
| 165 | + start_offset: 1800 | |
| 166 | + rpcs: | |
| 167 | + - https://rpc.linea.build | |
| 168 | + - https://linea-rpc.publicnode.com | |
| 169 | + - https://linea.drpc.org | |
| 170 | + | |
| 171 | + mantle: | |
| 172 | + family: evm | |
| 173 | + chain_id: 5000 | |
| 174 | + block_time: 2 | |
| 175 | + confirmations: 10 | |
| 176 | + max_range: 150 | |
| 177 | + start_offset: 1800 | |
| 178 | + rpcs: | |
| 179 | + - https://rpc.mantle.xyz | |
| 180 | + - https://mantle-rpc.publicnode.com | |
| 181 | + - https://mantle.drpc.org | |
| 182 | + | |
| 183 | + gnosis: | |
| 184 | + family: evm | |
| 185 | + chain_id: 100 | |
| 186 | + block_time: 5 | |
| 187 | + confirmations: 10 | |
| 188 | + max_range: 100 | |
| 189 | + start_offset: 720 | |
| 190 | + rpcs: | |
| 191 | + - https://rpc.gnosischain.com | |
| 192 | + - https://gnosis-rpc.publicnode.com | |
| 193 | + - https://gnosis.drpc.org | |
| 194 | + | |
| 195 | + zksync: | |
| 196 | + family: evm | |
| 197 | + chain_id: 324 | |
| 198 | + block_time: 1 | |
| 199 | + confirmations: 10 | |
| 200 | + max_range: 200 | |
| 201 | + start_offset: 3600 | |
| 202 | + rpcs: | |
| 203 | + - https://mainnet.era.zksync.io | |
| 204 | + - https://zksync.drpc.org | |
| 205 | + | |
| 206 | + blast: | |
| 207 | + family: evm | |
| 208 | + chain_id: 81457 | |
| 209 | + block_time: 2 | |
| 210 | + confirmations: 10 | |
| 211 | + max_range: 150 | |
| 212 | + start_offset: 1800 | |
| 213 | + rpcs: | |
| 214 | + - https://rpc.blast.io | |
| 215 | + - https://blast-rpc.publicnode.com | |
| 216 | + - https://blast.drpc.org | |
| 217 | + | |
| 218 | + unichain: | |
| 219 | + family: evm | |
| 220 | + chain_id: 130 | |
| 221 | + block_time: 1 | |
| 222 | + confirmations: 10 | |
| 223 | + max_range: 200 | |
| 224 | + start_offset: 3600 | |
| 225 | + rpcs: | |
| 226 | + - https://mainnet.unichain.org | |
| 227 | + - https://unichain-rpc.publicnode.com | |
| 228 | + - https://unichain.drpc.org | |
| 229 | + | |
| 230 | + world_chain: | |
| 231 | + family: evm | |
| 232 | + chain_id: 480 | |
| 233 | + block_time: 2 | |
| 234 | + confirmations: 10 | |
| 235 | + max_range: 150 | |
| 236 | + start_offset: 1800 | |
| 237 | + rpcs: | |
| 238 | + - https://worldchain-mainnet.g.alchemy.com/public | |
| 239 | + - https://worldchain-mainnet.gateway.tenderly.co | |
| 240 | + - https://worldchain.drpc.org | |
| 241 | + | |
| 242 | + sonic: | |
| 243 | + family: evm | |
| 244 | + chain_id: 146 | |
| 245 | + block_time: 0.5 | |
| 246 | + confirmations: 10 | |
| 247 | + max_range: 200 | |
| 248 | + start_offset: 3600 | |
| 249 | + rpcs: | |
| 250 | + - https://rpc.soniclabs.com | |
| 251 | + - https://sonic-rpc.publicnode.com | |
| 252 | + - https://sonic.drpc.org | |
| 253 | + | |
| 254 | + sei: | |
| 255 | + family: evm # Sei EVM view; native Cosmos side not indexed here | |
| 256 | + chain_id: 1329 | |
| 257 | + block_time: 0.4 | |
| 258 | + confirmations: 5 | |
| 259 | + max_range: 200 | |
| 260 | + start_offset: 4000 | |
| 261 | + rpcs: | |
| 262 | + - https://evm-rpc.sei-apis.com | |
| 263 | + - https://sei-evm-rpc.publicnode.com | |
| 264 | + - https://sei.drpc.org | |
| 265 | + | |
| 266 | + hyperevm: | |
| 267 | + family: evm | |
| 268 | + chain_id: 999 | |
| 269 | + block_time: 1 | |
| 270 | + confirmations: 5 | |
| 271 | + max_range: 100 # official RPC rate limits are strict | |
| 272 | + start_offset: 1800 | |
| 273 | + rpcs: | |
| 274 | + - {url: "https://rpc.hyperliquid.xyz/evm", rps: 2, burst: 4} # strict official limits | |
| 275 | + - https://hyperliquid.drpc.org | |
| 276 | + - https://rpc.hypurrscan.io | |
| 277 | + | |
| 278 | + # ================ NON-EVM (adapters: Deliverable 4) ================ | |
| 279 | + tron: | |
| 280 | + native: {symbol: TRX, min: 500000, decimals: 6} | |
| 281 | + family: tron | |
| 282 | + block_time: 3 | |
| 283 | + confirmations: 20 # ~solidity (SR 2/3) depth | |
| 284 | + rpcs: | |
| 285 | + - {url: "https://api.trongrid.io", rps: 3, burst: 5} # keyless: dynamic throttle + 30s 403 penalty | |
| 286 | + - https://tron-rpc.publicnode.com # serves /wallet/* HTTP API keyless | |
| 287 | + # bonus: https://tron-evm-rpc.publicnode.com speaks eth_-style JSON-RPC | |
| 288 | + | |
| 289 | + solana: | |
| 290 | + native: {symbol: SOL, min: 1000, decimals: 9} | |
| 291 | + family: solana | |
| 292 | + block_time: 0.4 | |
| 293 | + confirmations: 0 # index at 'finalized' commitment | |
| 294 | + rpcs: | |
| 295 | + - https://solana-rpc.publicnode.com # best keyless option | |
| 296 | + - {url: "https://api.mainnet-beta.solana.com", rps: 4, burst: 8} # 100 req/10s/IP, 40/10s/method | |
| 297 | + | |
| 298 | + ton: | |
| 299 | + family: ton | |
| 300 | + block_time: 5 | |
| 301 | + confirmations: 0 | |
| 302 | + rpcs: | |
| 303 | + - {url: "https://toncenter.com/api/v3/", rps: 0.9, burst: 1} # keyless: 1 req/s HARD limit | |
| 304 | + - {url: "https://toncenter.com/api/v2/", rps: 0.9, burst: 1} | |
| 305 | + | |
| 306 | + xrpl: | |
| 307 | + family: xrpl | |
| 308 | + block_time: 4 # ledger close time | |
| 309 | + confirmations: 0 # validated ledgers are final | |
| 310 | + rpcs: | |
| 311 | + - https://s1.ripple.com:51234/ | |
| 312 | + - https://xrplcluster.com/ | |
| 313 | + - https://s2.ripple.com:51234/ | |
| 314 | + | |
| 315 | + stellar: | |
| 316 | + family: stellar | |
| 317 | + block_time: 5 | |
| 318 | + confirmations: 0 | |
| 319 | + rpcs: | |
| 320 | + - {url: "https://horizon.stellar.org", rps: 0.9, burst: 3} # ~3600 req/h/IP; only ~1y history | |
| 321 | + | |
| 322 | + noble: | |
| 323 | + family: cosmos | |
| 324 | + block_time: 6 | |
| 325 | + confirmations: 0 # CometBFT instant finality | |
| 326 | + rpcs: # CometBFT RPC | |
| 327 | + - https://noble-rpc.polkachu.com | |
| 328 | + - https://rpc.lavenderfive.com:443/noble | |
| 329 | + rest: # LCD | |
| 330 | + - https://noble-api.polkachu.com | |
| 331 | + - https://rest.lavenderfive.com:443/noble | |
| 332 | + | |
| 333 | + aptos: | |
| 334 | + family: aptos | |
| 335 | + block_time: 0.3 | |
| 336 | + confirmations: 0 | |
| 337 | + rpcs: | |
| 338 | + - {url: "https://fullnode.mainnet.aptoslabs.com/v1", rps: 5, burst: 10} # anonymous CU quota per IP | |
| 339 | + | |
| 340 | + sui: | |
| 341 | + family: sui | |
| 342 | + block_time: 0.5 # checkpoint cadence | |
| 343 | + confirmations: 0 | |
| 344 | + rpcs: | |
| 345 | + - https://sui-rpc.publicnode.com # classic JSON-RPC, keyless | |
| 346 | + - https://graphql.mainnet.sui.io/graphql # official GraphQL | |
| 347 | + # official fullnode JSON-RPC shut down July 2026 — do not use | |
| 348 | + # https://fullnode.mainnet.sui.io (gRPC only now) | |
| 349 | + | |
| 350 | + algorand: | |
| 351 | + family: algorand | |
| 352 | + block_time: 2.8 | |
| 353 | + confirmations: 0 | |
| 354 | + rpcs: # algod | |
| 355 | + - https://mainnet-api.4160.nodely.dev | |
| 356 | + rest: # Nodely indexer | |
| 357 | + - https://mainnet-idx.4160.nodely.dev | |
| 358 | + | |
| 359 | + near: | |
| 360 | + family: near | |
| 361 | + block_time: 1.2 | |
| 362 | + confirmations: 0 | |
| 363 | + rpcs: | |
| 364 | + - https://free.rpc.fastnear.com | |
| 365 | + - https://near.drpc.org | |
| 366 | + # rpc.mainnet.near.org is deprecated (Jun 2025) — do not rely on it | |
| 367 | + | |
| 368 | + tezos: | |
| 369 | + family: tezos | |
| 370 | + block_time: 8 | |
| 371 | + confirmations: 2 | |
| 372 | + rpcs: | |
| 373 | + - https://rpc.tzbeta.net | |
| 374 | + - https://mainnet.smartpy.io | |
| 375 | + - https://rpc.tzkt.io/mainnet | |
| 376 | + rest: | |
| 377 | + - https://api.tzkt.io # TzKT indexer — ideal for FA2 transfers | |
| 378 | + | |
| 379 | + hedera: | |
| 380 | + family: hedera | |
| 381 | + block_time: 2 # mirror-node "blocks" | |
| 382 | + confirmations: 0 | |
| 383 | + rpcs: | |
| 384 | + - {url: "https://mainnet-public.mirrornode.hedera.com", rps: 10, burst: 20} # docs: ~50 req/s/IP; stay well under | |
| 385 | + | |
| 386 | + starknet: | |
| 387 | + family: starknet | |
| 388 | + block_time: 30 | |
| 389 | + confirmations: 1 | |
| 390 | + rpcs: | |
| 391 | + - https://starknet-rpc.publicnode.com # spec v0.8 | |
| 392 | + - https://rpc.starknet.lava.build | |
| 393 | + | |
| 394 | + polkadot_asset_hub: | |
| 395 | + family: substrate | |
| 396 | + block_time: 12 | |
| 397 | + confirmations: 0 # GRANDPA finality | |
| 398 | + rpcs: | |
| 399 | + - https://polkadot-asset-hub-rpc.polkadot.io | |
| 400 | + - https://statemint.api.onfinality.io/public | |
| 401 | + | |
| 402 | + eos: | |
| 403 | + family: antelope | |
| 404 | + block_time: 0.5 | |
| 405 | + confirmations: 336 # ~last irreversible block distance | |
| 406 | + rpcs: | |
| 407 | + - https://eos.greymass.com | |
| 408 | + - https://eos.api.eosnation.io | |
| 409 | + | |
| 410 | + bitcoin: | |
| 411 | + family: bitcoin | |
| 412 | + block_time: 600 | |
| 413 | + confirmations: 1 | |
| 414 | + start_offset: 3 | |
| 415 | + native: {symbol: BTC, min: 5, decimals: 8} # record outputs >= 5 BTC | |
| 416 | + rpcs: # esplora-compatible REST | |
| 417 | + - {url: "https://mempool.space/api", rps: 2, burst: 4} | |
| 418 | + - {url: "https://blockstream.info/api", rps: 2, burst: 4} | |
added
config/prices.yaml
+36 −0
@@ -0,0 +1,36 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +# Price feed: CoinGecko simple/price, keyless free tier (~10 req/min budget — | |
| 4 | +# we make ONE request per refresh for all ids). Stablecoins are seeded at 1.0. | |
| 5 | +provider: coingecko | |
| 6 | +refresh_seconds: 300 | |
| 7 | +ids: | |
| 8 | + BTC: bitcoin | |
| 9 | + ETH: ethereum | |
| 10 | + WETH: weth | |
| 11 | + WBTC: wrapped-bitcoin | |
| 12 | + cbBTC: coinbase-wrapped-btc | |
| 13 | + BNB: binancecoin | |
| 14 | + WBNB: wbnb | |
| 15 | + SOL: solana | |
| 16 | + TRX: tron | |
| 17 | + AVAX: avalanche-2 | |
| 18 | + WAVAX: wrapped-avax | |
| 19 | + POL: polygon-ecosystem-token | |
| 20 | + WPOL: polygon-ecosystem-token | |
| 21 | + LINK: chainlink | |
| 22 | + UNI: uniswap | |
| 23 | + AAVE: aave | |
| 24 | + SHIB: shiba-inu | |
| 25 | + PEPE: pepe | |
| 26 | + LDO: lido-dao | |
| 27 | + CRV: curve-dao-token | |
| 28 | + ONDO: ondo-finance | |
| 29 | + MKR: maker | |
| 30 | + ENA: ethena | |
| 31 | + ARB: arbitrum | |
| 32 | + OP: optimism | |
| 33 | + CAKE: pancakeswap-token | |
| 34 | + BONK: bonk | |
| 35 | + JUP: jupiter-exchange-solana | |
| 36 | + WIF: dogwifcoin | |
added
config/tokens.yaml
+228 −0
@@ -0,0 +1,228 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +# Stablecoin registry — every entry verified 2026-08-05 via issuer docs | |
| 4 | +# (tether.to, developers.circle.com, docs.ethena.fi, developers.skyeco.com, | |
| 5 | +# paxosglobal GitHub, docs.usdt0.to, docs.blast.io, ripple/RLUSD-Implementation) | |
| 6 | +# AND a live on-chain read (EVM: eth_call symbol()/decimals(), re-checked by | |
| 7 | +# scripts/verify_tokens.py; non-EVM: native RPC metadata reads). | |
| 8 | +# | |
| 9 | +# Fields: | |
| 10 | +# symbol OUR canonical grouping key — volume/API queries aggregate | |
| 11 | +# on it (so USDT0 deployments group under USDT) | |
| 12 | +# onchain_symbol what the contract actually reports, when it differs | |
| 13 | +# address EVM contract (family: evm chains) | |
| 14 | +# id / id_kind non-EVM identifier (mint, jetton_master, issuer, asa, | |
| 15 | +# coin_type, denom, token_id, asset_id, contract) | |
| 16 | +# native true = issued by the issuer on that chain (redeemable); | |
| 17 | +# false = bridged/pegged wrapper — bridge counterparty risk | |
| 18 | +# discontinued issuer no longer mints/redeems; kept for historical data | |
| 19 | +# | |
| 20 | +# NEVER add an EVM address without running scripts/verify_tokens.py. | |
| 21 | + | |
| 22 | +tokens: | |
| 23 | + # ============================ EVM ============================ | |
| 24 | + ethereum: | |
| 25 | + - {symbol: USDT, address: "0xdAC17F958D2ee523a2206206994597C13D831ec7", decimals: 6, native: true} | |
| 26 | + - {symbol: USDC, address: "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", decimals: 6, native: true} | |
| 27 | + - {symbol: DAI, address: "0x6B175474E89094C44Da98b954EedeAC495271d0F", decimals: 18, native: true} | |
| 28 | + - {symbol: USDS, address: "0xdC035D45d973E3EC169d2276DDab16f1e407384F", decimals: 18, native: true} | |
| 29 | + - {symbol: USDe, address: "0x4c9EDD5852cd905f086C759E8383e09bff1E68B3", decimals: 18, native: true} | |
| 30 | + - {symbol: FDUSD, address: "0xc5f0f7b66764F6ec8C8Dff7BA683102295E16409", decimals: 18, native: true} | |
| 31 | + - {symbol: PYUSD, address: "0x6c3ea9036406852006290770BEdFcAbA0e23A0e8", decimals: 6, native: true} | |
| 32 | + - {symbol: RLUSD, address: "0x8292Bb45bf1Ee4d140127049757C2E0fF06317eD", decimals: 18, native: true} # beware lookalike 0x708D2375... | |
| 33 | + - {symbol: USDG, address: "0xe343167631d89B6Ffc58B88d6b7fB0228795491D", decimals: 6, native: true} | |
| 34 | + - {symbol: TUSD, address: "0x0000000000085d4780B73119b644AE5ecd22b376", decimals: 18, native: true} | |
| 35 | + # --- major crypto (category: crypto — USD via price worker) --- | |
| 36 | + - {symbol: WETH, address: "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2", decimals: 18, native: true, category: crypto} | |
| 37 | + - {symbol: WBTC, address: "0x2260FAC5E5542a773Aa44fBCfeDf7C193bc2C599", decimals: 8, native: false, category: crypto} | |
| 38 | + - {symbol: LINK, address: "0x514910771AF9Ca656af840dff83E8264EcF986CA", decimals: 18, native: true, category: crypto} | |
| 39 | + - {symbol: UNI, address: "0x1f9840a85d5aF5bf1D1762F925BDADdC4201F984", decimals: 18, native: true, category: crypto} | |
| 40 | + - {symbol: AAVE, address: "0x7Fc66500c84A76Ad7e9c93437bFc5Ac33E2DDaE9", decimals: 18, native: true, category: crypto} | |
| 41 | + - {symbol: SHIB, address: "0x95aD61b0a150d79219dCF64E1E6Cc01f0B64C4cE", decimals: 18, native: true, category: crypto} | |
| 42 | + - {symbol: PEPE, address: "0x6982508145454Ce325dDbE47a25d4ec3d2311933", decimals: 18, native: true, category: crypto} | |
| 43 | + - {symbol: LDO, address: "0x5A98FcBEA516Cf06857215779Fd812CA3beF1B32", decimals: 18, native: true, category: crypto} | |
| 44 | + - {symbol: CRV, address: "0xD533a949740bb3306d119CC777fa900bA034cd52", decimals: 18, native: true, category: crypto} | |
| 45 | + - {symbol: ONDO, address: "0xfAbA6f8e4a5E8Ab82F62fe7C39859FA577269BE3", decimals: 18, native: true, category: crypto} | |
| 46 | + - {symbol: MKR, address: "0x9f8F72aA9304c8B593d555F12eF6589cC3A579A2", decimals: 18, native: true, category: crypto} # bytes32 symbol | |
| 47 | + - {symbol: ENA, address: "0x57e114B691Db790C35207b2e685D4A43181e6061", decimals: 18, native: true, category: crypto} | |
| 48 | + | |
| 49 | + base: | |
| 50 | + - {symbol: USDC, address: "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913", decimals: 6, native: true} | |
| 51 | + - {symbol: USDe, address: "0x5d3a1Ff2b6BAb83b63cd9AD0787074081a52ef34", decimals: 18, native: false} # LayerZero OFT | |
| 52 | + - {symbol: USDS, address: "0x820C137fa70C8691f0e44Dc420a5e53c168921Dc", decimals: 18, native: false} # SkyLink | |
| 53 | + - {symbol: DAI, address: "0x50c5725949A6F0c72E6C4a641F24049A917DB0Cb", decimals: 18, native: false} # canonical bridge | |
| 54 | + - {symbol: WETH, address: "0x4200000000000000000000000000000000000006", decimals: 18, native: false, category: crypto} | |
| 55 | + - {symbol: cbBTC, address: "0xcbB7C0000aB88B473b1f5aFd9ef808440eed33Bf", decimals: 8, native: true, category: crypto} | |
| 56 | + | |
| 57 | + arbitrum: | |
| 58 | + - {symbol: USDC, address: "0xaf88d065e77c8cC2239327C5EDb3A432268e5831", decimals: 6, native: true} | |
| 59 | + - {symbol: USDT, onchain_symbol: "USD₮0", address: "0xFd086bC7CD5C481DCC9C85ebE478A1C0b69FCbb9", decimals: 6, native: false} # USDT0 legacy mesh | |
| 60 | + - {symbol: USDe, address: "0x5d3a1Ff2b6BAb83b63cd9AD0787074081a52ef34", decimals: 18, native: false} | |
| 61 | + - {symbol: USDS, address: "0x6491c05A82219b8D1479057361ff1654749b876b", decimals: 18, native: false} # SkyLink | |
| 62 | + - {symbol: DAI, address: "0xDA10009cBd5D07dd0CeCc66161FC93D7c9000da1", decimals: 18, native: false} | |
| 63 | + - {symbol: PYUSD, address: "0x46850aD61C2B7d64d08c9C754F45254596696984", decimals: 6, native: true} # Paxos-issued | |
| 64 | + - {symbol: ARB, address: "0x912CE59144191C1204E64559FE8253a0e49E6548", decimals: 18, native: true, category: crypto} | |
| 65 | + - {symbol: WETH, address: "0x82aF49447D8a07e3bd95BD0d56f35241523fBab1", decimals: 18, native: false, category: crypto} | |
| 66 | + - {symbol: WBTC, address: "0x2f2a2543B76A4166549F7aaB2e75Bef0aefC5B0f", decimals: 8, native: false, category: crypto} | |
| 67 | + - {symbol: LINK, address: "0xf97f4df75117a78c1A5a0DBb814Af92458539FB4", decimals: 18, native: false, category: crypto} | |
| 68 | + | |
| 69 | + optimism: | |
| 70 | + - {symbol: USDC, address: "0x0b2C639c533813f4Aa9D7837CAf62653d097Ff85", decimals: 6, native: true} | |
| 71 | + - {symbol: USDT, address: "0x94b008aA00579c1307B0EF2c499aD98a8ce58e58", decimals: 6, native: false} # legacy canonical bridge (dominant liquidity) | |
| 72 | + - {symbol: USDT, onchain_symbol: "USD₮0", address: "0x01bFF41798a0BcF287b996046Ca68b395DbC1071", decimals: 6, native: false} # new USDT0 OFT | |
| 73 | + - {symbol: USDe, address: "0x5d3a1Ff2b6BAb83b63cd9AD0787074081a52ef34", decimals: 18, native: false} | |
| 74 | + - {symbol: DAI, address: "0xDA10009cBd5D07dd0CeCc66161FC93D7c9000da1", decimals: 18, native: false} | |
| 75 | + - {symbol: OP, address: "0x4200000000000000000000000000000000000042", decimals: 18, native: true, category: crypto} | |
| 76 | + - {symbol: WETH, address: "0x4200000000000000000000000000000000000006", decimals: 18, native: false, category: crypto} | |
| 77 | + | |
| 78 | + polygon: | |
| 79 | + - {symbol: USDT, onchain_symbol: "USDT0", address: "0xc2132D05D31c914a87C6611C10748AEb04B58e8F", decimals: 6, native: false} # USDT0 legacy mesh | |
| 80 | + - {symbol: USDC, address: "0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359", decimals: 6, native: true} | |
| 81 | + - {symbol: DAI, address: "0x8f3Cf7ad23Cd3CaDbD9735AFf958023239c6A063", decimals: 18, native: false} # PoS bridge | |
| 82 | + - {symbol: WPOL, address: "0x0d500B1d8E8eF31E21C99d1Db9A6444d3ADf1270", decimals: 18, native: true, category: crypto} | |
| 83 | + - {symbol: WETH, address: "0x7ceB23fD6bC0adD59E62ac25578270cFf1b9f619", decimals: 18, native: false, category: crypto} | |
| 84 | + | |
| 85 | + bsc: | |
| 86 | + - {symbol: USDT, address: "0x55d398326f99059fF775485246999027B3197955", decimals: 18, native: false} # Binance-Peg, 18 decimals! | |
| 87 | + - {symbol: USDC, address: "0x8AC76a51cc950d9822D68b83fE1Ad97B32Cd580d", decimals: 18, native: false} # Binance-Peg | |
| 88 | + - {symbol: USDe, address: "0x5d3a1Ff2b6BAb83b63cd9AD0787074081a52ef34", decimals: 18, native: false} | |
| 89 | + - {symbol: FDUSD, address: "0xc5f0f7b66764F6ec8C8Dff7BA683102295E16409", decimals: 18, native: true} | |
| 90 | + - {symbol: TUSD, address: "0x40af3827F39D0EAcBF4A168f8D4ee67c121D11c9", decimals: 18, native: true} # post-2023 contract swap | |
| 91 | + - {symbol: DAI, address: "0x1AF3F329e8BE154074D8769D1FFa4eE058B1DBc3", decimals: 18, native: false} # Binance-Peg | |
| 92 | + - {symbol: WBNB, address: "0xbb4CdB9CBd36B01bD1cBaEBF2De08d9173bc095c", decimals: 18, native: true, category: crypto} | |
| 93 | + - {symbol: CAKE, address: "0x0E09FaBB73Bd3Ade0a17ECC321fD13a19e81cE82", decimals: 18, native: true, category: crypto} | |
| 94 | + - {symbol: ETH, address: "0x2170Ed0880ac9A755fd29B2688956BD959F933F8", decimals: 18, native: false, category: crypto} # Binance-Peg | |
| 95 | + | |
| 96 | + avalanche: | |
| 97 | + - {symbol: USDT, address: "0x9702230A8Ea53601f5cD2dc00fDBc13d4dF4A8c7", decimals: 6, native: true} | |
| 98 | + - {symbol: USDC, address: "0xB97EF9Ef8734C71904D8002F8b6Bc66Dd9c48a6E", decimals: 6, native: true} | |
| 99 | + - {symbol: DAI, onchain_symbol: "DAI.e", address: "0xd586E7F844cEa2F87f50152665BCbc2C279D8d70", decimals: 18, native: false} # Avalanche Bridge | |
| 100 | + - {symbol: TUSD, address: "0x1C20E891Bab6b1727d14Da358FAe2984Ed9B59EB", decimals: 18, native: true} | |
| 101 | + - {symbol: WAVAX, address: "0xB31f66AA3C1e785363F0875A1B74E27b85FD66c7", decimals: 18, native: true, category: crypto} | |
| 102 | + - {symbol: WETH, onchain_symbol: "WETH.e", address: "0x49D5c2BdFfac6CE2BFdB6640F4F80f226bc10bAB", decimals: 18, native: false, category: crypto} | |
| 103 | + | |
| 104 | + celo: | |
| 105 | + - {symbol: USDT, address: "0x48065fbBE25f71C9282ddf5e1cD6D6A887483D5e", decimals: 6, native: true} # Tether native (2024) | |
| 106 | + - {symbol: USDC, address: "0xcebA9300f2b948710d2653dD7B07f33A8B32118C", decimals: 6, native: true} | |
| 107 | + | |
| 108 | + kaia: | |
| 109 | + - {symbol: USDT, address: "0xd077A400968890Eacc75cdc901F0356c943e4fDb", decimals: 6, native: true} # Tether native; NOT 0x5c13e3... (Wormhole) | |
| 110 | + | |
| 111 | + ink: | |
| 112 | + - {symbol: USDT, onchain_symbol: "USD₮0", address: "0x0200C29006150606B650577BBE7B6248F58470c1", decimals: 6, native: false} | |
| 113 | + - {symbol: USDC, address: "0x2D270e6886d130D724215A266106e6832161EAEd", decimals: 6, native: true} | |
| 114 | + - {symbol: USDG, address: "0xe343167631d89B6Ffc58B88d6b7fB0228795491D", decimals: 6, native: true} # same address as Ethereum | |
| 115 | + | |
| 116 | + scroll: | |
| 117 | + - {symbol: USDe, address: "0x5d3a1Ff2b6BAb83b63cd9AD0787074081a52ef34", decimals: 18, native: false} | |
| 118 | + - {symbol: USDC, address: "0x06eFdBFf2a14a7c8E15944D1F4A48F9F95F663A4", decimals: 6, native: false} # canonical bridge (no Circle-native) | |
| 119 | + - {symbol: USDT, address: "0xf55BEC9cafDbE8730f096Aa55dad6D22d44099Df", decimals: 6, native: false} # canonical bridge | |
| 120 | + | |
| 121 | + linea: | |
| 122 | + - {symbol: USDC, address: "0x176211869cA2b568f2A7D4EE941E073a821EE1ff", decimals: 6, native: true} | |
| 123 | + - {symbol: USDT, address: "0xA219439258ca9da29E9Cc4cE5596924745e12B93", decimals: 6, native: false} # canonical bridge | |
| 124 | + | |
| 125 | + mantle: | |
| 126 | + - {symbol: USDe, address: "0x5d3a1Ff2b6BAb83b63cd9AD0787074081a52ef34", decimals: 18, native: false} | |
| 127 | + - {symbol: USDT, onchain_symbol: "USDT0", address: "0x779Ded0c9e1022225f8E0630b35a9b54bE713736", decimals: 6, native: false} # USDT0 OFT | |
| 128 | + - {symbol: USDT, address: "0x201EBa5CC46D216Ce6DC03F6a759e8E766e956aE", decimals: 6, native: false} # legacy canonical bridge | |
| 129 | + - {symbol: USDC, address: "0x09Bc4E0D864854c6aFB6eB9A9cdF58aC190D0dF9", decimals: 6, native: false} # canonical bridge | |
| 130 | + | |
| 131 | + gnosis: | |
| 132 | + - {symbol: DAI, onchain_symbol: "WXDAI", address: "0xe91D153E0b41518A2CE8Dd3d7944Fa863463a97d", decimals: 18, native: false} # wrapped xDai (DAI is the gas token) | |
| 133 | + - {symbol: USDC, onchain_symbol: "USDC.e", address: "0x2a22f9c3b484c3629090FeED35F17Ff8F88f76F0", decimals: 6, native: false} # bridged standard | |
| 134 | + - {symbol: USDC, address: "0xDDAfbb505ad214D7b80b1f830fcCc89B60fb7A83", decimals: 6, native: false} # legacy omni-bridge "USD//C" | |
| 135 | + | |
| 136 | + zksync: | |
| 137 | + - {symbol: USDC, address: "0x1d17CBcF0D6D143135aE902365D2E5e2A16538D4", decimals: 6, native: true} | |
| 138 | + - {symbol: USDe, address: "0x39Fe7a0DACcE31Bd90418e3e659fb0b5f0B3Db0d", decimals: 18, native: false} # ZKsync CREATE exception address | |
| 139 | + | |
| 140 | + blast: | |
| 141 | + # USDB is a REBASING stablecoin: balances grow without Transfer events — | |
| 142 | + # transfer indexing works, balance tracking needs eth_call snapshots | |
| 143 | + - {symbol: USDB, address: "0x4300000000000000000000000000000000000003", decimals: 18, native: true} | |
| 144 | + | |
| 145 | + unichain: | |
| 146 | + - {symbol: USDC, address: "0x078D782b760474a361dDA0AF3839290b0EF57AD6", decimals: 6, native: true} | |
| 147 | + | |
| 148 | + world_chain: | |
| 149 | + - {symbol: USDC, address: "0x79A02482A880bCe3F13E09da970dC34dB4cD24D1", decimals: 6, native: true} | |
| 150 | + | |
| 151 | + sonic: | |
| 152 | + - {symbol: USDC, address: "0x29219dd400f2Bf60E5a23d13Be72B486D4038894", decimals: 6, native: true} | |
| 153 | + - {symbol: USDT, address: "0x6047828dc181963ba44974801FF68e538dA5eaF9", decimals: 6, native: false} # Sonic Gateway bridge; not Tether-redeemable | |
| 154 | + | |
| 155 | + sei: | |
| 156 | + - {symbol: USDT, onchain_symbol: "USD₮0", address: "0x9151434b16b9763660705744891fA906F660EcC5", decimals: 6, native: false} | |
| 157 | + - {symbol: USDC, address: "0xe15fC38F6D8c56aF07bbCBe3BAf5708A2Bf42392", decimals: 6, native: true} | |
| 158 | + | |
| 159 | + hyperevm: | |
| 160 | + - {symbol: USDT, onchain_symbol: "USD₮0", address: "0xB8CE59FC3717ada4C02eaDF9682A9e934F625ebb", decimals: 6, native: false} | |
| 161 | + - {symbol: USDC, address: "0xb88339CB7199b77E23DB6E890353E22632Ba630f", decimals: 6, native: true} | |
| 162 | + | |
| 163 | + # ========================== NON-EVM ========================== | |
| 164 | + # (indexed once their family adapters land — Deliverable 4) | |
| 165 | + tron: | |
| 166 | + - {symbol: USDT, id: "TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t", id_kind: contract, decimals: 6, native: true} | |
| 167 | + - {symbol: TUSD, id: "TUpMhErZL2fhh4sVNULAbNKLokS4GjC1F4", id_kind: contract, decimals: 18, native: true} | |
| 168 | + - {symbol: USDC, id: "TEkxiTehnzSmSe2XqrBj4w32RUN966rdz8", id_kind: contract, decimals: 6, native: false, discontinued: true} # Circle exited Tron 2024/25 | |
| 169 | + | |
| 170 | + solana: | |
| 171 | + - {symbol: USDC, id: "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v", id_kind: mint, decimals: 6, native: true} | |
| 172 | + - {symbol: USDT, id: "Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB", id_kind: mint, decimals: 6, native: true} | |
| 173 | + - {symbol: PYUSD, id: "2b1kV6DkPAnxd5ixfnxCpjxmKwqjjaYmCZfHsFu24GXo", id_kind: mint, decimals: 6, native: true, program: token-2022} | |
| 174 | + - {symbol: USDG, id: "2u1tszSeqZ3qBWF3uNGPFc8TzMk2tdiwknnRMWGWjGWH", id_kind: mint, decimals: 6, native: true, program: token-2022} | |
| 175 | + - {symbol: USDe, id: "DEkqHyPN7GMRJ5cArtQFAWefqbZb33Hyf6s5iCwjEonT", id_kind: mint, decimals: 9, native: false} # LayerZero OFT; 9 decimals! | |
| 176 | + - {symbol: BONK, id: "DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263", id_kind: mint, decimals: 5, native: true, category: crypto} | |
| 177 | + - {symbol: JUP, id: "JUPyiwrYJFskUPiHa7hkeR8VUtAeFoSYbKedZNsDvCN", id_kind: mint, decimals: 6, native: true, category: crypto} | |
| 178 | + - {symbol: WIF, id: "EKpQGSJtjMFqKZ9KQanSqYXRcF8fBopzLHYxdM65zcjm", id_kind: mint, decimals: 6, native: true, category: crypto} | |
| 179 | + | |
| 180 | + ton: | |
| 181 | + - {symbol: USDT, id: "EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs", id_kind: jetton_master, decimals: 6, native: true} | |
| 182 | + | |
| 183 | + xrpl: | |
| 184 | + # XRPL issued currencies: decimal string amounts, no fixed decimals | |
| 185 | + - {symbol: RLUSD, id: "rMxCKbEDwqr76QuheSUMdEGf4B9xJ8m5De", id_kind: issuer, currency_hex: "524C555344000000000000000000000000000000", native: true} | |
| 186 | + - {symbol: USDC, id: "rGm7WCVp9gb4jZHWTEtGUr4dd74z2XuWhE", id_kind: issuer, currency_hex: "5553444300000000000000000000000000000000", native: true} # Circle, June 2025 | |
| 187 | + | |
| 188 | + stellar: | |
| 189 | + - {symbol: USDC, id: "GA5ZSEJYB37JRC5AVCIA5MOP4RHTM335X2KGX3IHOJAPP5RE34K4KZVN", id_kind: issuer, decimals: 7, native: true} | |
| 190 | + | |
| 191 | + noble: | |
| 192 | + - {symbol: USDC, id: "uusdc", id_kind: denom, decimals: 6, native: true} | |
| 193 | + | |
| 194 | + aptos: | |
| 195 | + # fungible-asset METADATA object addresses (not package addresses) | |
| 196 | + - {symbol: USDC, id: "0xbae207659db88bea0cbead6da0ed00aac12edcdda169e591cd41c94180b46f3b", id_kind: coin_type, decimals: 6, native: true} | |
| 197 | + - {symbol: USDT, id: "0x357b0b74bc833e95a115ad22604854d6b0fca151cecd94111770e5d6ffc9dc2b", id_kind: coin_type, decimals: 6, native: true} | |
| 198 | + | |
| 199 | + sui: | |
| 200 | + - {symbol: USDC, id: "0xdba34672e30cb065b1f93e3ab55318768fd6fef66c15942c9f7cb846e2f900e7::usdc::USDC", id_kind: coin_type, decimals: 6, native: true} | |
| 201 | + - {symbol: USDT, id: "0xc060006111016b8a020ad5b33834984a437aaa7d3c74c18e09a95d48aceab08c::coin::COIN", id_kind: coin_type, decimals: 6, native: false} # Wormhole wUSDT (no Tether-native on Sui) | |
| 202 | + | |
| 203 | + algorand: | |
| 204 | + - {symbol: USDC, id: "31566704", id_kind: asa, decimals: 6, native: true} | |
| 205 | + - {symbol: USDT, id: "312769", id_kind: asa, decimals: 6, native: true, discontinued: true} # Tether wind-down Sept 2025 | |
| 206 | + | |
| 207 | + near: | |
| 208 | + - {symbol: USDT, id: "usdt.tether-token.near", id_kind: contract, decimals: 6, native: true} | |
| 209 | + - {symbol: USDC, id: "17208628f84f5d6ad33f0da3bbbeb27ffcb398eac501a31bd6ad2011e36133a1", id_kind: contract, decimals: 6, native: true} | |
| 210 | + | |
| 211 | + tezos: | |
| 212 | + - {symbol: USDT, id: "KT1XnTn74bUtxHfDtBmm2bGZAQfhPbvKWR8o", id_kind: contract, token_id: 0, decimals: 6, native: true} # FA2 | |
| 213 | + | |
| 214 | + hedera: | |
| 215 | + - {symbol: USDC, id: "0.0.456858", id_kind: token_id, decimals: 6, native: true} | |
| 216 | + | |
| 217 | + starknet: | |
| 218 | + - {symbol: USDC, id: "0x033068F6539f8e6e6b131e6B2B814e6c34A5224bC66947c47DaB9dFeE93b35fb", id_kind: contract, decimals: 6, native: true} # Circle native (CCTP V2) | |
| 219 | + - {symbol: USDT, id: "0x068f5c6a61780768455de69077e07e89787839bf8166decfbf92b645209c0fb8", id_kind: contract, decimals: 6, native: false} # StarkGate bridge | |
| 220 | + | |
| 221 | + polkadot_asset_hub: | |
| 222 | + - {symbol: USDT, id: "1984", id_kind: asset_id, decimals: 6, native: true} | |
| 223 | + - {symbol: USDC, id: "1337", id_kind: asset_id, decimals: 6, native: true} | |
| 224 | + | |
| 225 | + eos: | |
| 226 | + - {symbol: USDT, id: "tethertether", id_kind: contract, decimals: 4, native: true, discontinued: true} # Tether wind-down Sept 2025 | |
| 227 | + | |
| 228 | + bitcoin: [] # native BTC whale tracking only — configured in chains.yaml | |
added
docker-compose.yml
+61 −0
@@ -0,0 +1,61 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +# Stablecoin explorer stack. | |
| 4 | +# | |
| 5 | +# cp .env.example .env # then: docker compose up -d | |
| 6 | +# | |
| 7 | +# Storage today is SQLite on the shared `data` volume (indexer writes, API | |
| 8 | +# reads — WAL mode makes that safe). The postgres service is provisioned now | |
| 9 | +# so Deliverable 5 (PostgreSQL schema + migrations) is a config switch, not | |
| 10 | +# an infra change. | |
| 11 | + | |
| 12 | +services: | |
| 13 | + postgres: | |
| 14 | + image: postgres:16-alpine | |
| 15 | + environment: | |
| 16 | + POSTGRES_USER: ${POSTGRES_USER:-explorer} | |
| 17 | + POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-explorer} | |
| 18 | + POSTGRES_DB: ${POSTGRES_DB:-explorer} | |
| 19 | + volumes: | |
| 20 | + - pgdata:/var/lib/postgresql/data | |
| 21 | + ports: | |
| 22 | + - "127.0.0.1:${POSTGRES_PORT:-5433}:5432" | |
| 23 | + healthcheck: | |
| 24 | + test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-explorer}"] | |
| 25 | + interval: 10s | |
| 26 | + timeout: 5s | |
| 27 | + retries: 5 | |
| 28 | + | |
| 29 | + indexer: | |
| 30 | + build: . | |
| 31 | + command: python run_indexer.py | |
| 32 | + environment: | |
| 33 | + EXPLORER_DB: /data/explorer.db | |
| 34 | + EXPLORER_CHAINS: ${EXPLORER_CHAINS:-} | |
| 35 | + EXPLORER_BACKFILL_DAYS: ${EXPLORER_BACKFILL_DAYS:-0} | |
| 36 | + # set DATABASE_URL to use postgres instead of the SQLite volume | |
| 37 | + DATABASE_URL: ${DATABASE_URL:-} | |
| 38 | + volumes: | |
| 39 | + - data:/data | |
| 40 | + depends_on: | |
| 41 | + postgres: | |
| 42 | + condition: service_healthy | |
| 43 | + restart: unless-stopped | |
| 44 | + | |
| 45 | + api: | |
| 46 | + build: . | |
| 47 | + command: uvicorn api.main:app --host 0.0.0.0 --port 8080 | |
| 48 | + environment: | |
| 49 | + EXPLORER_DB: /data/explorer.db | |
| 50 | + DATABASE_URL: ${DATABASE_URL:-} | |
| 51 | + volumes: | |
| 52 | + - data:/data | |
| 53 | + ports: | |
| 54 | + - "${API_PORT:-8080}:8080" | |
| 55 | + depends_on: | |
| 56 | + - indexer | |
| 57 | + restart: unless-stopped | |
| 58 | + | |
| 59 | +volumes: | |
| 60 | + data: | |
| 61 | + pgdata: | |
added
docs/screenshot-api.png
+0 −0
Binary file not shown.
added
docs/screenshot-dark.png
+0 −0
Binary file not shown.
added
docs/screenshot-overview.png
+0 −0
Binary file not shown.
added
indexer/.DS_Store
+0 −0
Binary file not shown.
added
indexer/__init__.py
+2 −0
@@ -0,0 +1,2 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
added
indexer/adapters/__init__.py
+2 −0
@@ -0,0 +1,2 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
added
indexer/adapters/bitcoin.py
+141 −0
@@ -0,0 +1,141 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Bitcoin adapter — whale-only native BTC transfers via the esplora REST API | |
| 4 | +(mempool.space and blockstream.info expose the same schema, giving free | |
| 5 | +keyless failover). | |
| 6 | + | |
| 7 | +UTXO model: a transaction spends inputs and creates outputs — there is no | |
| 8 | +single from/to. We record each OUTPUT >= the configured minimum as one | |
| 9 | +canonical event: `to` = the output's address, `from` = the first input's | |
| 10 | +previous-output address when the tx has exactly one distinct input address | |
| 11 | +(else null — genuinely ambiguous, e.g. exchange sweep consolidations). | |
| 12 | +Coinbase (miner reward) txs have no input address. Amounts are satoshis | |
| 13 | +(8 decimals). | |
| 14 | + | |
| 15 | +Pace: ~10-minute blocks, txs fetched 25 per page — even a full block is | |
| 16 | +~150 paged calls in a 600s budget, well inside the 2 rps bucket. Reorg | |
| 17 | +safety: 1-confirmation trail + stored block-hash check with 2-block rewind. | |
| 18 | +""" | |
| 19 | + | |
| 20 | +import logging | |
| 21 | +import time | |
| 22 | + | |
| 23 | +from .. import db | |
| 24 | +from ..rpc import AllEndpointsDown, RestError, RestPool | |
| 25 | + | |
| 26 | +log = logging.getLogger("bitcoin") | |
| 27 | + | |
| 28 | +MAX_TX_PAGES = 400 # hard cap per block (~10k txs) — logged if ever hit | |
| 29 | + | |
| 30 | + | |
| 31 | +class BitcoinIndexer: | |
| 32 | + def __init__(self, chain, cfg, tokens, db_path): | |
| 33 | + self.chain = chain | |
| 34 | + self.cfg = cfg | |
| 35 | + self.db_path = db_path | |
| 36 | + self.conn = None | |
| 37 | + self.pool = RestPool(cfg["rpcs"], timeout=30) | |
| 38 | + self.native = cfg["native"] # {symbol: BTC, min, decimals: 8} | |
| 39 | + | |
| 40 | + # -- esplora helpers --------------------------------------------------- | |
| 41 | + | |
| 42 | + def head(self): | |
| 43 | + return int(self.pool.get("/blocks/tip/height")) | |
| 44 | + | |
| 45 | + def hash_at(self, height): | |
| 46 | + h = self.pool.get(f"/block-height/{height}") # text/plain hash | |
| 47 | + return h.strip() if isinstance(h, str) else None | |
| 48 | + | |
| 49 | + def block_txs(self, block_hash): | |
| 50 | + """All txs of a block, paged 25 at a time.""" | |
| 51 | + txs, start = [], 0 | |
| 52 | + for _ in range(MAX_TX_PAGES): | |
| 53 | + page = self.pool.get(f"/block/{block_hash}/txs/{start}") | |
| 54 | + if not page: | |
| 55 | + break | |
| 56 | + txs.extend(page) | |
| 57 | + if len(page) < 25: | |
| 58 | + break | |
| 59 | + start += 25 | |
| 60 | + else: | |
| 61 | + log.warning("%s: block %s hit the %d-page cap — tail not scanned", | |
| 62 | + self.chain, block_hash[:12], MAX_TX_PAGES) | |
| 63 | + return txs | |
| 64 | + | |
| 65 | + # -- pipeline ---------------------------------------------------------- | |
| 66 | + | |
| 67 | + def process_block(self, height, block_hash): | |
| 68 | + blk = self.pool.get(f"/block/{block_hash}") | |
| 69 | + ts = blk.get("timestamp") | |
| 70 | + min_sats = int(self.native["min"] * 10 ** self.native["decimals"]) | |
| 71 | + rows = [] | |
| 72 | + for tx in self.block_txs(block_hash): | |
| 73 | + in_addrs = {v.get("prevout", {}).get("scriptpubkey_address") | |
| 74 | + for v in tx.get("vin") or [] if v.get("prevout")} | |
| 75 | + in_addrs.discard(None) | |
| 76 | + frm = next(iter(in_addrs)) if len(in_addrs) == 1 else None | |
| 77 | + for oi, out in enumerate(tx.get("vout") or []): | |
| 78 | + val = out.get("value", 0) | |
| 79 | + to = out.get("scriptpubkey_address") | |
| 80 | + if val < min_sats or not to: | |
| 81 | + continue # below threshold or OP_RETURN/nonstandard | |
| 82 | + if to == frm: | |
| 83 | + continue # change back to the sender — not a transfer | |
| 84 | + rows.append({ | |
| 85 | + "chain": self.chain, "block": height, "block_hash": block_hash, | |
| 86 | + "tx_hash": tx["txid"], "log_index": oi, "timestamp": ts, | |
| 87 | + "token": "native", "symbol": self.native["symbol"], | |
| 88 | + "from": frm, "to": to, | |
| 89 | + "amount": str(val), "decimals": self.native["decimals"], | |
| 90 | + }) | |
| 91 | + db.insert_transfers(self.conn, rows) | |
| 92 | + return len(rows) | |
| 93 | + | |
| 94 | + def reorged(self, height, stored_hash): | |
| 95 | + if not stored_hash: | |
| 96 | + return False | |
| 97 | + h = self.hash_at(height) | |
| 98 | + return h is not None and h != stored_hash | |
| 99 | + | |
| 100 | + # -- main loop ----------------------------------------------------- | |
| 101 | + | |
| 102 | + def run(self, stop): | |
| 103 | + self.conn = db.connect(self.db_path) | |
| 104 | + cur = db.get_cursor(self.conn, self.chain) | |
| 105 | + confirmations = self.cfg.get("confirmations", 1) | |
| 106 | + while not stop.is_set(): | |
| 107 | + try: | |
| 108 | + head = self.head() | |
| 109 | + safe = head - confirmations | |
| 110 | + if cur is None: | |
| 111 | + start = max(0, safe - self.cfg.get("start_offset", 3)) | |
| 112 | + cur = (start, None, start) | |
| 113 | + db.set_cursor(self.conn, self.chain, start, None) | |
| 114 | + db.set_backfill(self.conn, self.chain, start) | |
| 115 | + log.info("%s: fresh start at height %d (tip %d)", self.chain, start, head) | |
| 116 | + height, stored_hash, _ = cur | |
| 117 | + | |
| 118 | + if self.reorged(height, stored_hash): | |
| 119 | + to_block = height - 2 | |
| 120 | + log.warning("%s: reorg at %d — rolling back to %d", | |
| 121 | + self.chain, height, to_block) | |
| 122 | + db.rollback(self.conn, self.chain, to_block) | |
| 123 | + height, stored_hash = to_block, None | |
| 124 | + | |
| 125 | + while height < safe and not stop.is_set(): | |
| 126 | + nxt = height + 1 | |
| 127 | + bh = self.hash_at(nxt) | |
| 128 | + if not bh: | |
| 129 | + break | |
| 130 | + n = self.process_block(nxt, bh) | |
| 131 | + height, stored_hash = nxt, bh | |
| 132 | + db.set_cursor(self.conn, self.chain, height, bh, head=head) | |
| 133 | + log.info("%s: %d whale outputs in block %d (tip %d)", | |
| 134 | + self.chain, n, height, head) | |
| 135 | + | |
| 136 | + cur = (height, stored_hash, None) | |
| 137 | + db.save_rpc_health(self.conn, self.chain, self.pool.stats(), int(time.time())) | |
| 138 | + stop.wait(60) # ~10-min blocks — checking every minute is plenty | |
| 139 | + except (RestError, AllEndpointsDown, Exception) as e: | |
| 140 | + log.error("%s: %s — retrying in 30s", self.chain, e) | |
| 141 | + stop.wait(30) | |
added
indexer/adapters/solana.py
+192 −0
@@ -0,0 +1,192 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Solana adapter — SPL stablecoin transfers via token-balance diffs. | |
| 4 | + | |
| 5 | +Why balance diffs instead of instruction parsing: SPL transfers can hide | |
| 6 | +inside CPIs (inner instructions) from any program, and PYUSD/USDG live on | |
| 7 | +Token-2022 while USDC/USDT are classic SPL Token. `preTokenBalances` / | |
| 8 | +`postTokenBalances` in tx meta cover BOTH programs and every nesting depth | |
| 9 | +— diffing them per mint is the robust way to see value movement. | |
| 10 | + | |
| 11 | +Owner vs token account: balances are held in token accounts (usually ATAs) | |
| 12 | +owned by a wallet. The meta rows carry the OWNER — that's what we store as | |
| 13 | +from/to, which is what users expect an explorer to show. | |
| 14 | + | |
| 15 | +Pairing: within one tx and mint, if exactly one owner's balance decreased, | |
| 16 | +every increase is a transfer from that owner. With several senders the | |
| 17 | +mapping is genuinely ambiguous (multi-hop swaps) — we still record each | |
| 18 | +increase (amount/receiver are exact) with from=None. Mints/burns show up | |
| 19 | +as unmatched increases/decreases, which D6 will classify. | |
| 20 | + | |
| 21 | +Finality: we read at `finalized` commitment, so reorgs are impossible and | |
| 22 | +confirmations=0. Skipped slots are normal (no block produced) — advance. | |
| 23 | +Free-RPC reality: ~2.5 blocks/s of large payloads can outrun keyless rate | |
| 24 | +limits; if lag exceeds `max_lag` slots we jump forward and log the gap — | |
| 25 | +a lagging-but-live explorer beats a complete-but-hours-behind one. | |
| 26 | +""" | |
| 27 | + | |
| 28 | +import logging | |
| 29 | +import time | |
| 30 | + | |
| 31 | +from .. import db | |
| 32 | +from ..rpc import AllEndpointsDown, RpcError, RpcPool | |
| 33 | + | |
| 34 | +log = logging.getLogger("solana") | |
| 35 | + | |
| 36 | +_BLOCK_OPTS = { | |
| 37 | + "encoding": "json", | |
| 38 | + "transactionDetails": "accounts", # balances only — much lighter than full | |
| 39 | + "rewards": False, | |
| 40 | + "maxSupportedTransactionVersion": 0, | |
| 41 | + "commitment": "finalized", | |
| 42 | +} | |
| 43 | + | |
| 44 | + | |
| 45 | +class SolanaIndexer: | |
| 46 | + def __init__(self, chain, cfg, tokens, db_path): | |
| 47 | + self.chain = chain | |
| 48 | + self.cfg = cfg | |
| 49 | + self.db_path = db_path | |
| 50 | + self.conn = None | |
| 51 | + self.pool = RpcPool(cfg["rpcs"]) | |
| 52 | + self.tokens = {t["id"]: t for t in tokens} # mint address → meta | |
| 53 | + self.native = cfg.get("native") # {symbol: SOL, min, decimals: 9} | |
| 54 | + | |
| 55 | + # -- RPC helpers ----------------------------------------------------- | |
| 56 | + | |
| 57 | + def head(self): | |
| 58 | + return self.pool.call("getSlot", [{"commitment": "finalized"}]) | |
| 59 | + | |
| 60 | + def get_block(self, slot): | |
| 61 | + """Block dict, 'skipped' (no block at this slot), or None (not yet | |
| 62 | + available / node behind — retry later).""" | |
| 63 | + try: | |
| 64 | + return self.pool.call("getBlock", [slot, _BLOCK_OPTS]) | |
| 65 | + except RpcError as e: | |
| 66 | + msg = (e.message or "").lower() | |
| 67 | + if e.code in (-32007, -32009) or "skipped" in msg or "purged" in msg: | |
| 68 | + return "skipped" | |
| 69 | + if e.code == -32004 or "not available" in msg: | |
| 70 | + return None | |
| 71 | + raise | |
| 72 | + | |
| 73 | + # -- pipeline ---------------------------------------------------------- | |
| 74 | + | |
| 75 | + def process_block(self, slot, blk): | |
| 76 | + ts = blk.get("blockTime") | |
| 77 | + rows = [] | |
| 78 | + for tx in blk.get("transactions") or []: | |
| 79 | + meta = tx.get("meta") or {} | |
| 80 | + if meta.get("err") is not None: | |
| 81 | + continue | |
| 82 | + # (mint, accountIndex) → [owner, pre_amount, post_amount] | |
| 83 | + state = {} | |
| 84 | + for phase, key in (("pre", "preTokenBalances"), ("post", "postTokenBalances")): | |
| 85 | + for b in meta.get(key) or []: | |
| 86 | + if b.get("mint") not in self.tokens: | |
| 87 | + continue | |
| 88 | + k = (b["mint"], b["accountIndex"]) | |
| 89 | + entry = state.setdefault(k, [b.get("owner"), 0, 0]) | |
| 90 | + entry[1 if phase == "pre" else 2] = int(b["uiTokenAmount"]["amount"]) | |
| 91 | + sig = (tx.get("transaction") or {}).get("signatures", [None])[0] | |
| 92 | + if not sig or (not state and not self.native): | |
| 93 | + continue # nothing trackable in this tx | |
| 94 | + # whale-only native SOL from lamport balance diffs (same tx meta) | |
| 95 | + if self.native: | |
| 96 | + min_lamports = int(self.native["min"] * 10 ** self.native["decimals"]) | |
| 97 | + keys = (tx.get("transaction") or {}).get("accountKeys") or [] | |
| 98 | + pre, post = meta.get("preBalances") or [], meta.get("postBalances") or [] | |
| 99 | + deltas = [] | |
| 100 | + for i, (a, b) in enumerate(zip(pre, post)): | |
| 101 | + if i < len(keys) and b - a != 0: | |
| 102 | + k = keys[i] | |
| 103 | + deltas.append((k.get("pubkey") if isinstance(k, dict) else k, b - a)) | |
| 104 | + senders = [d for d in deltas if d[1] < 0] | |
| 105 | + frm_sol = senders[0][0] if len(senders) == 1 else None | |
| 106 | + emitted_n = 0 | |
| 107 | + for owner, d in deltas: | |
| 108 | + if d >= min_lamports: | |
| 109 | + rows.append({ | |
| 110 | + "chain": self.chain, "block": slot, | |
| 111 | + "block_hash": blk.get("blockhash"), "tx_hash": sig, | |
| 112 | + "log_index": 100000 + emitted_n, "timestamp": ts, | |
| 113 | + "token": "native", "symbol": self.native["symbol"], | |
| 114 | + "from": frm_sol, "to": owner, | |
| 115 | + "amount": str(d), "decimals": self.native["decimals"], | |
| 116 | + }) | |
| 117 | + emitted_n += 1 | |
| 118 | + per_mint = {} | |
| 119 | + for (mint, _), (owner, pre, post) in state.items(): | |
| 120 | + per_mint.setdefault(mint, []).append((owner, post - pre)) | |
| 121 | + emitted = 0 | |
| 122 | + for mint, deltas in per_mint.items(): | |
| 123 | + m = self.tokens[mint] | |
| 124 | + senders = [(o, -d) for o, d in deltas if d < 0] | |
| 125 | + receivers = [(o, d) for o, d in deltas if d > 0] | |
| 126 | + frm = senders[0][0] if len(senders) == 1 else None | |
| 127 | + for owner, amount in receivers: | |
| 128 | + rows.append({ | |
| 129 | + "chain": self.chain, | |
| 130 | + "block": slot, | |
| 131 | + "block_hash": blk.get("blockhash"), | |
| 132 | + "tx_hash": sig, | |
| 133 | + "log_index": emitted, | |
| 134 | + "timestamp": ts, | |
| 135 | + "token": mint, | |
| 136 | + "symbol": m["symbol"], | |
| 137 | + "from": frm, | |
| 138 | + "to": owner, | |
| 139 | + "amount": str(amount), | |
| 140 | + "decimals": m["decimals"], | |
| 141 | + }) | |
| 142 | + emitted += 1 | |
| 143 | + db.insert_transfers(self.conn, rows) | |
| 144 | + return len(rows) | |
| 145 | + | |
| 146 | + # -- main loop ----------------------------------------------------- | |
| 147 | + | |
| 148 | + def run(self, stop): | |
| 149 | + self.conn = db.connect(self.db_path) | |
| 150 | + cur = db.get_cursor(self.conn, self.chain) | |
| 151 | + max_lag = self.cfg.get("max_lag", 2000) | |
| 152 | + while not stop.is_set(): | |
| 153 | + try: | |
| 154 | + safe = self.head() # finalized — no confirmations needed | |
| 155 | + if cur is None: | |
| 156 | + start = max(0, safe - self.cfg.get("start_offset", 100)) | |
| 157 | + cur = (start, None, start) | |
| 158 | + db.set_cursor(self.conn, self.chain, start, None) | |
| 159 | + db.set_backfill(self.conn, self.chain, start) | |
| 160 | + log.info("%s: fresh start at slot %d", self.chain, start) | |
| 161 | + cursor, _, _ = cur | |
| 162 | + | |
| 163 | + if safe - cursor > max_lag: | |
| 164 | + jump_to = safe - 50 | |
| 165 | + log.warning("%s: %d slots behind free-RPC pace — jumping " | |
| 166 | + "%d → %d (gap not indexed)", | |
| 167 | + self.chain, safe - cursor, cursor, jump_to) | |
| 168 | + cursor = jump_to | |
| 169 | + | |
| 170 | + moved = 0 | |
| 171 | + while cursor < safe and not stop.is_set(): | |
| 172 | + blk = self.get_block(cursor + 1) | |
| 173 | + if blk is None: | |
| 174 | + break # not yet queryable — next cycle | |
| 175 | + cursor += 1 | |
| 176 | + moved += 1 | |
| 177 | + if blk != "skipped": | |
| 178 | + n = self.process_block(cursor, blk) | |
| 179 | + if n: | |
| 180 | + log.info("%s: %d transfers in slot %d (lag %d)", | |
| 181 | + self.chain, n, cursor, safe - cursor) | |
| 182 | + if moved % 20 == 0: | |
| 183 | + db.set_cursor(self.conn, self.chain, cursor, None) | |
| 184 | + if moved: | |
| 185 | + db.set_cursor(self.conn, self.chain, cursor, None, head=safe) | |
| 186 | + | |
| 187 | + cur = (cursor, None, None) | |
| 188 | + db.save_rpc_health(self.conn, self.chain, self.pool.stats(), int(time.time())) | |
| 189 | + stop.wait(max(float(self.cfg.get("block_time", 0.4)), 1.0)) | |
| 190 | + except (AllEndpointsDown, Exception) as e: | |
| 191 | + log.error("%s: %s — retrying in 10s", self.chain, e) | |
| 192 | + stop.wait(10) | |
added
indexer/adapters/tron.py
+213 −0
@@ -0,0 +1,213 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Tron adapter — TRC-20 stablecoin transfers via the HTTP wallet API. | |
| 4 | + | |
| 5 | +How a TRC-20 transfer appears on Tron: the tx is a `TriggerSmartContract` | |
| 6 | +call, and the token contract emits the SAME Transfer event as ERC-20 | |
| 7 | +(Tron's TVM is EVM-derived). We read one block's worth of transaction | |
| 8 | +infos with /wallet/gettransactioninfobyblocknum and filter the `log` | |
| 9 | +entries by contract address + Transfer topic. | |
| 10 | + | |
| 11 | +Addresses: Tron shows Base58Check "T..." addresses; on-chain they are | |
| 12 | +21 bytes (0x41 prefix + 20-byte EVM-style address). Event topics carry | |
| 13 | +the bare 20 bytes left-padded to 32 — we convert back to T-addresses so | |
| 14 | +stored rows use the chain-native format. Base58Check is implemented by | |
| 15 | +hand below (double-SHA256 checksum) — no dependency. | |
| 16 | + | |
| 17 | +Finality: DPoS, ~3s blocks; a block is effectively final once 2/3 of the | |
| 18 | +27 SRs confirm (≈ 19 blocks) — we trail head by `confirmations` (20). | |
| 19 | +""" | |
| 20 | + | |
| 21 | +import hashlib | |
| 22 | +import logging | |
| 23 | +import time | |
| 24 | + | |
| 25 | +from .. import db | |
| 26 | +from ..rpc import AllEndpointsDown, RestError, RestPool | |
| 27 | + | |
| 28 | +log = logging.getLogger("tron") | |
| 29 | + | |
| 30 | +# keccak256("Transfer(address,address,uint256)") — Tron logs omit the 0x | |
| 31 | +TRANSFER_TOPIC = "ddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" | |
| 32 | + | |
| 33 | +_B58 = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz" | |
| 34 | +_B58_INDEX = {c: i for i, c in enumerate(_B58)} | |
| 35 | + | |
| 36 | + | |
| 37 | +def _sha256d(b): | |
| 38 | + return hashlib.sha256(hashlib.sha256(b).digest()).digest() | |
| 39 | + | |
| 40 | + | |
| 41 | +def b58check_decode(s): | |
| 42 | + num = 0 | |
| 43 | + for c in s: | |
| 44 | + num = num * 58 + _B58_INDEX[c] | |
| 45 | + raw = num.to_bytes((num.bit_length() + 7) // 8, "big") | |
| 46 | + raw = b"\x00" * (len(s) - len(s.lstrip("1"))) + raw # leading '1's = zero bytes | |
| 47 | + payload, checksum = raw[:-4], raw[-4:] | |
| 48 | + if _sha256d(payload)[:4] != checksum: | |
| 49 | + raise ValueError(f"bad Base58Check checksum for {s}") | |
| 50 | + return payload | |
| 51 | + | |
| 52 | + | |
| 53 | +def b58check_encode(payload): | |
| 54 | + raw = payload + _sha256d(payload)[:4] | |
| 55 | + num = int.from_bytes(raw, "big") | |
| 56 | + out = "" | |
| 57 | + while num: | |
| 58 | + num, rem = divmod(num, 58) | |
| 59 | + out = _B58[rem] + out | |
| 60 | + return "1" * (len(raw) - len(raw.lstrip(b"\x00"))) + out | |
| 61 | + | |
| 62 | + | |
| 63 | +def taddr_to_hex20(t_addr): | |
| 64 | + """T-address → bare 20-byte hex (what appears in event logs/topics).""" | |
| 65 | + payload = b58check_decode(t_addr) | |
| 66 | + if len(payload) != 21 or payload[0] != 0x41: | |
| 67 | + raise ValueError(f"not a Tron address: {t_addr}") | |
| 68 | + return payload[1:].hex() | |
| 69 | + | |
| 70 | + | |
| 71 | +def hex20_to_taddr(h): | |
| 72 | + """Bare 20-byte hex → Base58Check T-address (0x41 mainnet prefix).""" | |
| 73 | + return b58check_encode(b"\x41" + bytes.fromhex(h)) | |
| 74 | + | |
| 75 | + | |
| 76 | +class TronIndexer: | |
| 77 | + def __init__(self, chain, cfg, tokens, db_path): | |
| 78 | + self.chain = chain | |
| 79 | + self.cfg = cfg | |
| 80 | + self.db_path = db_path | |
| 81 | + self.conn = None | |
| 82 | + self.pool = RestPool(cfg["rpcs"]) | |
| 83 | + # hex20 (as seen in logs) → token meta | |
| 84 | + self.tokens = {taddr_to_hex20(t["id"]): t for t in tokens} | |
| 85 | + self.native = cfg.get("native") # {symbol: TRX, min, decimals: 6} | |
| 86 | + | |
| 87 | + # -- API helpers ----------------------------------------------------- | |
| 88 | + | |
| 89 | + def head(self): | |
| 90 | + blk = self.pool.post("/wallet/getnowblock") | |
| 91 | + return blk["block_header"]["raw_data"]["number"] | |
| 92 | + | |
| 93 | + def block_id(self, n): | |
| 94 | + blk = self.pool.post("/wallet/getblockbynum", {"num": n}) | |
| 95 | + return (blk or {}).get("blockID") | |
| 96 | + | |
| 97 | + def block_infos(self, n): | |
| 98 | + return self.pool.post("/wallet/gettransactioninfobyblocknum", {"num": n}) or [] | |
| 99 | + | |
| 100 | + # -- pipeline ---------------------------------------------------------- | |
| 101 | + | |
| 102 | + def process_block(self, n): | |
| 103 | + rows = [] | |
| 104 | + for info in self.block_infos(n): | |
| 105 | + receipt = info.get("receipt") or {} | |
| 106 | + if receipt.get("result") not in (None, "SUCCESS"): | |
| 107 | + continue # reverted contract call — logs are not effective | |
| 108 | + for i, lg in enumerate(info.get("log") or []): | |
| 109 | + addr = (lg.get("address") or "").lower()[-40:] # strip 41 prefix if present | |
| 110 | + meta = self.tokens.get(addr) | |
| 111 | + topics = lg.get("topics") or [] | |
| 112 | + if meta is None or len(topics) != 3 or topics[0].lower() != TRANSFER_TOPIC: | |
| 113 | + continue | |
| 114 | + data = (lg.get("data") or "").strip() or "0" | |
| 115 | + rows.append({ | |
| 116 | + "chain": self.chain, | |
| 117 | + "block": n, | |
| 118 | + "block_hash": None, | |
| 119 | + "tx_hash": info["id"], | |
| 120 | + "log_index": i, | |
| 121 | + "timestamp": info.get("blockTimeStamp", 0) // 1000 or None, | |
| 122 | + "token": meta["id"], # T-address, chain-native format | |
| 123 | + "symbol": meta["symbol"], | |
| 124 | + "from": hex20_to_taddr(topics[1][-40:]), | |
| 125 | + "to": hex20_to_taddr(topics[2][-40:]), | |
| 126 | + "amount": str(int(data, 16)), | |
| 127 | + "decimals": meta["decimals"], | |
| 128 | + }) | |
| 129 | + if self.native: | |
| 130 | + rows.extend(self.native_rows(n)) | |
| 131 | + db.insert_transfers(self.conn, rows) | |
| 132 | + return len(rows) | |
| 133 | + | |
| 134 | + def native_rows(self, n): | |
| 135 | + """Whale-only native TRX: TransferContract txs above the config min. | |
| 136 | + Addresses come back as 41-prefixed hex — convert to T-addresses.""" | |
| 137 | + min_sun = int(self.native["min"] * 10 ** self.native["decimals"]) | |
| 138 | + blk = self.pool.post("/wallet/getblockbynum", {"num": n}) or {} | |
| 139 | + ts = (blk.get("block_header", {}).get("raw_data", {}).get("timestamp", 0)) // 1000 or None | |
| 140 | + rows = [] | |
| 141 | + for tx in blk.get("transactions") or []: | |
| 142 | + for ci, c in enumerate(tx.get("raw_data", {}).get("contract") or []): | |
| 143 | + if c.get("type") != "TransferContract": | |
| 144 | + continue | |
| 145 | + v = c.get("parameter", {}).get("value", {}) | |
| 146 | + amount = v.get("amount", 0) | |
| 147 | + if amount < min_sun: | |
| 148 | + continue | |
| 149 | + def cvt(h): | |
| 150 | + h = (h or "").lower() | |
| 151 | + return hex20_to_taddr(h[-40:]) if len(h) >= 40 else (h or None) | |
| 152 | + rows.append({ | |
| 153 | + "chain": self.chain, "block": n, "block_hash": None, | |
| 154 | + "tx_hash": tx["txID"], "log_index": 100000 + ci, | |
| 155 | + "timestamp": ts, "token": "native", | |
| 156 | + "symbol": self.native["symbol"], | |
| 157 | + "from": cvt(v.get("owner_address")), | |
| 158 | + "to": cvt(v.get("to_address")), | |
| 159 | + "amount": str(amount), "decimals": self.native["decimals"], | |
| 160 | + }) | |
| 161 | + return rows | |
| 162 | + | |
| 163 | + def reorged(self, cursor_block, cursor_hash): | |
| 164 | + if not cursor_hash: | |
| 165 | + return False | |
| 166 | + bid = self.block_id(cursor_block) | |
| 167 | + return bid is not None and bid.lower() != cursor_hash.lower() | |
| 168 | + | |
| 169 | + # -- main loop ----------------------------------------------------- | |
| 170 | + | |
| 171 | + def run(self, stop): | |
| 172 | + self.conn = db.connect(self.db_path) | |
| 173 | + cur = db.get_cursor(self.conn, self.chain) | |
| 174 | + confirmations = self.cfg.get("confirmations", 20) | |
| 175 | + while not stop.is_set(): | |
| 176 | + try: | |
| 177 | + head = self.head() | |
| 178 | + safe = head - confirmations | |
| 179 | + if cur is None: | |
| 180 | + start = max(0, safe - self.cfg.get("start_offset", 100)) | |
| 181 | + cur = (start, None, start) | |
| 182 | + db.set_cursor(self.conn, self.chain, start, None) | |
| 183 | + db.set_backfill(self.conn, self.chain, start) | |
| 184 | + log.info("%s: fresh start at block %d (head %d)", self.chain, start, head) | |
| 185 | + cursor_block, cursor_hash, _ = cur | |
| 186 | + | |
| 187 | + if self.reorged(cursor_block, cursor_hash): | |
| 188 | + to_block = cursor_block - 2 * confirmations | |
| 189 | + log.warning("%s: reorg at %d — rolling back to %d", | |
| 190 | + self.chain, cursor_block, to_block) | |
| 191 | + db.rollback(self.conn, self.chain, to_block) | |
| 192 | + cursor_block, cursor_hash = to_block, None | |
| 193 | + | |
| 194 | + moved = 0 | |
| 195 | + while cursor_block < safe and not stop.is_set(): | |
| 196 | + n = self.process_block(cursor_block + 1) | |
| 197 | + cursor_block += 1 | |
| 198 | + moved += 1 | |
| 199 | + if n: | |
| 200 | + log.info("%s: %d transfers in block %d (lag %d)", | |
| 201 | + self.chain, n, cursor_block, head - cursor_block) | |
| 202 | + if moved % 20 == 0: # persist progress during catch-up | |
| 203 | + db.set_cursor(self.conn, self.chain, cursor_block, None) | |
| 204 | + if moved: | |
| 205 | + cursor_hash = self.block_id(cursor_block) | |
| 206 | + db.set_cursor(self.conn, self.chain, cursor_block, cursor_hash, head=head) | |
| 207 | + | |
| 208 | + cur = (cursor_block, cursor_hash, None) | |
| 209 | + db.save_rpc_health(self.conn, self.chain, self.pool.stats(), int(time.time())) | |
| 210 | + stop.wait(max(float(self.cfg.get("block_time", 3)), 2.0)) | |
| 211 | + except (RestError, AllEndpointsDown, Exception) as e: | |
| 212 | + log.error("%s: %s — retrying in 10s", self.chain, e) | |
| 213 | + stop.wait(10) | |
added
indexer/config.py
+62 −0
@@ -0,0 +1,62 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Central config loading — YAML files + environment overrides. | |
| 4 | + | |
| 5 | +Env vars (see .env.example): | |
| 6 | + EXPLORER_DB path to the SQLite DB (default ./explorer.db) | |
| 7 | + (PostgreSQL via DATABASE_URL lands in Deliverable 5) | |
| 8 | + EXPLORER_CHAINS comma-separated subset of chains to index (default all) | |
| 9 | + EXPLORER_CONFIG directory holding chains.yaml / tokens.yaml | |
| 10 | +""" | |
| 11 | + | |
| 12 | +import os | |
| 13 | +import pathlib | |
| 14 | + | |
| 15 | +import yaml | |
| 16 | + | |
| 17 | +ROOT = pathlib.Path(__file__).parent.parent | |
| 18 | +CONFIG_DIR = pathlib.Path(os.environ.get("EXPLORER_CONFIG", ROOT / "config")) | |
| 19 | + | |
| 20 | + | |
| 21 | +def db_path(): | |
| 22 | + return os.environ.get("EXPLORER_DB", str(ROOT / "explorer.db")) | |
| 23 | + | |
| 24 | + | |
| 25 | +def load_chains(): | |
| 26 | + return yaml.safe_load((CONFIG_DIR / "chains.yaml").read_text())["chains"] | |
| 27 | + | |
| 28 | + | |
| 29 | +def load_tokens(): | |
| 30 | + return yaml.safe_load((CONFIG_DIR / "tokens.yaml").read_text())["tokens"] | |
| 31 | + | |
| 32 | + | |
| 33 | +def backfill_blocks(chain_cfg): | |
| 34 | + """How many blocks of history to grow backwards. Per-chain | |
| 35 | + `backfill_days` wins; else EXPLORER_BACKFILL_DAYS env; else 0 (off).""" | |
| 36 | + days = chain_cfg.get("backfill_days") | |
| 37 | + if days is None: | |
| 38 | + days = float(os.environ.get("EXPLORER_BACKFILL_DAYS", 0) or 0) | |
| 39 | + if not days: | |
| 40 | + return 0 | |
| 41 | + return int(days * 86400 / float(chain_cfg.get("block_time", 12))) | |
| 42 | + | |
| 43 | + | |
| 44 | +def selected_chains(chains, tokens, cli_arg=None, families=("evm",)): | |
| 45 | + """Chains to run: CLI arg > EXPLORER_CHAINS env > every chain that has | |
| 46 | + both a chain config and a token list. Chains whose family has no | |
| 47 | + adapter yet are skipped with a warning.""" | |
| 48 | + env = os.environ.get("EXPLORER_CHAINS", "").strip() | |
| 49 | + if cli_arg: | |
| 50 | + wanted = list(cli_arg) | |
| 51 | + elif env: | |
| 52 | + wanted = [c.strip() for c in env.split(",") if c.strip()] | |
| 53 | + else: | |
| 54 | + wanted = [c for c in chains if c in tokens] | |
| 55 | + unknown = [c for c in wanted if c not in chains or c not in tokens] | |
| 56 | + if unknown: | |
| 57 | + raise SystemExit(f"unknown chains (need entries in both YAML files): {unknown}") | |
| 58 | + skipped = [c for c in wanted if chains[c].get("family", "evm") not in families] | |
| 59 | + if skipped: | |
| 60 | + import logging | |
| 61 | + logging.warning("skipping chains with no adapter yet: %s", skipped) | |
| 62 | + return [c for c in wanted if chains[c].get("family", "evm") in families] | |
added
indexer/db.py
+340 −0
@@ -0,0 +1,340 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Storage layer — SQLite (dev, zero-config) or PostgreSQL (production). | |
| 4 | + | |
| 5 | +Backend selection: DATABASE_URL env set → PostgreSQL via psycopg2; | |
| 6 | +otherwise SQLite at the given path (WAL mode so indexer threads write | |
| 7 | +while the API reads). | |
| 8 | + | |
| 9 | +The DDL below is written in the dialect intersection both engines accept. | |
| 10 | +amount is TEXT: token amounts are uint256 and overflow both engines' | |
| 11 | +64-bit integers (e.g. any DAI transfer over ~9.2M). Aggregations CAST to | |
| 12 | +DOUBLE PRECISION (REAL affinity on SQLite) — fine for dashboards, not | |
| 13 | +accounting. | |
| 14 | +""" | |
| 15 | + | |
| 16 | +import os | |
| 17 | +import sqlite3 | |
| 18 | + | |
| 19 | +SCHEMA = """ | |
| 20 | +CREATE TABLE IF NOT EXISTS chains ( | |
| 21 | + chain TEXT PRIMARY KEY, | |
| 22 | + family TEXT, | |
| 23 | + chain_id BIGINT | |
| 24 | +); | |
| 25 | + | |
| 26 | +CREATE TABLE IF NOT EXISTS transfers ( | |
| 27 | + chain TEXT NOT NULL, | |
| 28 | + block BIGINT NOT NULL, | |
| 29 | + block_hash TEXT, | |
| 30 | + tx_hash TEXT NOT NULL, | |
| 31 | + log_index INTEGER NOT NULL, | |
| 32 | + timestamp BIGINT, | |
| 33 | + token TEXT NOT NULL, | |
| 34 | + symbol TEXT, | |
| 35 | + "from" TEXT, | |
| 36 | + "to" TEXT, | |
| 37 | + amount TEXT, | |
| 38 | + decimals INTEGER, | |
| 39 | + PRIMARY KEY (chain, tx_hash, log_index) | |
| 40 | +); | |
| 41 | +CREATE INDEX IF NOT EXISTS idx_transfers_token_block ON transfers (chain, token, block); | |
| 42 | +CREATE INDEX IF NOT EXISTS idx_transfers_ts ON transfers (symbol, timestamp); | |
| 43 | +CREATE INDEX IF NOT EXISTS idx_transfers_ts_only ON transfers (timestamp); | |
| 44 | +CREATE INDEX IF NOT EXISTS idx_transfers_from ON transfers ("from"); | |
| 45 | +CREATE INDEX IF NOT EXISTS idx_transfers_to ON transfers ("to"); | |
| 46 | + | |
| 47 | +-- rolling aggregates, maintained by StatsWorker (query-time scans over the | |
| 48 | +-- full window don't scale past a few million rows on free-tier hardware) | |
| 49 | +CREATE TABLE IF NOT EXISTS agg_volume ( | |
| 50 | + window TEXT NOT NULL, | |
| 51 | + symbol TEXT NOT NULL, | |
| 52 | + chain TEXT NOT NULL, | |
| 53 | + volume REAL, | |
| 54 | + transfers INTEGER, | |
| 55 | + senders INTEGER, | |
| 56 | + receivers INTEGER, | |
| 57 | + updated BIGINT, | |
| 58 | + PRIMARY KEY (window, symbol, chain) | |
| 59 | +); | |
| 60 | + | |
| 61 | +CREATE TABLE IF NOT EXISTS agg_series ( | |
| 62 | + window TEXT NOT NULL, | |
| 63 | + symbol TEXT NOT NULL, | |
| 64 | + chain TEXT NOT NULL, | |
| 65 | + t BIGINT NOT NULL, | |
| 66 | + volume REAL, | |
| 67 | + transfers INTEGER, | |
| 68 | + updated BIGINT, | |
| 69 | + PRIMARY KEY (window, symbol, chain, t) | |
| 70 | +); | |
| 71 | + | |
| 72 | +-- pre-extracted large transfers (>= WHALE_TABLE_MIN_USD), scanned | |
| 73 | +-- incrementally so /whales never rescans the transfers table | |
| 74 | +CREATE TABLE IF NOT EXISTS whale_events ( | |
| 75 | + chain TEXT NOT NULL, | |
| 76 | + block BIGINT, | |
| 77 | + tx_hash TEXT NOT NULL, | |
| 78 | + log_index INTEGER NOT NULL, | |
| 79 | + timestamp BIGINT, | |
| 80 | + token TEXT, | |
| 81 | + symbol TEXT, | |
| 82 | + "from" TEXT, | |
| 83 | + "to" TEXT, | |
| 84 | + amount TEXT, | |
| 85 | + decimals INTEGER, | |
| 86 | + usd REAL, | |
| 87 | + PRIMARY KEY (chain, tx_hash, log_index) | |
| 88 | +); | |
| 89 | +CREATE INDEX IF NOT EXISTS idx_whales_ts ON whale_events (timestamp); | |
| 90 | +CREATE INDEX IF NOT EXISTS idx_whales_usd ON whale_events (usd); | |
| 91 | + | |
| 92 | +CREATE TABLE IF NOT EXISTS cursors ( | |
| 93 | + chain TEXT PRIMARY KEY, | |
| 94 | + last_block BIGINT NOT NULL, | |
| 95 | + last_hash TEXT, | |
| 96 | + backfill_block BIGINT, | |
| 97 | + head_block BIGINT | |
| 98 | +); | |
| 99 | + | |
| 100 | +CREATE TABLE IF NOT EXISTS rpc_health ( | |
| 101 | + chain TEXT NOT NULL, | |
| 102 | + url TEXT NOT NULL, | |
| 103 | + score REAL, | |
| 104 | + ok INTEGER, | |
| 105 | + fail INTEGER, | |
| 106 | + latency_ms REAL, | |
| 107 | + cooldown_s REAL, | |
| 108 | + updated BIGINT, | |
| 109 | + PRIMARY KEY (chain, url) | |
| 110 | +); | |
| 111 | + | |
| 112 | +CREATE TABLE IF NOT EXISTS tokens ( | |
| 113 | + chain TEXT NOT NULL, | |
| 114 | + address TEXT NOT NULL, | |
| 115 | + symbol TEXT, | |
| 116 | + decimals INTEGER, | |
| 117 | + native INTEGER, | |
| 118 | + PRIMARY KEY (chain, address) | |
| 119 | +); | |
| 120 | + | |
| 121 | +CREATE TABLE IF NOT EXISTS prices ( | |
| 122 | + symbol TEXT PRIMARY KEY, | |
| 123 | + usd REAL, | |
| 124 | + updated BIGINT | |
| 125 | +); | |
| 126 | + | |
| 127 | +CREATE TABLE IF NOT EXISTS supply_snapshots ( | |
| 128 | + chain TEXT NOT NULL, | |
| 129 | + token TEXT NOT NULL, | |
| 130 | + symbol TEXT, | |
| 131 | + supply TEXT, | |
| 132 | + decimals INTEGER, | |
| 133 | + timestamp BIGINT NOT NULL, | |
| 134 | + PRIMARY KEY (chain, token, timestamp) | |
| 135 | +); | |
| 136 | +CREATE INDEX IF NOT EXISTS idx_supply_ts ON supply_snapshots (symbol, timestamp); | |
| 137 | +""" | |
| 138 | + | |
| 139 | +COLUMNS = [ | |
| 140 | + "chain", "block", "block_hash", "tx_hash", "log_index", "timestamp", | |
| 141 | + "token", "symbol", "from", "to", "amount", "decimals", | |
| 142 | +] | |
| 143 | + | |
| 144 | +# shared SQL fragments: face-value USD (stablecoins ≈ $1) scaled by the | |
| 145 | +# latest known price for crypto assets; CASE avoids relying on POW() | |
| 146 | +USD_EXPR = ( | |
| 147 | + "CAST(amount AS DOUBLE PRECISION) / (CASE decimals " | |
| 148 | + + " ".join(f"WHEN {d} THEN 1e{d}" for d in range(19)) | |
| 149 | + + " ELSE 1e6 END)" | |
| 150 | +) | |
| 151 | +PRICE_JOIN = "LEFT JOIN prices ON prices.symbol = transfers.symbol" | |
| 152 | +USD_PRICED = f"(({USD_EXPR}) * COALESCE(prices.usd, 1.0))" | |
| 153 | + | |
| 154 | + | |
| 155 | +def is_postgres(): | |
| 156 | + return bool(os.environ.get("DATABASE_URL")) | |
| 157 | + | |
| 158 | + | |
| 159 | +class _PgConn: | |
| 160 | + """Thin psycopg2 wrapper exposing the sqlite3 surface our code uses. | |
| 161 | + Translates '?' placeholders to '%s' (no literal '?' appears in queries). | |
| 162 | + Rows come from DictCursor: indexable by position AND by column name.""" | |
| 163 | + | |
| 164 | + def __init__(self, dsn): | |
| 165 | + import psycopg2 | |
| 166 | + import psycopg2.extras | |
| 167 | + self._x = psycopg2.extras | |
| 168 | + self.raw = psycopg2.connect(dsn) | |
| 169 | + | |
| 170 | + def execute(self, sql, params=()): | |
| 171 | + cur = self.raw.cursor(cursor_factory=self._x.DictCursor) | |
| 172 | + cur.execute(sql.replace("?", "%s"), params) | |
| 173 | + return cur | |
| 174 | + | |
| 175 | + def executemany(self, sql, seq): | |
| 176 | + cur = self.raw.cursor() | |
| 177 | + cur.executemany(sql.replace("?", "%s"), list(seq)) | |
| 178 | + return cur | |
| 179 | + | |
| 180 | + def executescript(self, script): | |
| 181 | + cur = self.raw.cursor() | |
| 182 | + cur.execute(script) | |
| 183 | + self.raw.commit() | |
| 184 | + | |
| 185 | + def commit(self): | |
| 186 | + self.raw.commit() | |
| 187 | + | |
| 188 | + def close(self): | |
| 189 | + self.raw.close() | |
| 190 | + | |
| 191 | + | |
| 192 | +def connect(path=None, readonly=False): | |
| 193 | + dsn = os.environ.get("DATABASE_URL") | |
| 194 | + if dsn: | |
| 195 | + conn = _PgConn(dsn) | |
| 196 | + if not readonly: | |
| 197 | + conn.executescript(SCHEMA) | |
| 198 | + _migrate(conn) | |
| 199 | + return conn | |
| 200 | + if readonly: | |
| 201 | + conn = sqlite3.connect(f"file:{path}?mode=ro", uri=True, timeout=10) | |
| 202 | + conn.row_factory = sqlite3.Row | |
| 203 | + return conn | |
| 204 | + conn = sqlite3.connect(path, timeout=30) | |
| 205 | + conn.row_factory = sqlite3.Row # rows usable by name AND position | |
| 206 | + conn.execute("PRAGMA journal_mode=WAL") | |
| 207 | + conn.execute("PRAGMA synchronous=NORMAL") | |
| 208 | + conn.executescript(SCHEMA) | |
| 209 | + _migrate(conn) | |
| 210 | + return conn | |
| 211 | + | |
| 212 | + | |
| 213 | +def _migrate(conn): | |
| 214 | + """Additive column migrations for DBs created by earlier versions.""" | |
| 215 | + for col in ("backfill_block", "head_block"): | |
| 216 | + try: | |
| 217 | + conn.execute(f"ALTER TABLE cursors ADD COLUMN {col} BIGINT") | |
| 218 | + conn.commit() | |
| 219 | + except Exception: | |
| 220 | + try: # psycopg2 aborts the tx on error — reset it | |
| 221 | + conn.raw.rollback() | |
| 222 | + except AttributeError: | |
| 223 | + pass | |
| 224 | + | |
| 225 | + | |
| 226 | +def insert_transfers(conn, rows): | |
| 227 | + if not rows: | |
| 228 | + return | |
| 229 | + cols = ", ".join(f'"{c}"' for c in COLUMNS) | |
| 230 | + ph = ", ".join("?" for _ in COLUMNS) | |
| 231 | + conn.executemany( | |
| 232 | + f"INSERT INTO transfers ({cols}) VALUES ({ph}) ON CONFLICT DO NOTHING", | |
| 233 | + [tuple(r[c] for c in COLUMNS) for r in rows], | |
| 234 | + ) | |
| 235 | + conn.commit() | |
| 236 | + | |
| 237 | + | |
| 238 | +def get_cursor(conn, chain): | |
| 239 | + row = conn.execute( | |
| 240 | + "SELECT last_block, last_hash, backfill_block FROM cursors WHERE chain = ?", | |
| 241 | + (chain,), | |
| 242 | + ).fetchone() | |
| 243 | + return (row[0], row[1], row[2]) if row else None | |
| 244 | + | |
| 245 | + | |
| 246 | +def set_cursor(conn, chain, block, block_hash, head=None): | |
| 247 | + conn.execute( | |
| 248 | + "INSERT INTO cursors (chain, last_block, last_hash, head_block) VALUES (?, ?, ?, ?) " | |
| 249 | + "ON CONFLICT (chain) DO UPDATE SET last_block = ?, last_hash = ?, " | |
| 250 | + "head_block = COALESCE(?, cursors.head_block)", | |
| 251 | + (chain, block, block_hash, head, block, block_hash, head), | |
| 252 | + ) | |
| 253 | + conn.commit() | |
| 254 | + | |
| 255 | + | |
| 256 | +def set_backfill(conn, chain, block): | |
| 257 | + conn.execute( | |
| 258 | + "UPDATE cursors SET backfill_block = ? WHERE chain = ?", (block, chain) | |
| 259 | + ) | |
| 260 | + conn.commit() | |
| 261 | + | |
| 262 | + | |
| 263 | +def rollback(conn, chain, to_block): | |
| 264 | + """Reorg: drop everything above to_block and rewind the cursor.""" | |
| 265 | + conn.execute( | |
| 266 | + "DELETE FROM transfers WHERE chain = ? AND block > ?", (chain, to_block) | |
| 267 | + ) | |
| 268 | + conn.execute( | |
| 269 | + "UPDATE cursors SET last_block = ?, last_hash = NULL WHERE chain = ?", | |
| 270 | + (to_block, chain), | |
| 271 | + ) | |
| 272 | + conn.commit() | |
| 273 | + | |
| 274 | + | |
| 275 | +def save_rpc_health(conn, chain, stats, now): | |
| 276 | + conn.executemany( | |
| 277 | + "INSERT INTO rpc_health (chain, url, score, ok, fail, latency_ms, cooldown_s, updated) " | |
| 278 | + "VALUES (?, ?, ?, ?, ?, ?, ?, ?) " | |
| 279 | + "ON CONFLICT (chain, url) DO UPDATE SET score = excluded.score, ok = excluded.ok, " | |
| 280 | + "fail = excluded.fail, latency_ms = excluded.latency_ms, " | |
| 281 | + "cooldown_s = excluded.cooldown_s, updated = excluded.updated", | |
| 282 | + [ | |
| 283 | + (chain, s["url"], s["score"], s["ok"], s["fail"], s["latency_ms"], s["cooldown_s"], now) | |
| 284 | + for s in stats | |
| 285 | + ], | |
| 286 | + ) | |
| 287 | + conn.commit() | |
| 288 | + | |
| 289 | + | |
| 290 | +def upsert_chain(conn, chain, family, chain_id): | |
| 291 | + conn.execute( | |
| 292 | + "INSERT INTO chains (chain, family, chain_id) VALUES (?, ?, ?) " | |
| 293 | + "ON CONFLICT (chain) DO UPDATE SET family = ?, chain_id = ?", | |
| 294 | + (chain, family, chain_id, family, chain_id), | |
| 295 | + ) | |
| 296 | + conn.commit() | |
| 297 | + | |
| 298 | + | |
| 299 | +def upsert_tokens(conn, chain, tokens): | |
| 300 | + conn.executemany( | |
| 301 | + "INSERT INTO tokens (chain, address, symbol, decimals, native) " | |
| 302 | + "VALUES (?, ?, ?, ?, ?) " | |
| 303 | + "ON CONFLICT (chain, address) DO UPDATE SET " | |
| 304 | + "symbol = excluded.symbol, decimals = excluded.decimals, native = excluded.native", | |
| 305 | + [ | |
| 306 | + (chain, token_key(t), t["symbol"], t.get("decimals"), int(t.get("native", False))) | |
| 307 | + for t in tokens | |
| 308 | + ], | |
| 309 | + ) | |
| 310 | + conn.commit() | |
| 311 | + | |
| 312 | + | |
| 313 | +def insert_supply(conn, rows): | |
| 314 | + if not rows: | |
| 315 | + return | |
| 316 | + conn.executemany( | |
| 317 | + "INSERT INTO supply_snapshots (chain, token, symbol, supply, decimals, timestamp) " | |
| 318 | + "VALUES (?, ?, ?, ?, ?, ?) ON CONFLICT DO NOTHING", | |
| 319 | + rows, | |
| 320 | + ) | |
| 321 | + conn.commit() | |
| 322 | + | |
| 323 | + | |
| 324 | +def upsert_prices(conn, rows): | |
| 325 | + """rows: [(symbol, usd, updated)]""" | |
| 326 | + if not rows: | |
| 327 | + return | |
| 328 | + conn.executemany( | |
| 329 | + "INSERT INTO prices (symbol, usd, updated) VALUES (?, ?, ?) " | |
| 330 | + "ON CONFLICT (symbol) DO UPDATE SET usd = excluded.usd, updated = excluded.updated", | |
| 331 | + rows, | |
| 332 | + ) | |
| 333 | + conn.commit() | |
| 334 | + | |
| 335 | + | |
| 336 | +def token_key(t): | |
| 337 | + """Canonical stored identifier: EVM addresses lowercase; everything else | |
| 338 | + (Base58, coin types, denoms…) is case-sensitive and kept verbatim.""" | |
| 339 | + key = t.get("address") or t.get("id") | |
| 340 | + return key.lower() if key.startswith("0x") else key | |
added
indexer/decode.py
+55 −0
@@ -0,0 +1,55 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Hand-rolled ERC-20 event decoding — no ABI library. | |
| 4 | + | |
| 5 | +An ERC-20 transfer is a contract call emitting: | |
| 6 | + Transfer(address indexed from, address indexed to, uint256 value) | |
| 7 | + | |
| 8 | +In the log: | |
| 9 | + topics[0] = keccak256("Transfer(address,address,uint256)") | |
| 10 | + topics[1] = from, left-padded to 32 bytes (indexed params live in topics) | |
| 11 | + topics[2] = to, left-padded to 32 bytes | |
| 12 | + data = value as one 32-byte big-endian word (non-indexed) | |
| 13 | +""" | |
| 14 | + | |
| 15 | +from decimal import Decimal | |
| 16 | + | |
| 17 | +TRANSFER_TOPIC = ( | |
| 18 | + "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" | |
| 19 | +) | |
| 20 | + | |
| 21 | + | |
| 22 | +def topic_address(topic): | |
| 23 | + """A 32-byte topic holding an address: the address is the last 20 bytes.""" | |
| 24 | + return "0x" + topic[-40:].lower() | |
| 25 | + | |
| 26 | + | |
| 27 | +def word_uint(hexdata): | |
| 28 | + if hexdata in (None, "0x", ""): | |
| 29 | + return 0 | |
| 30 | + return int(hexdata, 16) | |
| 31 | + | |
| 32 | + | |
| 33 | +def format_amount(raw, decimals): | |
| 34 | + """Human-readable amount from raw integer units (raw may be str or int).""" | |
| 35 | + q = Decimal(int(raw)) / (Decimal(10) ** decimals) | |
| 36 | + return format(q.normalize(), "f") | |
| 37 | + | |
| 38 | + | |
| 39 | +def decode_transfer(chain, log, token_meta, timestamp): | |
| 40 | + """Normalize one raw Transfer log into the canonical cross-chain event.""" | |
| 41 | + return { | |
| 42 | + "chain": chain, | |
| 43 | + "block": int(log["blockNumber"], 16), | |
| 44 | + "block_hash": (log.get("blockHash") or "").lower() or None, | |
| 45 | + "tx_hash": log["transactionHash"].lower(), | |
| 46 | + "log_index": int(log["logIndex"], 16), | |
| 47 | + "timestamp": timestamp, | |
| 48 | + "token": log["address"].lower(), | |
| 49 | + "symbol": token_meta["symbol"], | |
| 50 | + "from": topic_address(log["topics"][1]), | |
| 51 | + "to": topic_address(log["topics"][2]), | |
| 52 | + # uint256 overflows SQLite's int64 — store raw units as TEXT | |
| 53 | + "amount": str(word_uint(log["data"])), | |
| 54 | + "decimals": token_meta["decimals"], | |
| 55 | + } | |
added
indexer/enrich.py
+311 −0
@@ -0,0 +1,311 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Enrichment worker — periodic supply snapshots per token per chain. | |
| 4 | + | |
| 5 | +Circulating supply is read straight from each chain (no market-data APIs): | |
| 6 | + EVM eth_call totalSupply() (selector 0x18160ddd) | |
| 7 | + Tron /wallet/triggerconstantcontract totalSupply() | |
| 8 | + Solana getTokenSupply(mint) | |
| 9 | + | |
| 10 | +One snapshot at startup (so the API has data immediately), then every | |
| 11 | +EXPLORER_SUPPLY_INTERVAL seconds (default hourly). Snapshots are keyed | |
| 12 | +(chain, token, timestamp) so history accumulates for supply charts. | |
| 13 | + | |
| 14 | +Mint/burn and whale detection are query-time concerns (see api/main.py): | |
| 15 | +mints are transfers FROM the zero address, burns TO it — no extra state. | |
| 16 | +""" | |
| 17 | + | |
| 18 | +import logging | |
| 19 | +import os | |
| 20 | +import time | |
| 21 | + | |
| 22 | +from . import db | |
| 23 | +from .rpc import AllEndpointsDown, RestError, RestPool, RpcError, RpcPool | |
| 24 | + | |
| 25 | +log = logging.getLogger("enrich") | |
| 26 | + | |
| 27 | +SEL_TOTAL_SUPPLY = "0x18160ddd" # keccak4("totalSupply()") | |
| 28 | + | |
| 29 | +# transfers from/to these are mints/burns (per family; Solana mints appear | |
| 30 | +# as balance increases with no sender — see adapter docs) | |
| 31 | +ZERO_ADDRESSES = { | |
| 32 | + "evm": "0x" + "00" * 20, | |
| 33 | + "tron": "T9yD14Nj9j7xAB4dbGeiX9h8unkKHxuWwb", # Base58Check of 0x41 + 20 zero bytes | |
| 34 | +} | |
| 35 | + | |
| 36 | + | |
| 37 | +class PriceWorker: | |
| 38 | + """USD prices for non-stable assets via CoinGecko's keyless free tier — | |
| 39 | + ONE bulk request per refresh. Stablecoins are seeded at 1.0. If the fetch | |
| 40 | + fails, previous prices stay in place (staleness is logged, not fatal).""" | |
| 41 | + | |
| 42 | + def __init__(self, tokens, db_path): | |
| 43 | + import pathlib | |
| 44 | + | |
| 45 | + import requests as _rq | |
| 46 | + import yaml | |
| 47 | + self.rq = _rq | |
| 48 | + cfg_path = pathlib.Path(__file__).parent.parent / "config" / "prices.yaml" | |
| 49 | + cfg = yaml.safe_load(cfg_path.read_text()) if cfg_path.exists() else {} | |
| 50 | + self.ids = cfg.get("ids", {}) | |
| 51 | + self.interval = int(cfg.get("refresh_seconds", 300)) | |
| 52 | + self.stables = sorted({ | |
| 53 | + t["symbol"] for toks in tokens.values() for t in toks | |
| 54 | + if t.get("category", "stablecoin") == "stablecoin" | |
| 55 | + }) | |
| 56 | + self.db_path = db_path | |
| 57 | + self.conn = None | |
| 58 | + self._fail_streak = 0 | |
| 59 | + | |
| 60 | + def fetch(self): | |
| 61 | + ids = ",".join(sorted(set(self.ids.values()))) | |
| 62 | + r = self.rq.get( | |
| 63 | + "https://api.coingecko.com/api/v3/simple/price", | |
| 64 | + params={"ids": ids, "vs_currencies": "usd"}, timeout=25, | |
| 65 | + ) | |
| 66 | + if r.status_code == 429: | |
| 67 | + raise RuntimeError("coingecko rate limited") | |
| 68 | + r.raise_for_status() | |
| 69 | + data = r.json() | |
| 70 | + now = int(time.time()) | |
| 71 | + rows = [] | |
| 72 | + for sym, cid in self.ids.items(): | |
| 73 | + usd = (data.get(cid) or {}).get("usd") | |
| 74 | + if usd is not None: | |
| 75 | + rows.append((sym, float(usd), now)) | |
| 76 | + return rows | |
| 77 | + | |
| 78 | + def run(self, stop): | |
| 79 | + self.conn = db.connect(self.db_path) | |
| 80 | + now = int(time.time()) | |
| 81 | + db.upsert_prices(self.conn, [(s, 1.0, now) for s in self.stables]) | |
| 82 | + while not stop.is_set(): | |
| 83 | + try: | |
| 84 | + rows = self.fetch() | |
| 85 | + db.upsert_prices(self.conn, rows) | |
| 86 | + self._fail_streak = 0 | |
| 87 | + log.info("prices: %d symbols refreshed", len(rows)) | |
| 88 | + except Exception as e: | |
| 89 | + self._fail_streak += 1 | |
| 90 | + log.warning("price fetch failed (%s) — keeping previous prices " | |
| 91 | + "(%d consecutive failures)", e, self._fail_streak) | |
| 92 | + # back off harder when the free tier pushes back | |
| 93 | + stop.wait(self.interval * min(4, 1 + self._fail_streak)) | |
| 94 | + | |
| 95 | + | |
| 96 | +class StatsWorker: | |
| 97 | + """Maintains the fast-read tables the API serves from: | |
| 98 | + | |
| 99 | + - whale_events: incremental scan of NEW transfer rows (by rowid on | |
| 100 | + SQLite, timestamp overlap on PG), extracting everything above the | |
| 101 | + USD floor. /whales reads this tiny table instead of rescanning | |
| 102 | + millions of transfer rows. | |
| 103 | + - agg_volume: rolling volume/transfers/active-address aggregates per | |
| 104 | + (window, symbol, chain). Heavy COUNT(DISTINCT) work happens HERE, | |
| 105 | + off the request path, every few minutes. | |
| 106 | + """ | |
| 107 | + | |
| 108 | + WHALE_FLOOR = float(os.environ.get("WHALE_TABLE_MIN_USD", 100_000)) | |
| 109 | + AGG_WINDOWS = {"1h": (3600, 180), "24h": (86400, 300), "7d": (604800, 1800)} | |
| 110 | + SERIES_STEP = {"1h": 300, "24h": 3600, "7d": 21600} # matches the UI | |
| 111 | + | |
| 112 | + def __init__(self, db_path): | |
| 113 | + self.db_path = db_path | |
| 114 | + self.conn = None | |
| 115 | + self._agg_last = {w: 0.0 for w in self.AGG_WINDOWS} | |
| 116 | + | |
| 117 | + # -- whale extraction -------------------------------------------------- | |
| 118 | + | |
| 119 | + def _whale_cursor(self): | |
| 120 | + row = self.conn.execute( | |
| 121 | + "SELECT last_block FROM cursors WHERE chain = '_whale_scan'" | |
| 122 | + ).fetchone() | |
| 123 | + return row[0] if row else 0 | |
| 124 | + | |
| 125 | + def scan_whales(self): | |
| 126 | + last = self._whale_cursor() | |
| 127 | + if db.is_postgres(): | |
| 128 | + # no rowid on PG: rescan a 10-min overlap; the PK dedupes | |
| 129 | + cond, args, head = "transfers.timestamp >= ?", [int(time.time()) - 600], last | |
| 130 | + else: | |
| 131 | + # snapshot the head rowid FIRST — rows inserted while we scan are | |
| 132 | + # picked up next cycle instead of being skipped forever | |
| 133 | + head = self.conn.execute( | |
| 134 | + "SELECT COALESCE(MAX(rowid), 0) FROM transfers").fetchone()[0] | |
| 135 | + cond, args = "transfers.rowid > ? AND transfers.rowid <= ?", [last, head] | |
| 136 | + rows = self.conn.execute( | |
| 137 | + f"SELECT transfers.*, {db.USD_PRICED} AS usd " | |
| 138 | + f"FROM transfers {db.PRICE_JOIN} WHERE {cond} AND {db.USD_PRICED} >= ?", | |
| 139 | + args + [self.WHALE_FLOOR], | |
| 140 | + ).fetchall() | |
| 141 | + if rows: | |
| 142 | + self.conn.executemany( | |
| 143 | + 'INSERT INTO whale_events (chain, block, tx_hash, log_index, timestamp, ' | |
| 144 | + 'token, symbol, "from", "to", amount, decimals, usd) ' | |
| 145 | + "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT DO NOTHING", | |
| 146 | + [(r["chain"], r["block"], r["tx_hash"], r["log_index"], r["timestamp"], | |
| 147 | + r["token"], r["symbol"], r["from"], r["to"], r["amount"], | |
| 148 | + r["decimals"], round(r["usd"], 2)) for r in rows], | |
| 149 | + ) | |
| 150 | + self.conn.execute( | |
| 151 | + "INSERT INTO cursors (chain, last_block, last_hash) VALUES ('_whale_scan', ?, NULL) " | |
| 152 | + "ON CONFLICT (chain) DO UPDATE SET last_block = ?", | |
| 153 | + (head, head), | |
| 154 | + ) | |
| 155 | + self.conn.commit() | |
| 156 | + if rows: | |
| 157 | + log.info("whales: +%d events (floor $%d)", len(rows), self.WHALE_FLOOR) | |
| 158 | + | |
| 159 | + # -- rolling aggregates -------------------------------------------------- | |
| 160 | + | |
| 161 | + def compute_agg(self, window, seconds): | |
| 162 | + since = int(time.time()) - seconds | |
| 163 | + rows = self.conn.execute( | |
| 164 | + "SELECT transfers.symbol AS symbol, chain, decimals, COUNT(*) AS n, " | |
| 165 | + "SUM(CAST(amount AS DOUBLE PRECISION)) AS raw_sum, " | |
| 166 | + 'COUNT(DISTINCT "from") AS senders, COUNT(DISTINCT "to") AS receivers, ' | |
| 167 | + "COALESCE(MAX(prices.usd), 1.0) AS price " | |
| 168 | + f"FROM transfers {db.PRICE_JOIN} WHERE timestamp >= ? " | |
| 169 | + "GROUP BY transfers.symbol, chain, decimals", | |
| 170 | + (since,), | |
| 171 | + ).fetchall() | |
| 172 | + now = int(time.time()) | |
| 173 | + acc = {} | |
| 174 | + for r in rows: | |
| 175 | + key = (r["symbol"], r["chain"]) | |
| 176 | + e = acc.setdefault(key, [0.0, 0, 0, 0]) | |
| 177 | + e[0] += (r["raw_sum"] or 0) / 10 ** r["decimals"] * (r["price"] or 1.0) | |
| 178 | + e[1] += r["n"] | |
| 179 | + e[2] += r["senders"] | |
| 180 | + e[3] += r["receivers"] | |
| 181 | + self.conn.execute("DELETE FROM agg_volume WHERE window = ?", (window,)) | |
| 182 | + self.conn.executemany( | |
| 183 | + "INSERT INTO agg_volume (window, symbol, chain, volume, transfers, " | |
| 184 | + "senders, receivers, updated) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", | |
| 185 | + [(window, s, c, round(v[0], 2), v[1], v[2], v[3], now) | |
| 186 | + for (s, c), v in acc.items()], | |
| 187 | + ) | |
| 188 | + self.conn.commit() | |
| 189 | + log.info("agg %s: %d (symbol, chain) rows", window, len(acc)) | |
| 190 | + | |
| 191 | + def compute_series(self, window, seconds): | |
| 192 | + """Bucketed volume per (symbol, chain) — ONE scan covers every token, | |
| 193 | + so the chart endpoint never aggregates on the request path.""" | |
| 194 | + step = self.SERIES_STEP[window] | |
| 195 | + since = int(time.time()) - seconds | |
| 196 | + rows = self.conn.execute( | |
| 197 | + f"SELECT (timestamp / {step}) * {step} AS t, transfers.symbol AS symbol, " | |
| 198 | + "chain, decimals, COUNT(*) AS n, " | |
| 199 | + "SUM(CAST(amount AS DOUBLE PRECISION)) AS raw_sum, " | |
| 200 | + "COALESCE(MAX(prices.usd), 1.0) AS price " | |
| 201 | + f"FROM transfers {db.PRICE_JOIN} WHERE timestamp >= ? " | |
| 202 | + "GROUP BY t, transfers.symbol, chain, decimals", | |
| 203 | + (since,), | |
| 204 | + ).fetchall() | |
| 205 | + now = int(time.time()) | |
| 206 | + acc = {} | |
| 207 | + for r in rows: | |
| 208 | + key = (r["symbol"], r["chain"], r["t"]) | |
| 209 | + e = acc.setdefault(key, [0.0, 0]) | |
| 210 | + e[0] += (r["raw_sum"] or 0) / 10 ** r["decimals"] * (r["price"] or 1.0) | |
| 211 | + e[1] += r["n"] | |
| 212 | + self.conn.execute("DELETE FROM agg_series WHERE window = ?", (window,)) | |
| 213 | + self.conn.executemany( | |
| 214 | + "INSERT INTO agg_series (window, symbol, chain, t, volume, transfers, updated) " | |
| 215 | + "VALUES (?, ?, ?, ?, ?, ?, ?)", | |
| 216 | + [(window, s, c, t, round(v[0], 2), v[1], now) for (s, c, t), v in acc.items()], | |
| 217 | + ) | |
| 218 | + self.conn.commit() | |
| 219 | + log.info("series %s: %d bucket rows", window, len(acc)) | |
| 220 | + | |
| 221 | + def run(self, stop): | |
| 222 | + self.conn = db.connect(self.db_path) | |
| 223 | + while not stop.is_set(): | |
| 224 | + try: | |
| 225 | + self.scan_whales() | |
| 226 | + except Exception as e: | |
| 227 | + log.warning("whale scan failed: %s", e) | |
| 228 | + now = time.monotonic() | |
| 229 | + for window, (seconds, every) in self.AGG_WINDOWS.items(): | |
| 230 | + if now - self._agg_last[window] >= every and not stop.is_set(): | |
| 231 | + try: | |
| 232 | + self.compute_agg(window, seconds) | |
| 233 | + self.compute_series(window, seconds) | |
| 234 | + self._agg_last[window] = time.monotonic() | |
| 235 | + except Exception as e: | |
| 236 | + log.warning("agg %s failed: %s", window, e) | |
| 237 | + stop.wait(60) | |
| 238 | + | |
| 239 | + | |
| 240 | +class SupplyWorker: | |
| 241 | + def __init__(self, chains, tokens, db_path): | |
| 242 | + self.chains = chains | |
| 243 | + self.tokens = tokens | |
| 244 | + self.db_path = db_path | |
| 245 | + self.conn = None | |
| 246 | + self.interval = int(os.environ.get("EXPLORER_SUPPLY_INTERVAL", 3600)) | |
| 247 | + self._pools = {} | |
| 248 | + | |
| 249 | + def pool(self, chain): | |
| 250 | + if chain not in self._pools: | |
| 251 | + cfg = self.chains[chain] | |
| 252 | + cls = RestPool if cfg.get("family") == "tron" else RpcPool | |
| 253 | + self._pools[chain] = cls(cfg["rpcs"]) | |
| 254 | + return self._pools[chain] | |
| 255 | + | |
| 256 | + # -- per-family supply reads ----------------------------------------- | |
| 257 | + | |
| 258 | + def evm_supply(self, chain, token): | |
| 259 | + res = self.pool(chain).call( | |
| 260 | + "eth_call", [{"to": token["address"], "data": SEL_TOTAL_SUPPLY}, "latest"] | |
| 261 | + ) | |
| 262 | + return int(res, 16) if res not in (None, "0x") else None | |
| 263 | + | |
| 264 | + def tron_supply(self, chain, token): | |
| 265 | + res = self.pool(chain).post("/wallet/triggerconstantcontract", { | |
| 266 | + "owner_address": ZERO_ADDRESSES["tron"], | |
| 267 | + "contract_address": token["id"], | |
| 268 | + "function_selector": "totalSupply()", | |
| 269 | + "visible": True, | |
| 270 | + }) | |
| 271 | + out = (res or {}).get("constant_result") or [] | |
| 272 | + return int(out[0], 16) if out else None | |
| 273 | + | |
| 274 | + def solana_supply(self, chain, token): | |
| 275 | + res = self.pool(chain).call("getTokenSupply", [token["id"]]) | |
| 276 | + val = (res or {}).get("value") or {} | |
| 277 | + return int(val["amount"]) if "amount" in val else None | |
| 278 | + | |
| 279 | + READERS = {"evm": evm_supply, "tron": tron_supply, "solana": solana_supply} | |
| 280 | + | |
| 281 | + # -- worker loop ------------------------------------------------------- | |
| 282 | + | |
| 283 | + def snapshot_once(self): | |
| 284 | + now = int(time.time()) | |
| 285 | + rows = [] | |
| 286 | + for chain, toks in self.tokens.items(): | |
| 287 | + family = self.chains.get(chain, {}).get("family", "evm") | |
| 288 | + reader = self.READERS.get(family) | |
| 289 | + if reader is None: | |
| 290 | + continue # family not indexed yet — no supply either | |
| 291 | + for t in toks: | |
| 292 | + try: | |
| 293 | + supply = reader(self, chain, t) | |
| 294 | + except (RpcError, RestError, AllEndpointsDown, ValueError) as e: | |
| 295 | + log.warning("supply %s/%s failed: %s", chain, t["symbol"], e) | |
| 296 | + continue | |
| 297 | + if supply is not None: | |
| 298 | + rows.append((chain, db.token_key(t), t["symbol"], str(supply), | |
| 299 | + t.get("decimals"), now)) | |
| 300 | + db.insert_supply(self.conn, rows) | |
| 301 | + log.info("supply snapshot: %d entries @ %d", len(rows), now) | |
| 302 | + return len(rows) | |
| 303 | + | |
| 304 | + def run(self, stop): | |
| 305 | + self.conn = db.connect(self.db_path) | |
| 306 | + while not stop.is_set(): | |
| 307 | + try: | |
| 308 | + self.snapshot_once() | |
| 309 | + except Exception as e: | |
| 310 | + log.error("snapshot failed: %s — retrying next interval", e) | |
| 311 | + stop.wait(self.interval) | |
added
indexer/ingest.py
+277 −0
@@ -0,0 +1,277 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Per-chain EVM ingestion loop. | |
| 4 | + | |
| 5 | +Head tailing: poll head → compute safe head (head - confirmations) → | |
| 6 | +verify the chain still links to our cursor (parent-hash check) → sweep | |
| 7 | +forward in eth_getLogs ranges → decode → insert → advance cursor. | |
| 8 | + | |
| 9 | +Backfill: after the head is caught up, grow history BACKWARDS one | |
| 10 | +slice per cycle (head tailing always has priority) until coverage reaches | |
| 11 | +`backfill_days` (chain config) / EXPLORER_BACKFILL_DAYS (env). The | |
| 12 | +`backfill_block` cursor is the lowest indexed block. | |
| 13 | + | |
| 14 | +Reorg safety: two independent checks — (1) each cycle re-fetches the | |
| 15 | +cursor block and compares its hash; (2) each advance fetches the next | |
| 16 | +block's header and verifies parentHash == our stored cursor hash. On | |
| 17 | +mismatch: delete the last 2×confirmations blocks of transfers and rewind. | |
| 18 | + | |
| 19 | +Range sizing is adaptive: free RPCs cap getLogs responses, so a semantic | |
| 20 | +refusal halves the range (down to 2); refusal at minimum range benches the | |
| 21 | +endpoint via pool.rotate(). Ten clean sweeps grow the range back. | |
| 22 | +""" | |
| 23 | + | |
| 24 | +import logging | |
| 25 | +import time | |
| 26 | + | |
| 27 | +from . import config, db | |
| 28 | +from .decode import TRANSFER_TOPIC, decode_transfer | |
| 29 | +from .rpc import RpcError, RpcPool | |
| 30 | + | |
| 31 | +log = logging.getLogger("ingest") | |
| 32 | + | |
| 33 | + | |
| 34 | +class ChainIndexer: | |
| 35 | + def __init__(self, chain, cfg, tokens, db_path): | |
| 36 | + self.chain = chain | |
| 37 | + self.cfg = cfg | |
| 38 | + self.db_path = db_path | |
| 39 | + self.conn = None # opened in run(): sqlite conns are thread-bound | |
| 40 | + self.pool = RpcPool(cfg["rpcs"]) | |
| 41 | + self.tokens = {t["address"].lower(): t for t in tokens} | |
| 42 | + self.max_range = cfg.get("max_range", 100) | |
| 43 | + self.range = self.max_range | |
| 44 | + self.backfill_blocks = config.backfill_blocks(cfg) | |
| 45 | + self.native = cfg.get("native") # {symbol, min, decimals} — whale-only | |
| 46 | + self._clean_sweeps = 0 | |
| 47 | + | |
| 48 | + # -- RPC helpers --------------------------------------------------- | |
| 49 | + | |
| 50 | + def head(self): | |
| 51 | + return int(self.pool.call("eth_blockNumber"), 16) | |
| 52 | + | |
| 53 | + def block_header(self, number): | |
| 54 | + return self.pool.call("eth_getBlockByNumber", [hex(number), False]) | |
| 55 | + | |
| 56 | + def fetch_logs(self, frm, to): | |
| 57 | + return self.pool.call( | |
| 58 | + "eth_getLogs", | |
| 59 | + [{ | |
| 60 | + "fromBlock": hex(frm), | |
| 61 | + "toBlock": hex(to), | |
| 62 | + "address": list(self.tokens.keys()), | |
| 63 | + "topics": [TRANSFER_TOPIC], | |
| 64 | + }], | |
| 65 | + ) | |
| 66 | + | |
| 67 | + def block_timestamps(self, numbers): | |
| 68 | + """Batch-fetch headers just for blocks that actually had transfers.""" | |
| 69 | + out = {} | |
| 70 | + nums = sorted(numbers) | |
| 71 | + for i in range(0, len(nums), 20): # small chunks: free nodes cap batches | |
| 72 | + chunk = nums[i : i + 20] | |
| 73 | + results = self.pool.batch( | |
| 74 | + [("eth_getBlockByNumber", [hex(n), False]) for n in chunk] | |
| 75 | + ) | |
| 76 | + for n, blk in zip(chunk, results): | |
| 77 | + if blk: | |
| 78 | + out[n] = int(blk["timestamp"], 16) | |
| 79 | + # batches drop entries under rate limits — retry stragglers one by one | |
| 80 | + for n in nums: | |
| 81 | + if n not in out: | |
| 82 | + try: | |
| 83 | + blk = self.block_header(n) | |
| 84 | + if blk: | |
| 85 | + out[n] = int(blk["timestamp"], 16) | |
| 86 | + except (RpcError, RuntimeError): | |
| 87 | + pass | |
| 88 | + return out | |
| 89 | + | |
| 90 | + # -- pipeline ------------------------------------------------------ | |
| 91 | + | |
| 92 | + def process_range(self, frm, to): | |
| 93 | + logs = self.fetch_logs(frm, to) | |
| 94 | + rows = [] | |
| 95 | + if logs: | |
| 96 | + ts = self.block_timestamps({int(l["blockNumber"], 16) for l in logs}) | |
| 97 | + for l in logs: | |
| 98 | + if l.get("removed"): | |
| 99 | + continue | |
| 100 | + meta = self.tokens.get(l["address"].lower()) | |
| 101 | + if meta is None or len(l.get("topics", [])) != 3: | |
| 102 | + continue # not one of ours / non-standard Transfer | |
| 103 | + rows.append( | |
| 104 | + decode_transfer(self.chain, l, meta, ts.get(int(l["blockNumber"], 16))) | |
| 105 | + ) | |
| 106 | + db.insert_transfers(self.conn, rows) | |
| 107 | + return len(rows) | |
| 108 | + | |
| 109 | + def try_range(self, frm, to, stop): | |
| 110 | + """One adaptive sweep starting at `frm`. Returns (end, n): the last | |
| 111 | + block actually covered (may be < `to` after halving) and row count.""" | |
| 112 | + while not stop.is_set(): | |
| 113 | + end = min(frm + self.range - 1, to) | |
| 114 | + try: | |
| 115 | + n = self.process_range(frm, end) | |
| 116 | + except RpcError as e: | |
| 117 | + if self.range > 2: | |
| 118 | + self.range = max(2, self.range // 2) | |
| 119 | + self._clean_sweeps = 0 | |
| 120 | + log.info("%s: getLogs refused (%s) — range now %d", | |
| 121 | + self.chain, e.message[:80], self.range) | |
| 122 | + continue | |
| 123 | + # refused even at minimum range: this provider just won't | |
| 124 | + # serve getLogs — bench it and let selection move on | |
| 125 | + log.warning("%s: getLogs refused at min range (%s) — rotating " | |
| 126 | + "away from %s", self.chain, e.message[:80], | |
| 127 | + self.pool.current_url) | |
| 128 | + self.pool.rotate() | |
| 129 | + stop.wait(5) | |
| 130 | + continue | |
| 131 | + self._clean_sweeps += 1 | |
| 132 | + if self._clean_sweeps >= 10 and self.range < self.max_range: | |
| 133 | + self.range = min(self.max_range, self.range * 2) | |
| 134 | + self._clean_sweeps = 0 | |
| 135 | + return end, n | |
| 136 | + return frm - 1, 0 | |
| 137 | + | |
| 138 | + # -- reorg checks ---------------------------------------------------- | |
| 139 | + | |
| 140 | + def process_native(self, frm, to): | |
| 141 | + """Whale-only native-coin transfers: full blocks are expensive, so we | |
| 142 | + only capture value >= min (config) and only while head-tailing over a | |
| 143 | + bounded range — never during backfill. A skipped stretch is logged.""" | |
| 144 | + cap = 60 | |
| 145 | + if to - frm + 1 > cap: | |
| 146 | + log.info("%s: native capture skipped for %d..%d (catch-up burst)", | |
| 147 | + self.chain, frm, to - cap) | |
| 148 | + frm = to - cap + 1 | |
| 149 | + min_wei = int(self.native["min"] * 10 ** self.native["decimals"]) | |
| 150 | + rows = [] | |
| 151 | + nums = list(range(frm, to + 1)) | |
| 152 | + for i in range(0, len(nums), 10): | |
| 153 | + chunk = nums[i : i + 10] | |
| 154 | + blocks = self.pool.batch( | |
| 155 | + [("eth_getBlockByNumber", [hex(n), True]) for n in chunk] | |
| 156 | + ) | |
| 157 | + for blk in blocks: | |
| 158 | + if not blk: | |
| 159 | + continue | |
| 160 | + ts = int(blk["timestamp"], 16) | |
| 161 | + for ti, t in enumerate(blk.get("transactions") or []): | |
| 162 | + if not isinstance(t, dict): | |
| 163 | + continue | |
| 164 | + val = int(t.get("value", "0x0"), 16) | |
| 165 | + if val < min_wei: | |
| 166 | + continue | |
| 167 | + rows.append({ | |
| 168 | + "chain": self.chain, | |
| 169 | + "block": int(blk["number"], 16), | |
| 170 | + "block_hash": blk["hash"].lower(), | |
| 171 | + "tx_hash": t["hash"].lower(), | |
| 172 | + "log_index": 100000 + ti, # never collides with log indexes | |
| 173 | + "timestamp": ts, | |
| 174 | + "token": "native", | |
| 175 | + "symbol": self.native["symbol"], | |
| 176 | + "from": (t.get("from") or "").lower() or None, | |
| 177 | + "to": (t.get("to") or "").lower() or None, | |
| 178 | + "amount": str(val), | |
| 179 | + "decimals": self.native["decimals"], | |
| 180 | + }) | |
| 181 | + db.insert_transfers(self.conn, rows) | |
| 182 | + if rows: | |
| 183 | + log.info("%s: %d native %s whale transfers in ..%d", | |
| 184 | + self.chain, len(rows), self.native["symbol"], to) | |
| 185 | + | |
| 186 | + def reorged(self, cursor_block, cursor_hash): | |
| 187 | + """True if the block we last indexed is no longer canonical.""" | |
| 188 | + if not cursor_hash: | |
| 189 | + return False | |
| 190 | + blk = self.block_header(cursor_block) | |
| 191 | + return blk is not None and blk["hash"].lower() != cursor_hash.lower() | |
| 192 | + | |
| 193 | + def links_to_cursor(self, cursor_block, cursor_hash): | |
| 194 | + """Parent-hash link: next block's parentHash must be our cursor hash.""" | |
| 195 | + if not cursor_hash: | |
| 196 | + return True | |
| 197 | + nxt = self.block_header(cursor_block + 1) | |
| 198 | + return nxt is None or nxt["parentHash"].lower() == cursor_hash.lower() | |
| 199 | + | |
| 200 | + # -- main loop ----------------------------------------------------- | |
| 201 | + | |
| 202 | + def run(self, stop): | |
| 203 | + self.conn = db.connect(self.db_path) | |
| 204 | + cur = db.get_cursor(self.conn, self.chain) | |
| 205 | + confirmations = self.cfg.get("confirmations", 6) | |
| 206 | + while not stop.is_set(): | |
| 207 | + try: | |
| 208 | + head = self.head() | |
| 209 | + safe = head - confirmations | |
| 210 | + if cur is None: | |
| 211 | + start = max(0, safe - self.cfg.get("start_offset", 300)) | |
| 212 | + cur = (start, None, start) | |
| 213 | + db.set_cursor(self.conn, self.chain, start, None) | |
| 214 | + db.set_backfill(self.conn, self.chain, start) | |
| 215 | + log.info("%s: fresh start at block %d (head %d)", self.chain, start, head) | |
| 216 | + cursor_block, cursor_hash, backfill_block = cur | |
| 217 | + if backfill_block is None: # DB predates backfill support | |
| 218 | + backfill_block = cursor_block | |
| 219 | + db.set_backfill(self.conn, self.chain, backfill_block) | |
| 220 | + | |
| 221 | + if self.reorged(cursor_block, cursor_hash): | |
| 222 | + to_block = cursor_block - 2 * confirmations | |
| 223 | + log.warning("%s: reorg at %d — rolling back to %d", | |
| 224 | + self.chain, cursor_block, to_block) | |
| 225 | + db.rollback(self.conn, self.chain, to_block) | |
| 226 | + cursor_block, cursor_hash = to_block, None | |
| 227 | + | |
| 228 | + # ---- head tailing (always first priority) ---- | |
| 229 | + while cursor_block < safe and not stop.is_set(): | |
| 230 | + if not self.links_to_cursor(cursor_block, cursor_hash): | |
| 231 | + to_block = cursor_block - 2 * confirmations | |
| 232 | + log.warning("%s: parent-hash mismatch after %d — rolling back to %d", | |
| 233 | + self.chain, cursor_block, to_block) | |
| 234 | + db.rollback(self.conn, self.chain, to_block) | |
| 235 | + cursor_block, cursor_hash = to_block, None | |
| 236 | + continue | |
| 237 | + end, n = self.try_range(cursor_block + 1, safe, stop) | |
| 238 | + if end < cursor_block + 1: | |
| 239 | + break # stopped mid-sweep | |
| 240 | + if self.native: | |
| 241 | + try: | |
| 242 | + self.process_native(cursor_block + 1, end) | |
| 243 | + except (RpcError, RuntimeError) as e: | |
| 244 | + log.warning("%s: native capture failed (%s) — " | |
| 245 | + "continuing", self.chain, e) | |
| 246 | + end_blk = self.block_header(end) | |
| 247 | + cursor_block = end | |
| 248 | + cursor_hash = end_blk["hash"].lower() if end_blk else None | |
| 249 | + db.set_cursor(self.conn, self.chain, cursor_block, cursor_hash, head=head) | |
| 250 | + if n: | |
| 251 | + log.info("%s: %d transfers in blocks ..%d (lag %d)", | |
| 252 | + self.chain, n, end, head - end) | |
| 253 | + | |
| 254 | + # ---- backfill: one slice per cycle, backwards ---- | |
| 255 | + if self.backfill_blocks and not stop.is_set(): | |
| 256 | + target = max(0, safe - self.backfill_blocks) | |
| 257 | + if backfill_block > target: | |
| 258 | + frm = max(target, backfill_block - self.range) | |
| 259 | + covered, total = frm - 1, 0 | |
| 260 | + while covered < backfill_block - 1 and not stop.is_set(): | |
| 261 | + end, n = self.try_range(covered + 1, backfill_block - 1, stop) | |
| 262 | + if end <= covered: | |
| 263 | + break | |
| 264 | + covered, total = end, total + n | |
| 265 | + if covered >= backfill_block - 1: # slice fully covered | |
| 266 | + db.set_backfill(self.conn, self.chain, frm) | |
| 267 | + backfill_block = frm | |
| 268 | + if total: | |
| 269 | + log.info("%s: backfill %d transfers, floor now %d (target %d)", | |
| 270 | + self.chain, total, frm, target) | |
| 271 | + | |
| 272 | + cur = (cursor_block, cursor_hash, backfill_block) | |
| 273 | + db.save_rpc_health(self.conn, self.chain, self.pool.stats(), int(time.time())) | |
| 274 | + stop.wait(max(float(self.cfg.get("block_time", 12)), 2.0)) | |
| 275 | + except Exception as e: | |
| 276 | + log.error("%s: %s — retrying in 10s", self.chain, e) | |
| 277 | + stop.wait(10) | |
added
indexer/rpc.py
+292 −0
@@ -0,0 +1,292 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""RPC/REST client layer for free public endpoints — raw HTTP, no SDKs. | |
| 4 | + | |
| 5 | +Free endpoints differ wildly in limits and reliability, so every endpoint | |
| 6 | +carries: | |
| 7 | + - a client-side TOKEN BUCKET (don't hit the server's limit in the first | |
| 8 | + place; per-endpoint rps/burst configurable in chains.yaml) | |
| 9 | + - a HEALTH SCORE (EMA of success rate) + latency EMA | |
| 10 | + - an exponential COOLDOWN after consecutive failures (2s → 4s → … 300s) | |
| 11 | + | |
| 12 | +Pools pick the healthiest non-cooling endpoint per request and fail over | |
| 13 | +on transport errors, HTTP 429, and rate-limit-shaped JSON-RPC errors | |
| 14 | +(some providers deliver rate limits as error objects, e.g. -32005 | |
| 15 | +"rate limited"). Semantic errors that mean "your query is too big" | |
| 16 | +(getLogs range caps) are raised to the caller, whose adaptive range | |
| 17 | +halving is the right response — not failover. | |
| 18 | + | |
| 19 | +Endpoint spec in config: either "https://url" or {url, rps, burst}. | |
| 20 | +""" | |
| 21 | + | |
| 22 | +import threading | |
| 23 | +import time | |
| 24 | + | |
| 25 | +import requests | |
| 26 | + | |
| 27 | +DEFAULT_RPS = 5.0 | |
| 28 | +DEFAULT_BURST = 10 | |
| 29 | + | |
| 30 | + | |
| 31 | +class RpcError(Exception): | |
| 32 | + """JSON-RPC semantic error the CALLER must handle (e.g. range too big).""" | |
| 33 | + | |
| 34 | + def __init__(self, code, message): | |
| 35 | + super().__init__(f"RPC error {code}: {message}") | |
| 36 | + self.code = code | |
| 37 | + self.message = message or "" | |
| 38 | + | |
| 39 | + | |
| 40 | +class RestError(Exception): | |
| 41 | + """HTTP 4xx from a REST endpoint — a caller problem, not an outage.""" | |
| 42 | + | |
| 43 | + def __init__(self, status, body): | |
| 44 | + super().__init__(f"HTTP {status}: {str(body)[:200]}") | |
| 45 | + self.status = status | |
| 46 | + self.body = body | |
| 47 | + | |
| 48 | + | |
| 49 | +class AllEndpointsDown(RuntimeError): | |
| 50 | + pass | |
| 51 | + | |
| 52 | + | |
| 53 | +# "Query too big" wording (getLogs caps) — must reach the caller unhandled | |
| 54 | +# so its adaptive range halving kicks in. | |
| 55 | +_RANGE_HINTS = ("result", "range", "block range", "response size", "10000", "log") | |
| 56 | +# Rate-limit wording delivered as a JSON-RPC error — handled by failover. | |
| 57 | +_RATE_HINTS = ( | |
| 58 | + "rate limit", "rate-limit", "rate limited", "too many request", | |
| 59 | + "quota", "capacity", "compute unit", "upgrade", "current plan", | |
| 60 | +) | |
| 61 | + | |
| 62 | + | |
| 63 | +def _is_rate_limit(message): | |
| 64 | + m = (message or "").lower() | |
| 65 | + if any(h in m for h in _RANGE_HINTS): | |
| 66 | + return False | |
| 67 | + return any(h in m for h in _RATE_HINTS) | |
| 68 | + | |
| 69 | + | |
| 70 | +class TokenBucket: | |
| 71 | + def __init__(self, rate, burst): | |
| 72 | + self.rate = float(rate) | |
| 73 | + self.burst = float(burst) | |
| 74 | + self.tokens = self.burst | |
| 75 | + self.t = time.monotonic() | |
| 76 | + self.lock = threading.Lock() | |
| 77 | + | |
| 78 | + def acquire(self): | |
| 79 | + """Take one token, sleeping outside the lock if we're in debt.""" | |
| 80 | + with self.lock: | |
| 81 | + now = time.monotonic() | |
| 82 | + self.tokens = min(self.burst, self.tokens + (now - self.t) * self.rate) | |
| 83 | + self.t = now | |
| 84 | + self.tokens -= 1 | |
| 85 | + wait = 0.0 if self.tokens >= 0 else -self.tokens / self.rate | |
| 86 | + if wait > 0: | |
| 87 | + time.sleep(wait) | |
| 88 | + | |
| 89 | + | |
| 90 | +class Endpoint: | |
| 91 | + def __init__(self, spec): | |
| 92 | + if isinstance(spec, str): | |
| 93 | + spec = {"url": spec} | |
| 94 | + self.url = spec["url"] | |
| 95 | + self.bucket = TokenBucket(spec.get("rps", DEFAULT_RPS), spec.get("burst", DEFAULT_BURST)) | |
| 96 | + self.score = 1.0 | |
| 97 | + self.latency_ms = 0.0 | |
| 98 | + self.ok = 0 | |
| 99 | + self.fail = 0 | |
| 100 | + self.consec_fail = 0 | |
| 101 | + self.cooldown_until = 0.0 | |
| 102 | + | |
| 103 | + def available(self): | |
| 104 | + return time.monotonic() >= self.cooldown_until | |
| 105 | + | |
| 106 | + def record(self, success, latency_ms=None, cooldown=None): | |
| 107 | + alpha = 0.15 | |
| 108 | + self.score = (1 - alpha) * self.score + alpha * (1.0 if success else 0.0) | |
| 109 | + if success: | |
| 110 | + self.ok += 1 | |
| 111 | + self.consec_fail = 0 | |
| 112 | + if latency_ms is not None: | |
| 113 | + self.latency_ms = ( | |
| 114 | + 0.8 * self.latency_ms + 0.2 * latency_ms if self.latency_ms else latency_ms | |
| 115 | + ) | |
| 116 | + else: | |
| 117 | + self.fail += 1 | |
| 118 | + self.consec_fail += 1 | |
| 119 | + self.cooldown_until = time.monotonic() + ( | |
| 120 | + cooldown if cooldown is not None else min(300, 2 ** min(self.consec_fail, 8)) | |
| 121 | + ) | |
| 122 | + | |
| 123 | + | |
| 124 | +class _Pool: | |
| 125 | + def __init__(self, specs, timeout=25, max_cycles=4): | |
| 126 | + if not specs: | |
| 127 | + raise ValueError("pool needs at least one endpoint") | |
| 128 | + self.endpoints = [Endpoint(s) for s in specs] | |
| 129 | + self.timeout = timeout | |
| 130 | + self.max_cycles = max_cycles | |
| 131 | + self.session = requests.Session() | |
| 132 | + self._lock = threading.Lock() | |
| 133 | + self._current = self.endpoints[0] | |
| 134 | + | |
| 135 | + @property | |
| 136 | + def current_url(self): | |
| 137 | + return self._current.url | |
| 138 | + | |
| 139 | + @property | |
| 140 | + def urls(self): | |
| 141 | + return [e.url for e in self.endpoints] | |
| 142 | + | |
| 143 | + def _attempts(self): | |
| 144 | + return self.max_cycles * len(self.endpoints) | |
| 145 | + | |
| 146 | + def _pick(self): | |
| 147 | + """Healthiest available endpoint; if all are cooling down, wait for | |
| 148 | + the one that recovers soonest (capped so we notice config problems).""" | |
| 149 | + with self._lock: | |
| 150 | + avail = [e for e in self.endpoints if e.available()] | |
| 151 | + if avail: | |
| 152 | + ep, wait = max(avail, key=lambda e: (e.score, -e.latency_ms)), 0.0 | |
| 153 | + else: | |
| 154 | + ep = min(self.endpoints, key=lambda e: e.cooldown_until) | |
| 155 | + wait = max(0.0, ep.cooldown_until - time.monotonic()) | |
| 156 | + self._current = ep | |
| 157 | + if wait: | |
| 158 | + time.sleep(min(wait, 30)) | |
| 159 | + return ep | |
| 160 | + | |
| 161 | + def rotate(self): | |
| 162 | + """Nudge selection off the current endpoint (used by callers when a | |
| 163 | + node keeps refusing a semantically-valid request).""" | |
| 164 | + self._current.record(False, cooldown=15) | |
| 165 | + | |
| 166 | + def stats(self): | |
| 167 | + now = time.monotonic() | |
| 168 | + return [ | |
| 169 | + { | |
| 170 | + "url": e.url, | |
| 171 | + "score": round(e.score, 3), | |
| 172 | + "ok": e.ok, | |
| 173 | + "fail": e.fail, | |
| 174 | + "latency_ms": round(e.latency_ms, 1), | |
| 175 | + "cooldown_s": round(max(0.0, e.cooldown_until - now), 1), | |
| 176 | + } | |
| 177 | + for e in self.endpoints | |
| 178 | + ] | |
| 179 | + | |
| 180 | + | |
| 181 | +class RpcPool(_Pool): | |
| 182 | + """JSON-RPC 2.0 over HTTP (EVM chains, Solana, Starknet, Sui classic…).""" | |
| 183 | + | |
| 184 | + def _request(self, payload): | |
| 185 | + last = None | |
| 186 | + for _ in range(self._attempts()): | |
| 187 | + ep = self._pick() | |
| 188 | + ep.bucket.acquire() | |
| 189 | + t0 = time.monotonic() | |
| 190 | + try: | |
| 191 | + r = self.session.post( | |
| 192 | + ep.url, json=payload, timeout=self.timeout, | |
| 193 | + headers={"Content-Type": "application/json"}, | |
| 194 | + ) | |
| 195 | + if r.status_code == 429: | |
| 196 | + ep.record(False, cooldown=30) | |
| 197 | + last = f"429 from {ep.url}" | |
| 198 | + continue | |
| 199 | + r.raise_for_status() | |
| 200 | + data = r.json() | |
| 201 | + except (requests.RequestException, ValueError) as e: | |
| 202 | + ep.record(False) | |
| 203 | + last = e | |
| 204 | + continue | |
| 205 | + if isinstance(data, dict) and data.get("error"): | |
| 206 | + err = data["error"] or {} | |
| 207 | + if _is_rate_limit(err.get("message")): | |
| 208 | + ep.record(False, cooldown=20) | |
| 209 | + last = f"{err.get('message')} ({ep.url})" | |
| 210 | + continue | |
| 211 | + ep.record(True, (time.monotonic() - t0) * 1000) | |
| 212 | + return data | |
| 213 | + raise AllEndpointsDown(f"all endpoints failing: {last}") | |
| 214 | + | |
| 215 | + def call(self, method, params=None): | |
| 216 | + res = self._request( | |
| 217 | + {"jsonrpc": "2.0", "id": 1, "method": method, "params": params or []} | |
| 218 | + ) | |
| 219 | + if not isinstance(res, dict): | |
| 220 | + raise RpcError(None, f"unexpected response shape: {type(res).__name__}") | |
| 221 | + if "error" in res and res["error"]: | |
| 222 | + err = res["error"] | |
| 223 | + raise RpcError(err.get("code"), err.get("message")) | |
| 224 | + return res.get("result") | |
| 225 | + | |
| 226 | + def batch(self, calls): | |
| 227 | + """calls: list of (method, params) → results in order (None on | |
| 228 | + individual failure). Sequential fallback for batch-rejecting nodes.""" | |
| 229 | + payload = [ | |
| 230 | + {"jsonrpc": "2.0", "id": i, "method": m, "params": p or []} | |
| 231 | + for i, (m, p) in enumerate(calls) | |
| 232 | + ] | |
| 233 | + try: | |
| 234 | + res = self._request(payload) | |
| 235 | + except AllEndpointsDown: | |
| 236 | + res = None | |
| 237 | + if not isinstance(res, list): | |
| 238 | + out = [] | |
| 239 | + for method, params in calls: | |
| 240 | + try: | |
| 241 | + out.append(self.call(method, params)) | |
| 242 | + except (RpcError, AllEndpointsDown): | |
| 243 | + out.append(None) | |
| 244 | + return out | |
| 245 | + by_id = {r.get("id"): r for r in res if isinstance(r, dict)} | |
| 246 | + return [ | |
| 247 | + by_id.get(i, {}).get("result") | |
| 248 | + if not by_id.get(i, {}).get("error") else None | |
| 249 | + for i in range(len(calls)) | |
| 250 | + ] | |
| 251 | + | |
| 252 | + | |
| 253 | +class RestPool(_Pool): | |
| 254 | + """REST/HTTP APIs (Tron wallet API, Horizon, LCD, mirror nodes, algod…). | |
| 255 | + | |
| 256 | + Fails over on 429/5xx/transport errors; 4xx raises RestError to the | |
| 257 | + caller (retrying a bad request on another node won't fix it).""" | |
| 258 | + | |
| 259 | + def _do(self, method, path, params=None, json_body=None): | |
| 260 | + last = None | |
| 261 | + for _ in range(self._attempts()): | |
| 262 | + ep = self._pick() | |
| 263 | + ep.bucket.acquire() | |
| 264 | + url = ep.url.rstrip("/") + "/" + path.lstrip("/") | |
| 265 | + t0 = time.monotonic() | |
| 266 | + try: | |
| 267 | + r = self.session.request( | |
| 268 | + method, url, params=params, json=json_body, timeout=self.timeout | |
| 269 | + ) | |
| 270 | + except requests.RequestException as e: | |
| 271 | + ep.record(False) | |
| 272 | + last = e | |
| 273 | + continue | |
| 274 | + if r.status_code == 429 or r.status_code >= 500: | |
| 275 | + ep.record(False, cooldown=30 if r.status_code == 429 else None) | |
| 276 | + last = f"HTTP {r.status_code} from {url}" | |
| 277 | + continue | |
| 278 | + ep.record(True, (time.monotonic() - t0) * 1000) | |
| 279 | + try: | |
| 280 | + body = r.json() | |
| 281 | + except ValueError: | |
| 282 | + body = r.text | |
| 283 | + if r.status_code >= 400: | |
| 284 | + raise RestError(r.status_code, body) | |
| 285 | + return body | |
| 286 | + raise AllEndpointsDown(f"all endpoints failing: {last}") | |
| 287 | + | |
| 288 | + def get(self, path, params=None): | |
| 289 | + return self._do("GET", path, params=params) | |
| 290 | + | |
| 291 | + def post(self, path, json_body=None): | |
| 292 | + return self._do("POST", path, json_body=json_body) | |
added
migrations/001_init.sql
+70 −0
@@ -0,0 +1,70 @@ | ||
| 1 | +-- Author: Simon-Pierre Boucher | |
| 2 | +-- Mail: contact@spboucher.ai | |
| 3 | +-- Canonical schema (auto-derived from indexer/db.py SCHEMA). | |
| 4 | +-- Runtime auto-applies this on connect; file kept for manual PostgreSQL setup. | |
| 5 | + | |
| 6 | +CREATE TABLE IF NOT EXISTS chains ( | |
| 7 | + chain TEXT PRIMARY KEY, | |
| 8 | + family TEXT, | |
| 9 | + chain_id BIGINT | |
| 10 | +); | |
| 11 | + | |
| 12 | +CREATE TABLE IF NOT EXISTS transfers ( | |
| 13 | + chain TEXT NOT NULL, | |
| 14 | + block BIGINT NOT NULL, | |
| 15 | + block_hash TEXT, | |
| 16 | + tx_hash TEXT NOT NULL, | |
| 17 | + log_index INTEGER NOT NULL, | |
| 18 | + timestamp BIGINT, | |
| 19 | + token TEXT NOT NULL, | |
| 20 | + symbol TEXT, | |
| 21 | + "from" TEXT, | |
| 22 | + "to" TEXT, | |
| 23 | + amount TEXT, | |
| 24 | + decimals INTEGER, | |
| 25 | + PRIMARY KEY (chain, tx_hash, log_index) | |
| 26 | +); | |
| 27 | +CREATE INDEX IF NOT EXISTS idx_transfers_token_block ON transfers (chain, token, block); | |
| 28 | +CREATE INDEX IF NOT EXISTS idx_transfers_ts ON transfers (symbol, timestamp); | |
| 29 | +CREATE INDEX IF NOT EXISTS idx_transfers_from ON transfers ("from"); | |
| 30 | +CREATE INDEX IF NOT EXISTS idx_transfers_to ON transfers ("to"); | |
| 31 | + | |
| 32 | +CREATE TABLE IF NOT EXISTS cursors ( | |
| 33 | + chain TEXT PRIMARY KEY, | |
| 34 | + last_block BIGINT NOT NULL, | |
| 35 | + last_hash TEXT, | |
| 36 | + backfill_block BIGINT, | |
| 37 | + head_block BIGINT | |
| 38 | +); | |
| 39 | + | |
| 40 | +CREATE TABLE IF NOT EXISTS rpc_health ( | |
| 41 | + chain TEXT NOT NULL, | |
| 42 | + url TEXT NOT NULL, | |
| 43 | + score REAL, | |
| 44 | + ok INTEGER, | |
| 45 | + fail INTEGER, | |
| 46 | + latency_ms REAL, | |
| 47 | + cooldown_s REAL, | |
| 48 | + updated BIGINT, | |
| 49 | + PRIMARY KEY (chain, url) | |
| 50 | +); | |
| 51 | + | |
| 52 | +CREATE TABLE IF NOT EXISTS tokens ( | |
| 53 | + chain TEXT NOT NULL, | |
| 54 | + address TEXT NOT NULL, | |
| 55 | + symbol TEXT, | |
| 56 | + decimals INTEGER, | |
| 57 | + native INTEGER, | |
| 58 | + PRIMARY KEY (chain, address) | |
| 59 | +); | |
| 60 | + | |
| 61 | +CREATE TABLE IF NOT EXISTS supply_snapshots ( | |
| 62 | + chain TEXT NOT NULL, | |
| 63 | + token TEXT NOT NULL, | |
| 64 | + symbol TEXT, | |
| 65 | + supply TEXT, | |
| 66 | + decimals INTEGER, | |
| 67 | + timestamp BIGINT NOT NULL, | |
| 68 | + PRIMARY KEY (chain, token, timestamp) | |
| 69 | +); | |
| 70 | +CREATE INDEX IF NOT EXISTS idx_supply_ts ON supply_snapshots (symbol, timestamp); | |
added
requirements.txt
+8 −0
@@ -0,0 +1,8 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +requests>=2.31 | |
| 4 | +PyYAML>=6.0 | |
| 5 | +fastapi>=0.110 | |
| 6 | +uvicorn>=0.29 | |
| 7 | +psycopg2-binary>=2.9 | |
| 8 | +websockets>=12 | |
added
run_indexer.py
+100 −0
@@ -0,0 +1,100 @@ | ||
| 1 | +#!/usr/bin/env python3 | |
| 2 | +# Author: Simon-Pierre Boucher | |
| 3 | +# Mail: contact@spboucher.ai | |
| 4 | +"""Run the indexer: one worker thread per chain + price/supply workers, | |
| 5 | +all under a watchdog that restarts any thread that dies. | |
| 6 | + | |
| 7 | + python3 run_indexer.py # all configured chains | |
| 8 | + python3 run_indexer.py --chains ethereum base # a subset | |
| 9 | +""" | |
| 10 | + | |
| 11 | +import argparse | |
| 12 | +import logging | |
| 13 | +import threading | |
| 14 | + | |
| 15 | +from indexer import config, db | |
| 16 | +from indexer.adapters.bitcoin import BitcoinIndexer | |
| 17 | +from indexer.adapters.solana import SolanaIndexer | |
| 18 | +from indexer.adapters.tron import TronIndexer | |
| 19 | +from indexer.enrich import PriceWorker, StatsWorker, SupplyWorker | |
| 20 | +from indexer.ingest import ChainIndexer | |
| 21 | + | |
| 22 | +ADAPTERS = { | |
| 23 | + "evm": ChainIndexer, | |
| 24 | + "tron": TronIndexer, | |
| 25 | + "solana": SolanaIndexer, | |
| 26 | + "bitcoin": BitcoinIndexer, | |
| 27 | +} | |
| 28 | + | |
| 29 | + | |
| 30 | +def main(): | |
| 31 | + ap = argparse.ArgumentParser() | |
| 32 | + ap.add_argument("--chains", nargs="*", help="subset of chains (default: all)") | |
| 33 | + ap.add_argument("--db", default=config.db_path()) | |
| 34 | + args = ap.parse_args() | |
| 35 | + | |
| 36 | + logging.basicConfig( | |
| 37 | + level=logging.INFO, | |
| 38 | + format="%(asctime)s %(levelname)-7s %(message)s", | |
| 39 | + datefmt="%H:%M:%S", | |
| 40 | + ) | |
| 41 | + | |
| 42 | + chains = config.load_chains() | |
| 43 | + tokens = config.load_tokens() | |
| 44 | + selected = config.selected_chains(chains, tokens, args.chains, set(ADAPTERS)) | |
| 45 | + | |
| 46 | + conn = db.connect(args.db) # main-thread conn: schema + token metadata only | |
| 47 | + for chain in selected: | |
| 48 | + db.upsert_chain(conn, chain, chains[chain].get("family", "evm"), | |
| 49 | + chains[chain].get("chain_id")) | |
| 50 | + db.upsert_tokens(conn, chain, tokens[chain]) | |
| 51 | + conn.close() | |
| 52 | + | |
| 53 | + stop = threading.Event() | |
| 54 | + | |
| 55 | + # every worker is a (name, factory) pair so the watchdog can rebuild it — | |
| 56 | + # workers keep their own retry loops; this is the belt AND the suspenders | |
| 57 | + def chain_factory(c): | |
| 58 | + return lambda: ADAPTERS[chains[c].get("family", "evm")]( | |
| 59 | + c, chains[c], tokens[c], args.db).run(stop) | |
| 60 | + | |
| 61 | + factories = {c: chain_factory(c) for c in selected} | |
| 62 | + factories["prices"] = lambda: PriceWorker( | |
| 63 | + {c: tokens[c] for c in selected}, args.db).run(stop) | |
| 64 | + factories["supply"] = lambda: SupplyWorker( | |
| 65 | + chains, {c: tokens[c] for c in selected}, args.db).run(stop) | |
| 66 | + factories["stats"] = lambda: StatsWorker(args.db).run(stop) | |
| 67 | + | |
| 68 | + def spawn(name): | |
| 69 | + t = threading.Thread(target=guarded(name), name=name, daemon=True) | |
| 70 | + t.start() | |
| 71 | + return t | |
| 72 | + | |
| 73 | + def guarded(name): | |
| 74 | + def inner(): | |
| 75 | + try: | |
| 76 | + factories[name]() | |
| 77 | + except Exception as e: # last-resort: workers shouldn't get here | |
| 78 | + logging.error("worker %s died: %s", name, e) | |
| 79 | + return inner | |
| 80 | + | |
| 81 | + threads = {name: spawn(name) for name in factories} | |
| 82 | + logging.info("started %d chain workers + prices + supply (watchdog on)", | |
| 83 | + len(selected)) | |
| 84 | + | |
| 85 | + try: | |
| 86 | + while not stop.is_set(): | |
| 87 | + stop.wait(30) | |
| 88 | + for name, t in list(threads.items()): | |
| 89 | + if not t.is_alive() and not stop.is_set(): | |
| 90 | + logging.warning("watchdog: restarting dead worker %s", name) | |
| 91 | + threads[name] = spawn(name) | |
| 92 | + except KeyboardInterrupt: | |
| 93 | + logging.info("stopping...") | |
| 94 | + stop.set() | |
| 95 | + for t in threads.values(): | |
| 96 | + t.join(timeout=10) | |
| 97 | + | |
| 98 | + | |
| 99 | +if __name__ == "__main__": | |
| 100 | + main() | |
added
scripts/verify_tokens.py
+96 −0
@@ -0,0 +1,96 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +#!/usr/bin/env python3 | |
| 4 | +"""Verify every configured token address on-chain before trusting it. | |
| 5 | + | |
| 6 | +For each chain: check eth_chainId matches the config; for each token, | |
| 7 | +eth_call symbol() and decimals() on the address and compare with | |
| 8 | +tokens.yaml. A wrong or dead address returns empty data and fails loudly. | |
| 9 | + | |
| 10 | + python3 scripts/verify_tokens.py | |
| 11 | +""" | |
| 12 | + | |
| 13 | +import pathlib | |
| 14 | +import sys | |
| 15 | + | |
| 16 | +import yaml | |
| 17 | + | |
| 18 | +sys.path.insert(0, str(pathlib.Path(__file__).parent.parent)) | |
| 19 | +from indexer.rpc import RpcError, RpcPool # noqa: E402 | |
| 20 | + | |
| 21 | +ROOT = pathlib.Path(__file__).parent.parent | |
| 22 | +SEL_SYMBOL = "0x95d89b41" # keccak4("symbol()") | |
| 23 | +SEL_DECIMALS = "0x313ce567" # keccak4("decimals()") | |
| 24 | + | |
| 25 | + | |
| 26 | +def decode_string(hexdata): | |
| 27 | + """ABI string return: word0 = offset, word1 = length, then bytes. | |
| 28 | + Some old tokens (MKR-style) return bytes32 instead — handle both.""" | |
| 29 | + if hexdata in (None, "0x", ""): | |
| 30 | + return None | |
| 31 | + raw = bytes.fromhex(hexdata[2:]) | |
| 32 | + if len(raw) >= 64: | |
| 33 | + offset = int.from_bytes(raw[:32], "big") | |
| 34 | + if offset == 32 and len(raw) >= 64: | |
| 35 | + length = int.from_bytes(raw[32:64], "big") | |
| 36 | + if 0 < length <= len(raw) - 64: | |
| 37 | + return raw[64 : 64 + length].decode("utf-8", "replace") | |
| 38 | + return raw.rstrip(b"\x00").decode("utf-8", "replace") or None | |
| 39 | + | |
| 40 | + | |
| 41 | +def main(): | |
| 42 | + chains = yaml.safe_load((ROOT / "config" / "chains.yaml").read_text())["chains"] | |
| 43 | + tokens = yaml.safe_load((ROOT / "config" / "tokens.yaml").read_text())["tokens"] | |
| 44 | + | |
| 45 | + failures = 0 | |
| 46 | + for chain, toks in tokens.items(): | |
| 47 | + cfg = chains[chain] | |
| 48 | + if cfg.get("family", "evm") != "evm": | |
| 49 | + # non-EVM identifiers are verified by their adapters (Deliverable 4) | |
| 50 | + print(f"{chain:10s} skipped (family={cfg['family']} — no eth_call)") | |
| 51 | + continue | |
| 52 | + p = RpcPool(cfg["rpcs"], max_cycles=2) | |
| 53 | + try: | |
| 54 | + chain_id = int(p.call("eth_chainId"), 16) | |
| 55 | + except (RpcError, RuntimeError) as e: | |
| 56 | + print(f"{chain:10s} UNREACHABLE: {e}") | |
| 57 | + failures += 1 | |
| 58 | + continue | |
| 59 | + cid_ok = "ok" if chain_id == cfg["chain_id"] else f"MISMATCH got {chain_id}" | |
| 60 | + if chain_id != cfg["chain_id"]: | |
| 61 | + failures += 1 | |
| 62 | + print(f"{chain:10s} chain_id {cfg['chain_id']} [{cid_ok}] via {p.current_url}") | |
| 63 | + | |
| 64 | + for t in toks: | |
| 65 | + addr = t["address"] | |
| 66 | + try: | |
| 67 | + sym = decode_string( | |
| 68 | + p.call("eth_call", [{"to": addr, "data": SEL_SYMBOL}, "latest"]) | |
| 69 | + ) | |
| 70 | + dec_hex = p.call( | |
| 71 | + "eth_call", [{"to": addr, "data": SEL_DECIMALS}, "latest"] | |
| 72 | + ) | |
| 73 | + dec = int(dec_hex, 16) if dec_hex not in (None, "0x") else None | |
| 74 | + except (RpcError, RuntimeError) as e: | |
| 75 | + print(f" {t['symbol']:5s} {addr} ERROR {e}") | |
| 76 | + failures += 1 | |
| 77 | + continue | |
| 78 | + # Tether uses '₮' in some deployments (USD₮, USD₮0) — normalize | |
| 79 | + def norm(s): | |
| 80 | + return (s or "").replace("₮", "T").upper() | |
| 81 | + expected = t.get("onchain_symbol", t["symbol"]) | |
| 82 | + ok = norm(sym) == norm(expected) and dec == t["decimals"] | |
| 83 | + mark = "ok" if ok else "MISMATCH" | |
| 84 | + if not ok: | |
| 85 | + failures += 1 | |
| 86 | + print(f" {t['symbol']:5s} {addr} on-chain: symbol={sym!r} decimals={dec} [{mark}]") | |
| 87 | + | |
| 88 | + print() | |
| 89 | + if failures: | |
| 90 | + print(f"FAILED: {failures} problem(s) — do not index until fixed.") | |
| 91 | + sys.exit(1) | |
| 92 | + print("All token addresses verified on-chain.") | |
| 93 | + | |
| 94 | + | |
| 95 | +if __name__ == "__main__": | |
| 96 | + main() | |
added
tests/test_adapters.py
+142 −0
@@ -0,0 +1,142 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Unit checks for the Tron and Solana adapters (plain asserts).""" | |
| 4 | + | |
| 5 | +import pathlib | |
| 6 | +import sys | |
| 7 | + | |
| 8 | +sys.path.insert(0, str(pathlib.Path(__file__).parent.parent)) | |
| 9 | + | |
| 10 | +from indexer import db | |
| 11 | +from indexer.adapters.solana import SolanaIndexer | |
| 12 | +from indexer.adapters.tron import ( | |
| 13 | + TronIndexer, b58check_decode, hex20_to_taddr, taddr_to_hex20, | |
| 14 | +) | |
| 15 | + | |
| 16 | +# Known vector: Tron USDT contract (verified via tronscan + tether.to) | |
| 17 | +USDT_T = "TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t" | |
| 18 | +USDT_HEX = "a614f803b6fd780986a42c78ec9c7f77e6ded13c" | |
| 19 | + | |
| 20 | + | |
| 21 | +def test_base58check_roundtrip(): | |
| 22 | + assert taddr_to_hex20(USDT_T) == USDT_HEX | |
| 23 | + assert hex20_to_taddr(USDT_HEX) == USDT_T | |
| 24 | + try: | |
| 25 | + b58check_decode("TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj7t") # corrupted | |
| 26 | + raise AssertionError("bad checksum accepted") | |
| 27 | + except ValueError: | |
| 28 | + pass | |
| 29 | + | |
| 30 | + | |
| 31 | +def make_tron(): | |
| 32 | + ix = TronIndexer( | |
| 33 | + "tron", {"rpcs": ["http://unused.invalid"]}, | |
| 34 | + [{"symbol": "USDT", "id": USDT_T, "decimals": 6, "native": True}], | |
| 35 | + ":memory:", | |
| 36 | + ) | |
| 37 | + ix.conn = db.connect(":memory:") | |
| 38 | + return ix | |
| 39 | + | |
| 40 | + | |
| 41 | +def test_tron_decodes_trc20_transfer(): | |
| 42 | + ix = make_tron() | |
| 43 | + frm_hex, to_hex = "11" * 20, "22" * 20 | |
| 44 | + ix.block_infos = lambda n: [{ | |
| 45 | + "id": "deadbeef" * 8, | |
| 46 | + "blockTimeStamp": 1785975000123, | |
| 47 | + "receipt": {"result": "SUCCESS"}, | |
| 48 | + "log": [{ | |
| 49 | + "address": "41" + USDT_HEX, # some nodes include the 41 prefix | |
| 50 | + "topics": [ | |
| 51 | + "ddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef", | |
| 52 | + frm_hex.rjust(64, "0"), | |
| 53 | + to_hex.rjust(64, "0"), | |
| 54 | + ], | |
| 55 | + "data": hex(5_000_000)[2:].rjust(64, "0"), | |
| 56 | + }], | |
| 57 | + }] | |
| 58 | + n = ix.process_block(75_000_000) | |
| 59 | + assert n == 1 | |
| 60 | + row = ix.conn.execute("SELECT token, symbol, \"from\", \"to\", amount, timestamp FROM transfers").fetchone() | |
| 61 | + assert row[0] == USDT_T and row[1] == "USDT" | |
| 62 | + assert row[2] == hex20_to_taddr(frm_hex) and row[3] == hex20_to_taddr(to_hex) | |
| 63 | + assert row[4] == "5000000" and row[5] == 1785975000 | |
| 64 | + | |
| 65 | + | |
| 66 | +USDC_MINT = "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v" | |
| 67 | + | |
| 68 | + | |
| 69 | +def make_solana(): | |
| 70 | + ix = SolanaIndexer( | |
| 71 | + "solana", {"rpcs": ["http://unused.invalid"]}, | |
| 72 | + [{"symbol": "USDC", "id": USDC_MINT, "decimals": 6, "native": True}], | |
| 73 | + ":memory:", | |
| 74 | + ) | |
| 75 | + ix.conn = db.connect(":memory:") | |
| 76 | + return ix | |
| 77 | + | |
| 78 | + | |
| 79 | +def bal(idx, owner, amount): | |
| 80 | + return {"accountIndex": idx, "mint": USDC_MINT, "owner": owner, | |
| 81 | + "uiTokenAmount": {"amount": str(amount)}} | |
| 82 | + | |
| 83 | + | |
| 84 | +def test_solana_balance_diff_pairing(): | |
| 85 | + ix = make_solana() | |
| 86 | + blk = { | |
| 87 | + "blockTime": 1785975100, | |
| 88 | + "blockhash": "H" * 44, | |
| 89 | + "transactions": [{ | |
| 90 | + "meta": { | |
| 91 | + "err": None, | |
| 92 | + "preTokenBalances": [bal(1, "AliceOwner", 10_000_000), bal(2, "BobOwner", 0)], | |
| 93 | + "postTokenBalances": [bal(1, "AliceOwner", 4_000_000), bal(2, "BobOwner", 6_000_000)], | |
| 94 | + }, | |
| 95 | + "transaction": {"signatures": ["5igSig"]}, | |
| 96 | + }], | |
| 97 | + } | |
| 98 | + n = ix.process_block(360_000_000, blk) | |
| 99 | + assert n == 1 | |
| 100 | + row = ix.conn.execute('SELECT "from", "to", amount, symbol FROM transfers').fetchone() | |
| 101 | + assert tuple(row) == ("AliceOwner", "BobOwner", "6000000", "USDC") | |
| 102 | + | |
| 103 | + | |
| 104 | +def test_solana_ambiguous_senders_kept_with_null_from(): | |
| 105 | + ix = make_solana() | |
| 106 | + blk = { | |
| 107 | + "blockTime": 1785975100, | |
| 108 | + "blockhash": "H" * 44, | |
| 109 | + "transactions": [{ | |
| 110 | + "meta": { | |
| 111 | + "err": None, | |
| 112 | + "preTokenBalances": [bal(1, "A", 5), bal(2, "B", 5), bal(3, "C", 0)], | |
| 113 | + "postTokenBalances": [bal(1, "A", 0), bal(2, "B", 0), bal(3, "C", 10)], | |
| 114 | + }, | |
| 115 | + "transaction": {"signatures": ["sig2"]}, | |
| 116 | + }], | |
| 117 | + } | |
| 118 | + assert ix.process_block(360_000_001, blk) == 1 | |
| 119 | + row = ix.conn.execute('SELECT "from", "to", amount FROM transfers').fetchone() | |
| 120 | + assert tuple(row) == (None, "C", "10") # amount exact, sender ambiguous | |
| 121 | + | |
| 122 | + | |
| 123 | +def test_solana_failed_tx_skipped(): | |
| 124 | + ix = make_solana() | |
| 125 | + blk = { | |
| 126 | + "blockTime": 1, | |
| 127 | + "transactions": [{ | |
| 128 | + "meta": {"err": {"InstructionError": [0, "Custom"]}, | |
| 129 | + "preTokenBalances": [bal(1, "A", 5)], | |
| 130 | + "postTokenBalances": [bal(1, "A", 0)]}, | |
| 131 | + "transaction": {"signatures": ["sig3"]}, | |
| 132 | + }], | |
| 133 | + } | |
| 134 | + assert ix.process_block(1, blk) == 0 | |
| 135 | + | |
| 136 | + | |
| 137 | +if __name__ == "__main__": | |
| 138 | + for name, fn in sorted(globals().items()): | |
| 139 | + if name.startswith("test_"): | |
| 140 | + fn() | |
| 141 | + print(f"ok {name}") | |
| 142 | + print("all adapter tests passed") | |
added
tests/test_ingest.py
+135 −0
@@ -0,0 +1,135 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Unit checks for the EVM ingestion loop (plain asserts — run with python).""" | |
| 4 | + | |
| 5 | +import pathlib | |
| 6 | +import sys | |
| 7 | +import threading | |
| 8 | + | |
| 9 | +sys.path.insert(0, str(pathlib.Path(__file__).parent.parent)) | |
| 10 | + | |
| 11 | +from indexer import db | |
| 12 | +from indexer.decode import TRANSFER_TOPIC | |
| 13 | +from indexer.ingest import ChainIndexer | |
| 14 | +from indexer.rpc import RpcError | |
| 15 | + | |
| 16 | +USDT = "0xdac17f958d2ee523a2206206994597c13d831ec7" | |
| 17 | +CFG = {"rpcs": ["http://unused.invalid"], "max_range": 8, "confirmations": 2} | |
| 18 | +TOKENS = [{"symbol": "USDT", "address": USDT, "decimals": 6, "native": True}] | |
| 19 | + | |
| 20 | + | |
| 21 | +def fake_log(block, log_index=0, amount=1_000_000): | |
| 22 | + pad = lambda a: "0x" + a[2:].rjust(64, "0") | |
| 23 | + return { | |
| 24 | + "address": USDT, | |
| 25 | + "topics": [TRANSFER_TOPIC, pad("0x" + "11" * 20), pad("0x" + "22" * 20)], | |
| 26 | + "data": hex(amount), | |
| 27 | + "blockNumber": hex(block), | |
| 28 | + "blockHash": "0x" + f"{block:x}".rjust(64, "b"), | |
| 29 | + "transactionHash": "0x" + f"{block:x}{log_index:x}".rjust(64, "a"), | |
| 30 | + "logIndex": hex(log_index), | |
| 31 | + } | |
| 32 | + | |
| 33 | + | |
| 34 | +class FakePool: | |
| 35 | + """Serves synthetic logs; refuses getLogs ranges wider than `cap`.""" | |
| 36 | + | |
| 37 | + def __init__(self, cap=2, blocks_with_logs=()): | |
| 38 | + self.cap = cap | |
| 39 | + self.blocks = set(blocks_with_logs) | |
| 40 | + self.getlogs_calls = [] | |
| 41 | + self.rotations = 0 | |
| 42 | + | |
| 43 | + def call(self, method, params=None): | |
| 44 | + if method == "eth_getLogs": | |
| 45 | + f = params[0] | |
| 46 | + frm, to = int(f["fromBlock"], 16), int(f["toBlock"], 16) | |
| 47 | + self.getlogs_calls.append((frm, to)) | |
| 48 | + if to - frm + 1 > self.cap: | |
| 49 | + raise RpcError(-32005, "query returned more than 10000 results") | |
| 50 | + return [fake_log(b) for b in range(frm, to + 1) if b in self.blocks] | |
| 51 | + if method == "eth_getBlockByNumber": | |
| 52 | + n = int(params[0], 16) | |
| 53 | + return {"number": hex(n), "timestamp": hex(1_785_000_000 + n), | |
| 54 | + "hash": "0x" + f"{n:x}".rjust(64, "b"), | |
| 55 | + "parentHash": "0x" + f"{n - 1:x}".rjust(64, "b")} | |
| 56 | + raise AssertionError(f"unexpected method {method}") | |
| 57 | + | |
| 58 | + def batch(self, calls): | |
| 59 | + return [self.call(m, p) for m, p in calls] | |
| 60 | + | |
| 61 | + def rotate(self): | |
| 62 | + self.rotations += 1 | |
| 63 | + | |
| 64 | + @property | |
| 65 | + def current_url(self): | |
| 66 | + return "http://fake" | |
| 67 | + | |
| 68 | + def stats(self): | |
| 69 | + return [] | |
| 70 | + | |
| 71 | + | |
| 72 | +def make_indexer(pool): | |
| 73 | + ix = ChainIndexer("testchain", CFG, TOKENS, ":memory:") | |
| 74 | + ix.pool = pool | |
| 75 | + ix.conn = db.connect(":memory:") | |
| 76 | + return ix | |
| 77 | + | |
| 78 | + | |
| 79 | +def test_try_range_halves_and_stays_contiguous(): | |
| 80 | + pool = FakePool(cap=2, blocks_with_logs={101, 103, 105}) | |
| 81 | + ix = make_indexer(pool) | |
| 82 | + stop = threading.Event() | |
| 83 | + covered, rows = 100, 0 | |
| 84 | + while covered < 106: | |
| 85 | + end, n = ix.try_range(covered + 1, 106, stop) | |
| 86 | + assert end > covered, "no forward progress" | |
| 87 | + covered, rows = end, rows + n | |
| 88 | + assert covered == 106 | |
| 89 | + assert rows == 3, f"expected 3 transfers, got {rows}" | |
| 90 | + assert ix.range == 2, "range should have halved to the node's cap" | |
| 91 | + # every accepted sweep must be adjacent to the previous one (no gaps) | |
| 92 | + accepted = [c for c in pool.getlogs_calls if c[1] - c[0] + 1 <= pool.cap] | |
| 93 | + for (f1, t1), (f2, _) in zip(accepted, accepted[1:]): | |
| 94 | + assert f2 == t1 + 1, f"gap between sweeps: ..{t1} then {f2}.." | |
| 95 | + | |
| 96 | + | |
| 97 | +def test_decode_lands_in_db(): | |
| 98 | + pool = FakePool(cap=10, blocks_with_logs={42}) | |
| 99 | + ix = make_indexer(pool) | |
| 100 | + n = ix.process_range(40, 45) | |
| 101 | + assert n == 1 | |
| 102 | + row = ix.conn.execute("SELECT * FROM transfers").fetchone() | |
| 103 | + assert row is not None | |
| 104 | + (chain, block, _bh, _tx, _li, ts, token, symbol, frm, to, amount, decimals) = row | |
| 105 | + assert (chain, block, token, symbol) == ("testchain", 42, USDT, "USDT") | |
| 106 | + assert frm == "0x" + "11" * 20 and to == "0x" + "22" * 20 | |
| 107 | + assert amount == "1000000" and decimals == 6 | |
| 108 | + assert ts == 1_785_000_000 + 42 | |
| 109 | + | |
| 110 | + | |
| 111 | +def test_rollback_removes_reorged_rows(): | |
| 112 | + pool = FakePool(cap=10, blocks_with_logs={10, 11, 12}) | |
| 113 | + ix = make_indexer(pool) | |
| 114 | + ix.process_range(10, 12) | |
| 115 | + db.set_cursor(ix.conn, "testchain", 12, "0xdead") | |
| 116 | + db.rollback(ix.conn, "testchain", 10) | |
| 117 | + left = [r[0] for r in ix.conn.execute("SELECT block FROM transfers")] | |
| 118 | + assert left == [10], f"only block 10 should survive, got {left}" | |
| 119 | + assert db.get_cursor(ix.conn, "testchain")[0] == 10 | |
| 120 | + | |
| 121 | + | |
| 122 | +def test_parent_link_check(): | |
| 123 | + pool = FakePool(cap=10) | |
| 124 | + ix = make_indexer(pool) | |
| 125 | + good = "0x" + f"{50:x}".rjust(64, "b") # header(51).parentHash | |
| 126 | + assert ix.links_to_cursor(50, good) | |
| 127 | + assert not ix.links_to_cursor(50, "0x" + "f" * 64) | |
| 128 | + | |
| 129 | + | |
| 130 | +if __name__ == "__main__": | |
| 131 | + for name, fn in sorted(globals().items()): | |
| 132 | + if name.startswith("test_"): | |
| 133 | + fn() | |
| 134 | + print(f"ok {name}") | |
| 135 | + print("all ingest tests passed") | |
added
tests/test_rpc.py
+65 −0
@@ -0,0 +1,65 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Unit checks for the RPC layer (plain asserts — run with python).""" | |
| 4 | + | |
| 5 | +import pathlib | |
| 6 | +import sys | |
| 7 | +import time | |
| 8 | + | |
| 9 | +sys.path.insert(0, str(pathlib.Path(__file__).parent.parent)) | |
| 10 | + | |
| 11 | +from indexer.rpc import Endpoint, RpcPool, TokenBucket, _is_rate_limit | |
| 12 | + | |
| 13 | + | |
| 14 | +def test_token_bucket_paces(): | |
| 15 | + b = TokenBucket(rate=10, burst=2) | |
| 16 | + t0 = time.monotonic() | |
| 17 | + for _ in range(6): # burst of 2 free, then 4 paced at 10/s ≈ 0.4s | |
| 18 | + b.acquire() | |
| 19 | + elapsed = time.monotonic() - t0 | |
| 20 | + assert 0.3 <= elapsed <= 1.0, f"pacing off: {elapsed:.2f}s" | |
| 21 | + | |
| 22 | + | |
| 23 | +def test_rate_limit_classification(): | |
| 24 | + # failover-worthy (rate limits dressed as JSON-RPC errors) | |
| 25 | + assert _is_rate_limit("rate limited") | |
| 26 | + assert _is_rate_limit("Too many requests, slow down") | |
| 27 | + assert _is_rate_limit("You've reached the usage limit for your current plan") | |
| 28 | + # caller-worthy (query too big — range halving is the right response) | |
| 29 | + assert not _is_rate_limit("query returned more than 10000 results") | |
| 30 | + assert not _is_rate_limit("block range is too wide") | |
| 31 | + assert not _is_rate_limit("Log response size exceeded") | |
| 32 | + assert not _is_rate_limit("limit exceeded") # BSC getLogs cap — ambiguous, caller decides | |
| 33 | + | |
| 34 | + | |
| 35 | +def test_health_scoring_and_cooldown(): | |
| 36 | + e = Endpoint("https://example.invalid") | |
| 37 | + assert e.score == 1.0 and e.available() | |
| 38 | + e.record(False) | |
| 39 | + e.record(False) | |
| 40 | + assert e.score < 1.0 and not e.available() # cooling down | |
| 41 | + assert e.consec_fail == 2 | |
| 42 | + e.cooldown_until = 0 # simulate recovery | |
| 43 | + e.record(True, latency_ms=50) | |
| 44 | + assert e.consec_fail == 0 and e.available() | |
| 45 | + | |
| 46 | + | |
| 47 | +def test_pool_prefers_healthy_endpoint(): | |
| 48 | + pool = RpcPool(["https://a.invalid", "https://b.invalid"]) | |
| 49 | + a, b = pool.endpoints | |
| 50 | + a.record(False) # a benched + score down | |
| 51 | + assert pool._pick() is b | |
| 52 | + assert pool.current_url == "https://b.invalid" | |
| 53 | + | |
| 54 | + | |
| 55 | +def test_endpoint_spec_dict(): | |
| 56 | + e = Endpoint({"url": "https://x.invalid", "rps": 1, "burst": 3}) | |
| 57 | + assert e.bucket.rate == 1 and e.bucket.burst == 3 | |
| 58 | + | |
| 59 | + | |
| 60 | +if __name__ == "__main__": | |
| 61 | + for name, fn in sorted(globals().items()): | |
| 62 | + if name.startswith("test_"): | |
| 63 | + fn() | |
| 64 | + print(f"ok {name}") | |
| 65 | + print("all rpc tests passed") | |
added
tests/test_v3.py
+102 −0
@@ -0,0 +1,102 @@ | ||
| 1 | +# Author: Simon-Pierre Boucher | |
| 2 | +# Mail: contact@spboucher.ai | |
| 3 | +"""Unit checks for V3: bitcoin output parsing, EVM native whales, price rows.""" | |
| 4 | + | |
| 5 | +import pathlib | |
| 6 | +import sys | |
| 7 | + | |
| 8 | +sys.path.insert(0, str(pathlib.Path(__file__).parent.parent)) | |
| 9 | + | |
| 10 | +from indexer import db | |
| 11 | +from indexer.adapters.bitcoin import BitcoinIndexer | |
| 12 | +from indexer.ingest import ChainIndexer | |
| 13 | + | |
| 14 | +BTC_CFG = {"rpcs": ["http://unused.invalid"], "confirmations": 1, | |
| 15 | + "native": {"symbol": "BTC", "min": 5, "decimals": 8}} | |
| 16 | + | |
| 17 | + | |
| 18 | +def make_btc(): | |
| 19 | + ix = BitcoinIndexer("bitcoin", BTC_CFG, [], ":memory:") | |
| 20 | + ix.conn = db.connect(":memory:") | |
| 21 | + return ix | |
| 22 | + | |
| 23 | + | |
| 24 | +def test_bitcoin_whale_outputs(): | |
| 25 | + ix = make_btc() | |
| 26 | + ix.pool.get = lambda path: ( | |
| 27 | + {"timestamp": 1786000000} if path == "/block/HASH" else None) | |
| 28 | + ix.block_txs = lambda h: [ | |
| 29 | + { # 12 BTC to bob from a single-input-address tx → recorded | |
| 30 | + "txid": "aa" * 32, | |
| 31 | + "vin": [{"prevout": {"scriptpubkey_address": "bc1alice"}}], | |
| 32 | + "vout": [ | |
| 33 | + {"value": 12_0000_0000, "scriptpubkey_address": "bc1bob"}, | |
| 34 | + {"value": 3_0000_0000, "scriptpubkey_address": "bc1alice"}, # change < min anyway | |
| 35 | + ], | |
| 36 | + }, | |
| 37 | + { # multi-input (ambiguous sender) 7 BTC → recorded with from=None | |
| 38 | + "txid": "bb" * 32, | |
| 39 | + "vin": [{"prevout": {"scriptpubkey_address": "bc1x"}}, | |
| 40 | + {"prevout": {"scriptpubkey_address": "bc1y"}}], | |
| 41 | + "vout": [{"value": 7_0000_0000, "scriptpubkey_address": "bc1z"}], | |
| 42 | + }, | |
| 43 | + { # 20 BTC change back to the same sender → filtered out | |
| 44 | + "txid": "cc" * 32, | |
| 45 | + "vin": [{"prevout": {"scriptpubkey_address": "bc1self"}}], | |
| 46 | + "vout": [{"value": 20_0000_0000, "scriptpubkey_address": "bc1self"}], | |
| 47 | + }, | |
| 48 | + { # below threshold → ignored | |
| 49 | + "txid": "dd" * 32, | |
| 50 | + "vin": [{"prevout": {"scriptpubkey_address": "bc1small"}}], | |
| 51 | + "vout": [{"value": 1_0000_0000, "scriptpubkey_address": "bc1w"}], | |
| 52 | + }, | |
| 53 | + ] | |
| 54 | + n = ix.process_block(961000, "HASH") | |
| 55 | + rows = ix.conn.execute('SELECT tx_hash, "from", "to", amount FROM transfers ORDER BY tx_hash').fetchall() | |
| 56 | + assert n == 2 and len(rows) == 2 | |
| 57 | + assert rows[0][1] == "bc1alice" and rows[0][2] == "bc1bob" and rows[0][3] == "1200000000" | |
| 58 | + assert rows[1][1] is None and rows[1][2] == "bc1z" | |
| 59 | + | |
| 60 | + | |
| 61 | +EVM_CFG = {"rpcs": ["http://unused.invalid"], "max_range": 8, | |
| 62 | + "confirmations": 2, "native": {"symbol": "ETH", "min": 50, "decimals": 18}} | |
| 63 | + | |
| 64 | + | |
| 65 | +def test_evm_native_whales(): | |
| 66 | + ix = ChainIndexer("ethereum", EVM_CFG, [], ":memory:") | |
| 67 | + ix.conn = db.connect(":memory:") | |
| 68 | + | |
| 69 | + def fake_call(method, params=None): | |
| 70 | + assert method == "eth_getBlockByNumber" and params[1] is True | |
| 71 | + n = int(params[0], 16) | |
| 72 | + return { | |
| 73 | + "number": hex(n), "hash": "0x" + f"{n:x}".rjust(64, "b"), | |
| 74 | + "timestamp": hex(1786000000 + n), | |
| 75 | + "transactions": [ | |
| 76 | + {"hash": "0x" + f"{n:x}1".rjust(64, "a"), "from": "0xAA", "to": "0xBB", | |
| 77 | + "value": hex(60 * 10**18)}, # 60 ETH → recorded | |
| 78 | + {"hash": "0x" + f"{n:x}2".rjust(64, "a"), "from": "0xCC", "to": "0xDD", | |
| 79 | + "value": hex(1 * 10**18)}, # 1 ETH → ignored | |
| 80 | + ], | |
| 81 | + } | |
| 82 | + ix.pool.batch = lambda calls: [fake_call(m, p) for m, p in calls] | |
| 83 | + ix.process_native(100, 101) | |
| 84 | + rows = ix.conn.execute("SELECT symbol, token, amount FROM transfers").fetchall() | |
| 85 | + assert len(rows) == 2 # one whale per block | |
| 86 | + assert all(r[0] == "ETH" and r[1] == "native" and r[2] == str(60 * 10**18) for r in rows) | |
| 87 | + | |
| 88 | + | |
| 89 | +def test_prices_upsert_and_join(): | |
| 90 | + conn = db.connect(":memory:") | |
| 91 | + db.upsert_prices(conn, [("ETH", 3000.0, 1786000000), ("USDT", 1.0, 1786000000)]) | |
| 92 | + db.upsert_prices(conn, [("ETH", 3100.0, 1786000300)]) # update wins | |
| 93 | + r = conn.execute("SELECT usd FROM prices WHERE symbol = 'ETH'").fetchone() | |
| 94 | + assert r[0] == 3100.0 | |
| 95 | + | |
| 96 | + | |
| 97 | +if __name__ == "__main__": | |
| 98 | + for name, fn in sorted(globals().items()): | |
| 99 | + if name.startswith("test_"): | |
| 100 | + fn() | |
| 101 | + print(f"ok {name}") | |
| 102 | + print("all v3 tests passed") | |
added
ui/address.html
+60 −0
@@ -0,0 +1,60 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>address — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1>Address</h1><span class="sub mono" id="addr" style="max-width:none"></span></div> | |
| 13 | + | |
| 14 | + <section class="card tile"><h2>Transfers indexed</h2><div class="big" id="tCount">—</div></section> | |
| 15 | + <section class="card tile"><h2>Received</h2><div class="big up" id="tIn">—</div></section> | |
| 16 | + <section class="card tile"><h2>Sent</h2><div class="big down" id="tOut">—</div></section> | |
| 17 | + <section class="card tile"><h2>Chains seen</h2><div class="big" id="tChains">—</div><div class="note" id="tChainsN"></div></section> | |
| 18 | + | |
| 19 | + <section class="card"><h2>Cross-chain transfer history <span class="sub">(within indexed window)</span></h2> | |
| 20 | + <div class="scroll" style="max-height:65vh"><table> | |
| 21 | + <thead><tr><th></th><th>chain</th><th>token</th><th class="num">amount</th><th>counterparty</th><th>tx</th><th>when</th></tr></thead> | |
| 22 | + <tbody id="rows"></tbody></table></div></section> | |
| 23 | +</main> | |
| 24 | + | |
| 25 | +<script type="module"> | |
| 26 | +import { $, qs, j, fmtUsd, fmtN, fmtAmt, mountNav, addrLink, txLink, chainLink, tokenLink, ago } from "/assets/app.js"; | |
| 27 | +mountNav("Transfers"); | |
| 28 | +const A = qs.get("a") || ""; | |
| 29 | +$("addr").textContent = A || "no address given"; | |
| 30 | +document.title = `${A.slice(0, 14)}… — coinexplorer`; | |
| 31 | + | |
| 32 | +async function load() { | |
| 33 | + if (!A) return; | |
| 34 | + const rows = await j(`/v1/address/${encodeURIComponent(A)}/transfers?limit=500`); | |
| 35 | + const me = (x) => x === A || (A.startsWith("0x") && x === A.toLowerCase()); | |
| 36 | + let recv = 0, sent = 0; | |
| 37 | + const chains = new Set(); | |
| 38 | + for (const r of rows) { | |
| 39 | + chains.add(r.chain); | |
| 40 | + if (me(r.to)) recv += (r.usd ?? parseFloat(r.value)); | |
| 41 | + if (me(r.from)) sent += (r.usd ?? parseFloat(r.value)); | |
| 42 | + } | |
| 43 | + $("tCount").textContent = fmtN(rows.length) + (rows.length === 500 ? "+" : ""); | |
| 44 | + $("tIn").textContent = "+" + fmtUsd(recv); | |
| 45 | + $("tOut").textContent = "−" + fmtUsd(sent); | |
| 46 | + $("tChains").textContent = chains.size; | |
| 47 | + $("tChainsN").textContent = [...chains].join(", "); | |
| 48 | + $("rows").innerHTML = rows.map((r) => { | |
| 49 | + const out = me(r.from); | |
| 50 | + return `<tr><td class="${out ? "burn" : "mint"}">${out ? "↑ out" : "↓ in"}</td> | |
| 51 | + <td>${chainLink(r.chain)}</td><td>${tokenLink(r.symbol)}</td> | |
| 52 | + <td class="num">${fmtAmt(r)}</td> | |
| 53 | + <td>${addrLink(out ? r.to : r.from)}</td><td>${txLink(r.chain, r.tx_hash)}</td> | |
| 54 | + <td>${ago(r.timestamp)} ago</td></tr>`; | |
| 55 | + }).join("") || `<tr><td colspan="7" class="empty">nothing indexed for this address (index covers the recent window only)</td></tr>`; | |
| 56 | +} | |
| 57 | +load(); | |
| 58 | +</script> | |
| 59 | +</body> | |
| 60 | +</html> | |
added
ui/api.html
+160 −0
@@ -0,0 +1,160 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>API — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="hero"> | |
| 13 | + <h1>coinexplorer API</h1> | |
| 14 | + <p>Free, keyless, self-hosted market data for stablecoins and major crypto — | |
| 15 | + indexed straight from public RPCs across 24 chains. Every endpoint below is | |
| 16 | + live: hit <b>Try</b> to run it against this instance right now. | |
| 17 | + OpenAPI/Swagger available at <a href="/docs">/docs</a>.</p> | |
| 18 | + <div class="kpis"> | |
| 19 | + <div><b id="kChains">—</b><span>chains</span></div> | |
| 20 | + <div><b id="kTransfers">—</b><span>transfers indexed</span></div> | |
| 21 | + <div><b id="kTokens">—</b><span>assets tracked</span></div> | |
| 22 | + <div><b id="kPrices">—</b><span>live prices</span></div> | |
| 23 | + </div> | |
| 24 | + </div> | |
| 25 | + | |
| 26 | + <div class="docwrap"> | |
| 27 | + <nav class="toc" id="toc" aria-label="Endpoints"></nav> | |
| 28 | + <div class="docbody" id="body"></div> | |
| 29 | + </div> | |
| 30 | +</main> | |
| 31 | + | |
| 32 | +<script type="module"> | |
| 33 | +import { $, j, fmtN, mountNav } from "/assets/app.js"; | |
| 34 | +mountNav("API"); | |
| 35 | + | |
| 36 | +const BASE = location.origin; | |
| 37 | + | |
| 38 | +const GROUPS = [ | |
| 39 | +["Market data", [ | |
| 40 | + { m: "GET", p: "/v1/stablecoins/supply", d: "Latest on-chain circulating supply per token per chain (totalSupply / getTokenSupply / triggerconstantcontract), with USD totals. Snapshots hourly.", | |
| 41 | + q: [["token", "filter one symbol, e.g. USDT", "no"]], t: "/v1/stablecoins/supply?token=USDT" }, | |
| 42 | + { m: "GET", p: "/v1/stablecoins/supply/series", d: "Supply snapshots over time per chain — powers the supply history charts.", | |
| 43 | + q: [["token", "symbol (default USDT)", "no"], ["window", "15m / 24h / 7d style (default 7d)", "no"]], t: "/v1/stablecoins/supply/series?token=USDC&window=7d" }, | |
| 44 | + { m: "GET", p: "/v1/stablecoins/volume", d: "Rolling transfer volume in USD + transfer counts + distinct active senders/receivers, per chain. Served from precomputed aggregates (ms responses at any index size).", | |
| 45 | + q: [["token", "symbol (default USDT)", "no"], ["window", "1h / 24h / 7d (default 24h)", "no"]], t: "/v1/stablecoins/volume?token=USDT&window=24h" }, | |
| 46 | + { m: "GET", p: "/v1/stablecoins/volume/series", d: "Time-bucketed volume per chain for line charts, precomputed for the standard windows.", | |
| 47 | + q: [["token", "symbol", "no"], ["window", "1h / 24h / 7d", "no"], ["interval", "bucket size, e.g. 1h", "no"], ["chain", "restrict to one chain", "no"]], t: "/v1/stablecoins/volume/series?token=USDT&window=24h&interval=1h" }, | |
| 48 | + { m: "GET", p: "/v1/stablecoins/whales", d: "Largest transfers in the window, USD-valued with live prices (native BTC/ETH/SOL/TRX/BNB whales included). Backed by an incrementally-maintained whale table — floor $100K.", | |
| 49 | + q: [["token", "symbol filter", "no"], ["window", "default 24h", "no"], ["min_usd", "threshold (default $1M)", "no"], ["limit", "max 500", "no"]], t: "/v1/stablecoins/whales?window=24h&min_usd=10000000&limit=5" }, | |
| 50 | + { m: "GET", p: "/v1/stablecoins/mints-burns", d: "Issuance events: transfers from the zero address are mints, to it are burns (EVM + Tron zero addresses).", | |
| 51 | + q: [["token", "symbol filter", "no"], ["window", "default 24h", "no"], ["limit", "max 500", "no"]], t: "/v1/stablecoins/mints-burns?token=USDC&window=24h&limit=5" }, | |
| 52 | + { m: "GET", p: "/v1/stablecoins/flows", d: "Net issuance over time: minted minus burned per bucket — the flows chart.", | |
| 53 | + q: [["token", "symbol filter", "no"], ["window", "default 7d", "no"], ["interval", "bucket size (default 1d)", "no"]], t: "/v1/stablecoins/flows?token=USDC&window=7d&interval=1d" }, | |
| 54 | + { m: "GET", p: "/v1/prices", d: "Latest USD prices used for valuation (CoinGecko keyless tier; stablecoins seeded at 1.0).", | |
| 55 | + q: [], t: "/v1/prices" }, | |
| 56 | +]], | |
| 57 | +["Explorer", [ | |
| 58 | + { m: "GET", p: "/v1/{chain}/transfers", d: "Recent indexed transfers on one chain, all tokens. Paginate with before_block.", | |
| 59 | + q: [["symbol", "token filter", "no"], ["min_amount", "min USD", "no"], ["before_block", "pagination cursor", "no"], ["limit", "max 500", "no"]], t: "/v1/ethereum/transfers?limit=3" }, | |
| 60 | + { m: "GET", p: "/v1/{chain}/token/{id}/transfers", d: "Indexed transfers of one token contract/mint on one chain.", | |
| 61 | + q: [["from / to", "address filters", "no"], ["min_amount", "min token units", "no"], ["since", "unix ts lower bound", "no"], ["before_block", "pagination", "no"]], t: "/v1/ethereum/token/0xdAC17F958D2ee523a2206206994597C13D831ec7/transfers?limit=3" }, | |
| 62 | + { m: "GET", p: "/v1/address/{addr}/transfers", d: "Cross-chain transfer history for any address (EVM hex, Tron T-addr, Solana pubkey, BTC bech32…).", | |
| 63 | + q: [["limit", "max 500", "no"]], t: null }, | |
| 64 | + { m: "GET", p: "/v1/{chain}/tx/{hash}", d: "Transaction detail with decoded stablecoin/crypto transfers. EVM chains answer live from RPC; other families answer from the index.", | |
| 65 | + q: [], t: null }, | |
| 66 | + { m: "GET", p: "/v1/{chain}/block/latest", d: "Live head block summary straight from the chain's free RPC pool (EVM).", | |
| 67 | + q: [], t: "/v1/ethereum/block/latest" }, | |
| 68 | + { m: "GET", p: "/v1/{chain}/block/{n}", d: "Block summary + every stablecoin transfer inside it, decoded live (EVM).", | |
| 69 | + q: [], t: null }, | |
| 70 | + { m: "GET", p: "/v1/{chain}/summary", d: "One-call chain profile: token registry, 24h volume by symbol, cursor/lag.", | |
| 71 | + q: [], t: "/v1/bitcoin/summary" }, | |
| 72 | + { m: "GET", p: "/v1/search", d: "Universal resolver: classifies a tx hash / address / token symbol / chain name — checks the index first, so it can tell you which chain a tx lives on.", | |
| 73 | + q: [["q", "the query string", "yes"]], t: "/v1/search?q=USDT" }, | |
| 74 | +]], | |
| 75 | +["Live", [ | |
| 76 | + { m: "WS", p: "/v1/stream/transfers", d: "WebSocket pushing new transfers every ~2s as JSON arrays. Filters via query params.", | |
| 77 | + q: [["token", "symbol filter", "no"], ["chain", "chain filter", "no"], ["min_usd", "value floor", "no"]], t: null, | |
| 78 | + code: `const ws = new WebSocket("wss://${location.host}/v1/stream/transfers?min_usd=100000"); | |
| 79 | +ws.onmessage = (e) => JSON.parse(e.data).forEach(t => | |
| 80 | + console.log(t.chain, t.symbol, t.value, "$" + t.usd, t.tx_hash));` }, | |
| 81 | +]], | |
| 82 | +["Meta & infra", [ | |
| 83 | + { m: "GET", p: "/v1/tokens", d: "The verified asset registry: every token grouped by symbol with per-chain identifier, decimals, native vs bridged, category (stablecoin / crypto).", | |
| 84 | + q: [], t: "/v1/tokens" }, | |
| 85 | + { m: "GET", p: "/v1/chains", d: "Chain registry: family, chain id, block time, confirmations, tokens.", | |
| 86 | + q: [], t: "/v1/chains" }, | |
| 87 | + { m: "GET", p: "/v1/status", d: "Indexer cursors per chain: last block, live head, lag, backfill floor, row counts.", | |
| 88 | + q: [], t: "/v1/status" }, | |
| 89 | + { m: "GET", p: "/v1/rpc/health", d: "Free-RPC endpoint health as scored by the indexer: success-rate EMA, latency, cooldowns.", | |
| 90 | + q: [], t: "/v1/rpc/health" }, | |
| 91 | + { m: "GET", p: "/metrics", d: "Prometheus text exposition: transfers, per-chain lag, per-endpoint scores.", | |
| 92 | + q: [], t: null }, | |
| 93 | +]], | |
| 94 | +]; | |
| 95 | + | |
| 96 | +const slug = (p) => p.replace(/[^a-z0-9]+/gi, "-").replace(/^-|-$/g, ""); | |
| 97 | +const esc = (s) => s.replace(/&/g, "&").replace(/</g, "<"); | |
| 98 | + | |
| 99 | +function card(ep) { | |
| 100 | + const id = slug(ep.p); | |
| 101 | + const params = ep.q.length ? ` | |
| 102 | + <div class="params"><table> | |
| 103 | + <thead><tr><th>param</th><th>description</th><th>required</th></tr></thead> | |
| 104 | + <tbody>${ep.q.map(([n, d, r]) => | |
| 105 | + `<tr><td class="mono" style="max-width:none">${n}</td><td style="white-space:normal">${d}</td><td>${r}</td></tr>`).join("")} | |
| 106 | + </tbody></table></div>` : ""; | |
| 107 | + const curl = ep.m === "WS" ? "" : | |
| 108 | + `<pre class="code"><span class="cm"># curl</span> | |
| 109 | +curl -s "${BASE}${ep.t || ep.p}"</pre>`; | |
| 110 | + const code = ep.code ? `<pre class="code">${esc(ep.code)}</pre>` : ""; | |
| 111 | + const tryBtn = ep.t ? ` | |
| 112 | + <div class="tryrow"> | |
| 113 | + <button class="btn" data-try="${ep.t}" data-out="out-${id}">▶ Try it</button> | |
| 114 | + <span class="sub" id="ms-${id}"></span> | |
| 115 | + </div> | |
| 116 | + <pre class="code try-out" id="out-${id}"></pre>` : ""; | |
| 117 | + return `<section class="ep" id="${id}"> | |
| 118 | + <div class="head"><span class="method ${ep.m === "WS" ? "ws" : ""}">${ep.m}</span> | |
| 119 | + <span class="path">${ep.p}</span> | |
| 120 | + <button class="btn copybtn" data-copy="${BASE}${ep.t || ep.p}">copy URL</button></div> | |
| 121 | + <p class="desc">${ep.d}</p> | |
| 122 | + ${params}${curl}${code}${tryBtn} | |
| 123 | + </section>`; | |
| 124 | +} | |
| 125 | + | |
| 126 | +$("toc").innerHTML = GROUPS.map(([g, eps]) => | |
| 127 | + `<div class="group">${g}</div>` + eps.map((e) => | |
| 128 | + `<a href="#${slug(e.p)}">${e.p.replace("/v1/", "").replace("stablecoins/", "")}</a>`).join("")).join(""); | |
| 129 | +$("body").innerHTML = GROUPS.map(([g, eps]) => eps.map(card).join("")).join(""); | |
| 130 | + | |
| 131 | +document.querySelectorAll("[data-try]").forEach((b) => b.addEventListener("click", async () => { | |
| 132 | + const out = $(b.dataset.out); | |
| 133 | + out.style.display = "block"; | |
| 134 | + out.textContent = "…"; | |
| 135 | + const t0 = performance.now(); | |
| 136 | + try { | |
| 137 | + const r = await fetch(b.dataset.try); | |
| 138 | + const data = await r.json(); | |
| 139 | + const ms = Math.round(performance.now() - t0); | |
| 140 | + document.getElementById("ms-" + b.dataset.out.slice(4)).textContent = `${r.status} · ${ms} ms`; | |
| 141 | + let text = JSON.stringify(data, null, 2); | |
| 142 | + if (text.length > 4000) text = text.slice(0, 4000) + "\n… (truncated)"; | |
| 143 | + out.textContent = text; | |
| 144 | + } catch (e) { out.textContent = "request failed: " + e; } | |
| 145 | +})); | |
| 146 | +document.querySelectorAll("[data-copy]").forEach((b) => b.addEventListener("click", () => { | |
| 147 | + navigator.clipboard?.writeText(b.dataset.copy); | |
| 148 | + b.textContent = "copied ✓"; | |
| 149 | + setTimeout(() => (b.textContent = "copy URL"), 1200); | |
| 150 | +})); | |
| 151 | + | |
| 152 | +Promise.all([j("/v1/status"), j("/v1/tokens"), j("/v1/prices")]).then(([s, t, p]) => { | |
| 153 | + $("kChains").textContent = Object.keys(s.cursors).length; | |
| 154 | + $("kTransfers").textContent = fmtN(Object.values(s.indexed_transfers).reduce((a, b) => a + b, 0)); | |
| 155 | + $("kTokens").textContent = Object.keys(t).length; | |
| 156 | + $("kPrices").textContent = Object.keys(p).length; | |
| 157 | +}).catch(() => {}); | |
| 158 | +</script> | |
| 159 | +</body> | |
| 160 | +</html> | |
added
ui/assets/app.js
+377 −0
@@ -0,0 +1,377 @@ | ||
| 1 | +// Author: Simon-Pierre Boucher | |
| 2 | +// Mail: contact@spboucher.ai | |
| 3 | +// Shared front-end: nav, search, formatting, tooltip, bar rows, SVG line/column charts. | |
| 4 | + | |
| 5 | +export const $ = (id) => document.getElementById(id); | |
| 6 | +export const qs = new URLSearchParams(location.search); | |
| 7 | + | |
| 8 | +export async function j(url) { | |
| 9 | + const r = await fetch(url); | |
| 10 | + if (!r.ok) throw new Error(`${r.status} ${url}`); | |
| 11 | + return r.json(); | |
| 12 | +} | |
| 13 | + | |
| 14 | +// ---------- formatting ---------- | |
| 15 | +export const fmtUsd = (n) => { | |
| 16 | + const a = Math.abs(n); | |
| 17 | + if (a >= 1e12) return "$" + (n / 1e12).toFixed(2) + "T"; | |
| 18 | + if (a >= 1e9) return "$" + (n / 1e9).toFixed(2) + "B"; | |
| 19 | + if (a >= 1e6) return "$" + (n / 1e6).toFixed(1) + "M"; | |
| 20 | + if (a >= 1e3) return "$" + (n / 1e3).toFixed(1) + "K"; | |
| 21 | + return "$" + n.toFixed(2); | |
| 22 | +}; | |
| 23 | +export const fmtN = (n) => (n ?? 0).toLocaleString("en-US"); | |
| 24 | +export const fmtQty = (n) => { | |
| 25 | + const a = Math.abs(n); | |
| 26 | + if (a >= 1e9) return (n / 1e9).toFixed(2) + "B"; | |
| 27 | + if (a >= 1e6) return (n / 1e6).toFixed(2) + "M"; | |
| 28 | + if (a >= 1e3) return (n / 1e3).toFixed(1) + "K"; | |
| 29 | + return n.toFixed(a < 1 ? 4 : 2); | |
| 30 | +}; | |
| 31 | + | |
| 32 | +// token categories (stablecoin vs crypto) — loaded once by mountNav | |
| 33 | +export let CATS = {}; | |
| 34 | +export const isStable = (sym) => (CATS[sym] || "stablecoin") !== "crypto"; | |
| 35 | + | |
| 36 | +// row amount: stablecoins read as USD directly; crypto shows units + USD | |
| 37 | +export function fmtAmt(r) { | |
| 38 | + const v = parseFloat(r.value); | |
| 39 | + if (!isStable(r.symbol) && r.usd != null) | |
| 40 | + return `${fmtQty(v)} ${r.symbol} <span style="color:var(--muted)">· ${fmtUsd(r.usd)}</span>`; | |
| 41 | + return fmtUsd(r.usd ?? v); | |
| 42 | +} | |
| 43 | +export const short = (s) => (s ? s.slice(0, 8) + "…" + s.slice(-4) : "—"); | |
| 44 | +export const ago = (ts) => { | |
| 45 | + if (!ts) return "—"; | |
| 46 | + const d = Math.max(0, Date.now() / 1000 - ts); | |
| 47 | + return d < 90 ? Math.round(d) + "s" : d < 5400 ? Math.round(d / 60) + "m" | |
| 48 | + : d < 129600 ? Math.round(d / 3600) + "h" : Math.round(d / 86400) + "d"; | |
| 49 | +}; | |
| 50 | +const timeLabel = (t, span) => { | |
| 51 | + const d = new Date(t * 1000); | |
| 52 | + return span > 3 * 86400 | |
| 53 | + ? d.toLocaleDateString("en-US", { month: "short", day: "numeric" }) | |
| 54 | + : d.toLocaleTimeString("en-US", { hour: "2-digit", minute: "2-digit" }); | |
| 55 | +}; | |
| 56 | +export const addrLink = (a) => a | |
| 57 | + ? `<a href="/address.html?a=${encodeURIComponent(a)}" class="mono" title="${a}">${short(a)}</a>` : "—"; | |
| 58 | +export const txLink = (chain, h) => | |
| 59 | + `<a href="/tx.html?chain=${chain}&hash=${encodeURIComponent(h)}" class="mono" title="${h}">${short(h)}</a>`; | |
| 60 | + | |
| 61 | +// ---------- asset & chain icons ---------- | |
| 62 | +// Real logos from the cryptocurrency-icons CDN where available; everything | |
| 63 | +// else falls back to a brand-colored coin with the ticker's initial — the | |
| 64 | +// <img> sits above the fallback and simply removes itself on 404. | |
| 65 | +const BRAND = { | |
| 66 | + BTC:"#f7931a", WBTC:"#f09242", cbBTC:"#0052ff", ETH:"#627eea", WETH:"#627eea", | |
| 67 | + USDT:"#26a17b", USDC:"#2775ca", DAI:"#f5ac37", USDS:"#f5ac37", USDe:"#24354f", | |
| 68 | + FDUSD:"#01c38d", PYUSD:"#0070e0", RLUSD:"#0085c0", USDG:"#02414c", TUSD:"#1a5aff", | |
| 69 | + USDB:"#fcfc03", LINK:"#2a5ada", UNI:"#ff007a", AAVE:"#9391f7", SHIB:"#ffa409", | |
| 70 | + PEPE:"#4c9641", LDO:"#f69988", CRV:"#870f9e", ONDO:"#0f1728", MKR:"#1aab9b", | |
| 71 | + ENA:"#11314f", ARB:"#12aaff", OP:"#ff0420", BNB:"#f3ba2f", WBNB:"#f3ba2f", | |
| 72 | + CAKE:"#d1884f", AVAX:"#e84142", WAVAX:"#e84142", POL:"#8247e5", WPOL:"#8247e5", | |
| 73 | + SOL:"#9945ff", TRX:"#ff0013", BONK:"#f9a33d", JUP:"#16bee2", WIF:"#a58b6f", | |
| 74 | +}; | |
| 75 | +const CHAIN_BRAND = { | |
| 76 | + ethereum:"#627eea", bitcoin:"#f7931a", tron:"#ff0013", solana:"#9945ff", | |
| 77 | + bsc:"#f3ba2f", polygon:"#8247e5", avalanche:"#e84142", arbitrum:"#12aaff", | |
| 78 | + optimism:"#ff0420", base:"#0052ff", celo:"#fcff52", gnosis:"#04795b", | |
| 79 | + linea:"#121212", scroll:"#eb7106", mantle:"#141414", zksync:"#8c8dfc", | |
| 80 | + blast:"#fcfc03", unichain:"#ff007a", world_chain:"#1e1e1e", sonic:"#0a1e3f", | |
| 81 | + sei:"#8c1d18", hyperevm:"#0f3933", kaia:"#5f7f12", ink:"#7132f5", | |
| 82 | +}; | |
| 83 | +const CDN = "https://cdn.jsdelivr.net/npm/cryptocurrency-icons@0.18.1/svg/color/"; | |
| 84 | +const CDN_SYM = { // package symbol overrides / aliases | |
| 85 | + WETH:"eth", cbBTC:"btc", WBNB:"bnb", WAVAX:"avax", WPOL:"matic", POL:"matic", | |
| 86 | +}; | |
| 87 | +const CHAIN_SYM = { // chain → package symbol (only where a real logo exists) | |
| 88 | + ethereum:"eth", bitcoin:"btc", tron:"trx", solana:"sol", bsc:"bnb", | |
| 89 | + polygon:"matic", avalanche:"avax", | |
| 90 | +}; | |
| 91 | +const NO_CDN = new Set(["USDe","FDUSD","PYUSD","RLUSD","USDG","USDB","PEPE","ONDO", | |
| 92 | + "ENA","ARB","OP","CAKE","BONK","JUP","WIF","LDO","USDS","SHIB"]); | |
| 93 | + | |
| 94 | +const luma = (hex) => { | |
| 95 | + const n = parseInt(hex.slice(1), 16); | |
| 96 | + return (0.299 * (n >> 16) + 0.587 * ((n >> 8) & 255) + 0.114 * (n & 255)) / 255; | |
| 97 | +}; | |
| 98 | +function coinHtml(text, color, cdnSym, size) { | |
| 99 | + const fg = luma(color) > 0.62 ? "#0b0b0b" : "#fff"; | |
| 100 | + const img = cdnSym | |
| 101 | + ? `<img src="${CDN}${cdnSym}.svg" alt="" loading="lazy" onerror="this.remove()">` : ""; | |
| 102 | + return `<span class="coin" style="--sz:${size}px">` + | |
| 103 | + `<i style="background:linear-gradient(135deg,${color},color-mix(in oklab,${color} 65%,#000));color:${fg}">${text[0]}</i>${img}</span>`; | |
| 104 | +} | |
| 105 | +export function coinIcon(sym, size = 16) { | |
| 106 | + const color = BRAND[sym] || "#6b7280"; | |
| 107 | + const cdn = NO_CDN.has(sym) ? null : (CDN_SYM[sym] || sym.toLowerCase()); | |
| 108 | + return coinHtml(sym, color, cdn, size); | |
| 109 | +} | |
| 110 | +export function chainIcon(chain, size = 16) { | |
| 111 | + const color = CHAIN_BRAND[chain] || "#6b7280"; | |
| 112 | + return coinHtml(chain.toUpperCase(), color, CHAIN_SYM[chain] || null, size); | |
| 113 | +} | |
| 114 | +export const chainLink = (c) => | |
| 115 | + `<a class="asset" href="/chain.html?chain=${c}">${chainIcon(c)}<span>${c}</span></a>`; | |
| 116 | +export const tokenLink = (s) => | |
| 117 | + `<a class="asset" href="/token.html?symbol=${s}">${coinIcon(s)}<span>${s}</span></a>`; | |
| 118 | + | |
| 119 | +// ---------- nav ---------- | |
| 120 | +const NAV = [ | |
| 121 | + ["Overview", "/"], ["Tokens", "/token.html"], ["Chains", "/chain.html"], | |
| 122 | + ["Transfers", "/transfers.html"], ["Whales", "/whales.html"], | |
| 123 | + ["Flows", "/flows.html"], ["API", "/api.html"], ["Status", "/status.html"], | |
| 124 | +]; | |
| 125 | +export function mountNav(active) { | |
| 126 | + const el = document.createElement("div"); | |
| 127 | + el.className = "topnav"; | |
| 128 | + el.innerHTML = ` | |
| 129 | + <a class="logo" href="/">coin<span>explorer</span></a> | |
| 130 | + <nav>${NAV.map(([n, h]) => | |
| 131 | + `<a href="${h}" class="${n === active ? "active" : ""}">${n}</a>`).join("")}</nav> | |
| 132 | + <form class="searchbox" id="searchForm"> | |
| 133 | + <input id="searchInput" placeholder="tx hash · address · token · chain" aria-label="Search"> | |
| 134 | + </form>`; | |
| 135 | + document.body.prepend(el); | |
| 136 | + j("/v1/tokens").then((t) => { | |
| 137 | + for (const [sym, d] of Object.entries(t)) CATS[sym] = d.category; | |
| 138 | + // natives tracked via chains.yaml aren't in /v1/tokens — they're crypto | |
| 139 | + for (const s of ["ETH", "BTC", "BNB", "SOL", "TRX", "AVAX", "POL"]) | |
| 140 | + if (!(s in CATS)) CATS[s] = "crypto"; | |
| 141 | + }).catch(() => {}); | |
| 142 | + const tip = document.createElement("div"); | |
| 143 | + tip.id = "tooltip"; | |
| 144 | + document.body.appendChild(tip); | |
| 145 | + $("searchForm").addEventListener("submit", async (e) => { | |
| 146 | + e.preventDefault(); | |
| 147 | + const q = $("searchInput").value.trim(); | |
| 148 | + if (!q) return; | |
| 149 | + try { | |
| 150 | + const r = await j(`/v1/search?q=${encodeURIComponent(q)}`); | |
| 151 | + if (r.type === "token") location.href = `/token.html?symbol=${r.symbol}`; | |
| 152 | + else if (r.type === "chain") location.href = `/chain.html?chain=${r.chain}`; | |
| 153 | + else if (r.type === "tx") location.href = `/tx.html?hash=${encodeURIComponent(r.hash)}${r.chain ? `&chain=${r.chain}` : ""}`; | |
| 154 | + else if (r.type === "address") location.href = `/address.html?a=${encodeURIComponent(r.address)}`; | |
| 155 | + else alert("No match — try a tx hash, address, token symbol or chain name."); | |
| 156 | + } catch { alert("Search failed."); } | |
| 157 | + }); | |
| 158 | + const f = document.createElement("footer"); | |
| 159 | + f.innerHTML = "coinexplorer · self-hosted stablecoin explorer · free public RPCs only · " + | |
| 160 | + '<a href="/docs">API</a> · <a href="/metrics">metrics</a> · not financial advice'; | |
| 161 | + document.body.appendChild(f); | |
| 162 | +} | |
| 163 | + | |
| 164 | +// ---------- tooltip ---------- | |
| 165 | +export function tipShow(e, html) { | |
| 166 | + const tip = $("tooltip"); | |
| 167 | + tip.innerHTML = html; | |
| 168 | + tip.style.display = "block"; | |
| 169 | + tip.style.left = Math.min(e.clientX + 14, innerWidth - 230) + "px"; | |
| 170 | + tip.style.top = Math.min(e.clientY + 14, innerHeight - 90) + "px"; | |
| 171 | +} | |
| 172 | +export function tipHide() { $("tooltip").style.display = "none"; } | |
| 173 | +export function attachTip(el, html) { | |
| 174 | + el.addEventListener("mousemove", (e) => tipShow(e, typeof html === "function" ? html() : html)); | |
| 175 | + el.addEventListener("mouseleave", tipHide); | |
| 176 | +} | |
| 177 | + | |
| 178 | +// ---------- horizontal bars (single-measure magnitude) ---------- | |
| 179 | +export function hbars(el, entries, fmt = fmtUsd) { | |
| 180 | + el.innerHTML = ""; | |
| 181 | + if (!entries.length) { el.innerHTML = `<div class="empty">no data yet</div>`; return; } | |
| 182 | + const max = Math.max(...entries.map((e) => e.v), 1e-9); | |
| 183 | + for (const e of entries) { | |
| 184 | + const row = document.createElement("div"); | |
| 185 | + row.className = "bar-row"; | |
| 186 | + row.innerHTML = `<span class="bar-label">${e.link || e.k}</span> | |
| 187 | + <span class="bar-track"><span class="bar-fill" style="width:${(100 * e.v / max).toFixed(2)}%"></span></span> | |
| 188 | + <span class="bar-val">${fmt(e.v)}</span>`; | |
| 189 | + attachTip(row, `<b>${e.k}</b>${fmt(e.v)}${e.extra || ""}`); | |
| 190 | + el.appendChild(row); | |
| 191 | + } | |
| 192 | +} | |
| 193 | + | |
| 194 | +// ---------- SVG helpers ---------- | |
| 195 | +const NS = "http://www.w3.org/2000/svg"; | |
| 196 | +const svgEl = (tag, attrs) => { | |
| 197 | + const n = document.createElementNS(NS, tag); | |
| 198 | + for (const [k, v] of Object.entries(attrs)) n.setAttribute(k, v); | |
| 199 | + return n; | |
| 200 | +}; | |
| 201 | +const niceTicks = (max, n = 4) => { | |
| 202 | + if (max <= 0) return [0, 1]; | |
| 203 | + const step0 = max / n, mag = 10 ** Math.floor(Math.log10(step0)); | |
| 204 | + const step = [1, 2, 2.5, 5, 10].map((m) => m * mag).find((s) => max / s <= n) || mag * 10; | |
| 205 | + const out = []; | |
| 206 | + for (let v = 0; v <= max * 1.001; v += step) out.push(v); | |
| 207 | + return out; | |
| 208 | +}; | |
| 209 | +const SLOTS = ["var(--s1)", "var(--s2)", "var(--s3)", "var(--s4)"]; | |
| 210 | + | |
| 211 | +// ---------- multi-series line chart with crosshair ---------- | |
| 212 | +export function lineChart(el, series, { fmtY = fmtUsd, area = true } = {}) { | |
| 213 | + el.innerHTML = ""; | |
| 214 | + series = series.filter((s) => s.points.length); | |
| 215 | + if (!series.length) { el.innerHTML = `<div class="empty">no data yet</div>`; return; } | |
| 216 | + const W = 820, H = 250, L = 58, R = 10, T = 12, B = 26; | |
| 217 | + const ts = [...new Set(series.flatMap((s) => s.points.map((p) => p[0])))].sort((a, b) => a - b); | |
| 218 | + const t0 = ts[0], t1 = ts[ts.length - 1] || t0 + 1; | |
| 219 | + const span = Math.max(1, t1 - t0); | |
| 220 | + const vmax = Math.max(...series.flatMap((s) => s.points.map((p) => p[1])), 1e-9) * 1.06; | |
| 221 | + const x = (t) => L + ((t - t0) / span) * (W - L - R); | |
| 222 | + const y = (v) => T + (1 - v / vmax) * (H - T - B); | |
| 223 | + const svg = svgEl("svg", { viewBox: `0 0 ${W} ${H}`, role: "img" }); | |
| 224 | + | |
| 225 | + for (const v of niceTicks(vmax)) { // grid + y labels | |
| 226 | + svg.appendChild(svgEl("line", { x1: L, x2: W - R, y1: y(v), y2: y(v), | |
| 227 | + stroke: "var(--grid)", "stroke-width": 1 })); | |
| 228 | + const t = svgEl("text", { x: L - 6, y: y(v) + 4, "text-anchor": "end", | |
| 229 | + "font-size": 10.5, fill: "var(--muted)" }); | |
| 230 | + t.textContent = fmtY(v).replace("$", "$"); | |
| 231 | + svg.appendChild(t); | |
| 232 | + } | |
| 233 | + const nx = Math.min(6, ts.length); // x labels | |
| 234 | + for (let i = 0; i < nx; i++) { | |
| 235 | + const t = t0 + (span * i) / Math.max(1, nx - 1); | |
| 236 | + const lbl = svgEl("text", { x: x(t), y: H - 8, "text-anchor": "middle", | |
| 237 | + "font-size": 10.5, fill: "var(--muted)" }); | |
| 238 | + lbl.textContent = timeLabel(t, span); | |
| 239 | + svg.appendChild(lbl); | |
| 240 | + } | |
| 241 | + svg.appendChild(svgEl("line", { x1: L, x2: W - R, y1: y(0), y2: y(0), | |
| 242 | + stroke: "var(--baseline)", "stroke-width": 1 })); | |
| 243 | + | |
| 244 | + series.forEach((s, i) => { | |
| 245 | + const c = SLOTS[i % SLOTS.length]; | |
| 246 | + const d = s.points.map((p, k) => `${k ? "L" : "M"}${x(p[0]).toFixed(1)},${y(p[1]).toFixed(1)}`).join(""); | |
| 247 | + if (area && series.length === 1) { | |
| 248 | + const a = d + `L${x(s.points.at(-1)[0])},${y(0)}L${x(s.points[0][0])},${y(0)}Z`; | |
| 249 | + svg.appendChild(svgEl("path", { d: a, fill: c, opacity: 0.12 })); | |
| 250 | + } | |
| 251 | + svg.appendChild(svgEl("path", { d, fill: "none", stroke: c, "stroke-width": 2, | |
| 252 | + "stroke-linejoin": "round", "stroke-linecap": "round" })); | |
| 253 | + }); | |
| 254 | + | |
| 255 | + // direct labels at line ends (≤4 series), nudged apart | |
| 256 | + if (series.length > 1 && series.length <= 4) { | |
| 257 | + const ends = series.map((s, i) => ({ name: s.name, i, yy: y(s.points.at(-1)[1]) })) | |
| 258 | + .sort((a, b) => a.yy - b.yy); | |
| 259 | + for (let k = 1; k < ends.length; k++) | |
| 260 | + if (ends[k].yy - ends[k - 1].yy < 12) ends[k].yy = ends[k - 1].yy + 12; | |
| 261 | + for (const e of ends) { | |
| 262 | + const t = svgEl("text", { x: W - R - 2, y: Math.min(e.yy + 3, H - B - 2), | |
| 263 | + "text-anchor": "end", "font-size": 10.5, fill: "var(--ink-2)", "font-weight": 600 }); | |
| 264 | + t.textContent = e.name; | |
| 265 | + svg.appendChild(t); | |
| 266 | + } | |
| 267 | + } | |
| 268 | + | |
| 269 | + // crosshair + tooltip | |
| 270 | + const cross = svgEl("line", { y1: T, y2: H - B, stroke: "var(--baseline)", | |
| 271 | + "stroke-width": 1, "stroke-dasharray": "3,3", visibility: "hidden" }); | |
| 272 | + svg.appendChild(cross); | |
| 273 | + const dots = series.map((_, i) => { | |
| 274 | + const d = svgEl("circle", { r: 4, fill: SLOTS[i % SLOTS.length], | |
| 275 | + stroke: "var(--surface)", "stroke-width": 2, visibility: "hidden" }); | |
| 276 | + svg.appendChild(d); | |
| 277 | + return d; | |
| 278 | + }); | |
| 279 | + const hit = svgEl("rect", { x: L, y: T, width: W - L - R, height: H - T - B, | |
| 280 | + fill: "transparent" }); | |
| 281 | + svg.appendChild(hit); | |
| 282 | + hit.addEventListener("mousemove", (e) => { | |
| 283 | + const r = svg.getBoundingClientRect(); | |
| 284 | + const tx = t0 + ((e.clientX - r.left) * (W / r.width) - L) / (W - L - R) * span; | |
| 285 | + const ti = ts.reduce((b, t) => Math.abs(t - tx) < Math.abs(b - tx) ? t : b, ts[0]); | |
| 286 | + cross.setAttribute("x1", x(ti)); cross.setAttribute("x2", x(ti)); | |
| 287 | + cross.setAttribute("visibility", "visible"); | |
| 288 | + let html = `<b>${timeLabel(ti, span)}</b>`; | |
| 289 | + series.forEach((s, i) => { | |
| 290 | + const p = s.points.find((p) => p[0] === ti); | |
| 291 | + dots[i].setAttribute("visibility", p ? "visible" : "hidden"); | |
| 292 | + if (p) { | |
| 293 | + dots[i].setAttribute("cx", x(ti)); dots[i].setAttribute("cy", y(p[1])); | |
| 294 | + html += `<span class="sw" style="background:${SLOTS[i % SLOTS.length]}"></span>${s.name}: ${fmtY(p[1])}<br>`; | |
| 295 | + } | |
| 296 | + }); | |
| 297 | + tipShow(e, html); | |
| 298 | + }); | |
| 299 | + hit.addEventListener("mouseleave", () => { | |
| 300 | + cross.setAttribute("visibility", "hidden"); | |
| 301 | + dots.forEach((d) => d.setAttribute("visibility", "hidden")); | |
| 302 | + tipHide(); | |
| 303 | + }); | |
| 304 | + | |
| 305 | + const wrap = document.createElement("div"); | |
| 306 | + wrap.className = "chart"; | |
| 307 | + wrap.appendChild(svg); | |
| 308 | + el.appendChild(wrap); | |
| 309 | + if (series.length > 1) { // legend (≥2 series) | |
| 310 | + const lg = document.createElement("div"); | |
| 311 | + lg.className = "legend"; | |
| 312 | + lg.innerHTML = series.map((s, i) => | |
| 313 | + `<span><span class="sw" style="background:${SLOTS[i % SLOTS.length]}"></span>${s.name}</span>`).join(""); | |
| 314 | + el.appendChild(lg); | |
| 315 | + } | |
| 316 | +} | |
| 317 | + | |
| 318 | +// ---------- signed column chart (net flows: diverging blue↔red) ---------- | |
| 319 | +export function columns(el, points, { fmtY = fmtUsd } = {}) { | |
| 320 | + el.innerHTML = ""; | |
| 321 | + if (!points.length) { el.innerHTML = `<div class="empty">no data yet</div>`; return; } | |
| 322 | + const W = 820, H = 250, L = 58, R = 10, T = 12, B = 26; | |
| 323 | + const vmax = Math.max(...points.map((p) => Math.abs(p.v)), 1e-9) * 1.08; | |
| 324 | + const y = (v) => T + (1 - (v + vmax) / (2 * vmax)) * (H - T - B); | |
| 325 | + const bw = Math.max(3, Math.min(40, (W - L - R) / points.length - 2)); | |
| 326 | + const x = (i) => L + (i + 0.5) * ((W - L - R) / points.length); | |
| 327 | + const svg = svgEl("svg", { viewBox: `0 0 ${W} ${H}`, role: "img" }); | |
| 328 | + for (const v of [-vmax, -vmax / 2, 0, vmax / 2, vmax]) { | |
| 329 | + svg.appendChild(svgEl("line", { x1: L, x2: W - R, y1: y(v), y2: y(v), | |
| 330 | + stroke: v === 0 ? "var(--baseline)" : "var(--grid)", "stroke-width": 1 })); | |
| 331 | + const t = svgEl("text", { x: L - 6, y: y(v) + 4, "text-anchor": "end", | |
| 332 | + "font-size": 10.5, fill: "var(--muted)" }); | |
| 333 | + t.textContent = fmtY(v); | |
| 334 | + svg.appendChild(t); | |
| 335 | + } | |
| 336 | + const span = points.at(-1).t - points[0].t || 1; | |
| 337 | + points.forEach((p, i) => { | |
| 338 | + const h = Math.max(1, Math.abs(y(p.v) - y(0))); | |
| 339 | + const rect = svgEl("rect", { | |
| 340 | + x: x(i) - bw / 2, y: p.v >= 0 ? y(p.v) : y(0), width: bw, height: h, | |
| 341 | + rx: 3, fill: p.v >= 0 ? "var(--s1)" : "var(--div-neg)", | |
| 342 | + }); | |
| 343 | + attachTip(rect, () => | |
| 344 | + `<b>${timeLabel(p.t, span)}</b>net ${fmtY(p.v)}<br>minted ${fmtY(p.minted)} · burned ${fmtY(p.burned)}`); | |
| 345 | + svg.appendChild(rect); | |
| 346 | + if (i % Math.ceil(points.length / 6) === 0) { | |
| 347 | + const t = svgEl("text", { x: x(i), y: H - 8, "text-anchor": "middle", | |
| 348 | + "font-size": 10.5, fill: "var(--muted)" }); | |
| 349 | + t.textContent = timeLabel(p.t, span); | |
| 350 | + svg.appendChild(t); | |
| 351 | + } | |
| 352 | + }); | |
| 353 | + const wrap = document.createElement("div"); | |
| 354 | + wrap.className = "chart"; | |
| 355 | + wrap.appendChild(svg); | |
| 356 | + el.appendChild(wrap); | |
| 357 | +} | |
| 358 | + | |
| 359 | +// ---------- series shaping: top-N chains + Other ---------- | |
| 360 | +export function topSeries(chainsObj, n = 3, key = "volume") { | |
| 361 | + const totals = Object.entries(chainsObj) | |
| 362 | + .map(([c, pts]) => [c, pts.reduce((a, p) => a + (p[key] ?? p.supply ?? 0), 0)]) | |
| 363 | + .sort((a, b) => b[1] - a[1]); | |
| 364 | + const top = totals.slice(0, n).map(([c]) => c); | |
| 365 | + const others = totals.slice(n).map(([c]) => c); | |
| 366 | + const series = top.map((c) => ({ | |
| 367 | + name: c, points: chainsObj[c].map((p) => [p.t, p[key] ?? p.supply ?? 0]), | |
| 368 | + })); | |
| 369 | + if (others.length) { | |
| 370 | + const acc = {}; | |
| 371 | + for (const c of others) | |
| 372 | + for (const p of chainsObj[c]) acc[p.t] = (acc[p.t] || 0) + (p[key] ?? p.supply ?? 0); | |
| 373 | + series.push({ name: "other", points: Object.entries(acc) | |
| 374 | + .map(([t, v]) => [+t, v]).sort((a, b) => a[0] - b[0]) }); | |
| 375 | + } | |
| 376 | + return series; | |
| 377 | +} | |
added
ui/assets/style.css
+282 −0
@@ -0,0 +1,282 @@ | ||
| 1 | +/* Author: Simon-Pierre Boucher */ | |
| 2 | +/* Mail: contact@spboucher.ai */ | |
| 3 | +/* coinexplorer design system v4 — color roles from the validated dataviz | |
| 4 | + reference palette; chrome, depth and rhythm layered on top. */ | |
| 5 | + | |
| 6 | +:root { | |
| 7 | + color-scheme: light; | |
| 8 | + --page: #f7f7f4; --surface: #fdfdfc; --surface-2: #f2f1ed; | |
| 9 | + --ink: #0b0b0b; --ink-2: #52514e; --muted: #898781; | |
| 10 | + --grid: #e1e0d9; --baseline: #c3c2b7; --ring: rgba(11,11,11,.08); | |
| 11 | + --s1: #2a78d6; --s2: #eb6834; --s3: #1baf7a; --s4: #eda100; | |
| 12 | + --seq-150: #b7d3f6; | |
| 13 | + --good-text: #006300; --bad: #d03b3b; --div-neg: #e34948; | |
| 14 | + --shadow-1: 0 1px 2px rgba(11,11,11,.04), 0 2px 8px rgba(11,11,11,.04); | |
| 15 | + --shadow-2: 0 2px 6px rgba(11,11,11,.06), 0 10px 28px rgba(11,11,11,.08); | |
| 16 | + --glass: rgba(253,253,252,.82); | |
| 17 | + --radius: 14px; | |
| 18 | +} | |
| 19 | +@media (prefers-color-scheme: dark) { | |
| 20 | + :root { | |
| 21 | + color-scheme: dark; | |
| 22 | + --page: #0c0c0b; --surface: #171716; --surface-2: #1f1f1d; | |
| 23 | + --ink: #ffffff; --ink-2: #c3c2b7; --muted: #898781; | |
| 24 | + --grid: #2c2c2a; --baseline: #383835; --ring: rgba(255,255,255,.09); | |
| 25 | + --s1: #3987e5; --s2: #d95926; --s3: #199e70; --s4: #c98500; | |
| 26 | + --seq-150: #184f95; | |
| 27 | + --good-text: #0ca30c; --bad: #e66767; --div-neg: #e66767; | |
| 28 | + --shadow-1: none; --shadow-2: 0 12px 32px rgba(0,0,0,.5); | |
| 29 | + --glass: rgba(20,20,19,.82); | |
| 30 | + } | |
| 31 | +} | |
| 32 | + | |
| 33 | +* { box-sizing: border-box; } | |
| 34 | +html { -webkit-text-size-adjust: 100%; } | |
| 35 | +body { | |
| 36 | + margin: 0; background: var(--page); color: var(--ink); | |
| 37 | + font: 14px/1.5 system-ui, -apple-system, "Segoe UI", sans-serif; | |
| 38 | + background-image: radial-gradient(1100px 380px at 50% -180px, | |
| 39 | + color-mix(in oklab, var(--s1) 7%, transparent), transparent); | |
| 40 | + background-repeat: no-repeat; | |
| 41 | +} | |
| 42 | +a { color: var(--s1); text-decoration: none; } | |
| 43 | +a:hover { text-decoration: underline; text-underline-offset: 2px; } | |
| 44 | +:focus-visible { outline: 2px solid var(--s1); outline-offset: 2px; border-radius: 4px; } | |
| 45 | + | |
| 46 | +/* ================= top nav ================= */ | |
| 47 | +.topnav { | |
| 48 | + position: sticky; top: 0; z-index: 40; | |
| 49 | + background: var(--glass); backdrop-filter: blur(14px) saturate(1.4); | |
| 50 | + -webkit-backdrop-filter: blur(14px) saturate(1.4); | |
| 51 | + border-bottom: 1px solid var(--ring); | |
| 52 | + display: flex; align-items: center; gap: 14px; | |
| 53 | + padding: 0 clamp(12px, 3vw, 24px); min-height: 54px; flex-wrap: wrap; | |
| 54 | +} | |
| 55 | +.topnav .logo { | |
| 56 | + font-weight: 800; font-size: 16px; letter-spacing: -.02em; | |
| 57 | + color: var(--ink); white-space: nowrap; padding: 12px 0; | |
| 58 | +} | |
| 59 | +.topnav .logo span { | |
| 60 | + background: linear-gradient(100deg, var(--s1), color-mix(in oklab, var(--s1) 55%, var(--s3))); | |
| 61 | + -webkit-background-clip: text; background-clip: text; color: transparent; | |
| 62 | +} | |
| 63 | +.topnav nav { | |
| 64 | + display: flex; gap: 2px; overflow-x: auto; scrollbar-width: none; | |
| 65 | + -webkit-overflow-scrolling: touch; max-width: 100%; | |
| 66 | +} | |
| 67 | +.topnav nav::-webkit-scrollbar { display: none; } | |
| 68 | +.topnav nav a { | |
| 69 | + color: var(--ink-2); padding: 7px 12px; border-radius: 99px; | |
| 70 | + font-size: 13px; font-weight: 500; white-space: nowrap; | |
| 71 | +} | |
| 72 | +.topnav nav a:hover { | |
| 73 | + background: color-mix(in oklab, var(--s1) 9%, transparent); | |
| 74 | + text-decoration: none; color: var(--ink); | |
| 75 | +} | |
| 76 | +.topnav nav a.active { | |
| 77 | + color: #fff; font-weight: 600; | |
| 78 | + background: linear-gradient(135deg, var(--s1), color-mix(in oklab, var(--s1) 80%, #000)); | |
| 79 | + box-shadow: 0 2px 8px color-mix(in oklab, var(--s1) 35%, transparent); | |
| 80 | +} | |
| 81 | +.searchbox { margin-left: auto; display: flex; min-width: 150px; flex: 0 1 320px; padding: 8px 0; } | |
| 82 | +.searchbox input { | |
| 83 | + width: 100%; background: var(--surface-2); color: var(--ink); | |
| 84 | + border: 1px solid var(--ring); border-radius: 99px; padding: 8px 16px; font: inherit; | |
| 85 | + transition: box-shadow .15s, background .15s; | |
| 86 | +} | |
| 87 | +.searchbox input:focus { | |
| 88 | + outline: none; background: var(--surface); | |
| 89 | + box-shadow: 0 0 0 2px color-mix(in oklab, var(--s1) 55%, transparent); | |
| 90 | +} | |
| 91 | +@media (max-width: 760px) { | |
| 92 | + .searchbox { order: 9; flex-basis: 100%; margin-left: 0; padding-top: 0; } | |
| 93 | + .topnav .logo { padding: 10px 0 6px; } | |
| 94 | +} | |
| 95 | + | |
| 96 | +/* ================= layout ================= */ | |
| 97 | +main { | |
| 98 | + display: grid; gap: clamp(10px, 1.6vw, 16px); | |
| 99 | + padding: clamp(12px, 2.5vw, 20px) clamp(12px, 3vw, 24px) 48px; | |
| 100 | + grid-template-columns: repeat(12, 1fr); max-width: 1320px; margin: 0 auto; | |
| 101 | +} | |
| 102 | +.card { | |
| 103 | + background: var(--surface); border: 1px solid var(--ring); | |
| 104 | + border-radius: var(--radius); padding: clamp(12px, 2vw, 18px); | |
| 105 | + grid-column: span 12; box-shadow: var(--shadow-1); min-width: 0; | |
| 106 | +} | |
| 107 | +@media (min-width: 980px) { | |
| 108 | + .tile { grid-column: span 3; } | |
| 109 | + .half { grid-column: span 6; } | |
| 110 | + .third { grid-column: span 4; } | |
| 111 | + .twothird { grid-column: span 8; } | |
| 112 | +} | |
| 113 | +@media (min-width: 560px) and (max-width: 979px) { | |
| 114 | + .tile { grid-column: span 6; } | |
| 115 | + .half, .third, .twothird { grid-column: span 12; } | |
| 116 | +} | |
| 117 | +.card h2 { | |
| 118 | + font-size: 12px; margin: 0 0 10px; color: var(--muted); font-weight: 700; | |
| 119 | + text-transform: uppercase; letter-spacing: .06em; | |
| 120 | + display: flex; align-items: baseline; gap: 8px; flex-wrap: wrap; | |
| 121 | +} | |
| 122 | +.card h2 .sub { color: var(--muted); font-weight: 400; font-size: 12px; text-transform: none; letter-spacing: 0; } | |
| 123 | +.tile { position: relative; overflow: hidden; } | |
| 124 | +.tile::before { | |
| 125 | + content: ""; position: absolute; inset: 0 auto 0 0; width: 3px; | |
| 126 | + background: linear-gradient(180deg, var(--s1), color-mix(in oklab, var(--s1) 40%, transparent)); | |
| 127 | + border-radius: 3px 0 0 3px; | |
| 128 | +} | |
| 129 | +.tile .big { | |
| 130 | + font-size: clamp(22px, 2.4vw, 28px); font-weight: 800; letter-spacing: -.02em; | |
| 131 | + font-variant-numeric: tabular-nums; | |
| 132 | +} | |
| 133 | +.tile .note { color: var(--muted); font-size: 12px; margin-top: 3px; } | |
| 134 | +.up { color: var(--good-text); } .down { color: var(--bad); } | |
| 135 | + | |
| 136 | +.pagehead { grid-column: span 12; display: flex; align-items: baseline; gap: 12px; flex-wrap: wrap; } | |
| 137 | +.pagehead h1 { font-size: clamp(19px, 2.6vw, 23px); margin: 6px 0 0; letter-spacing: -.02em; } | |
| 138 | +.pagehead .sub { color: var(--muted); font-size: 13px; } | |
| 139 | +.badge { | |
| 140 | + font-size: 10.5px; font-weight: 600; padding: 3px 9px; border-radius: 99px; | |
| 141 | + border: 1px solid var(--ring); color: var(--ink-2); | |
| 142 | + text-transform: uppercase; letter-spacing: .04em; | |
| 143 | +} | |
| 144 | +.badge.native { color: var(--good-text); border-color: color-mix(in oklab, var(--good-text) 45%, transparent); background: color-mix(in oklab, var(--good-text) 7%, transparent); } | |
| 145 | +.badge.bridged { color: var(--s2); border-color: color-mix(in oklab, var(--s2) 45%, transparent); background: color-mix(in oklab, var(--s2) 8%, transparent); } | |
| 146 | +.badge.discontinued { color: var(--bad); border-color: color-mix(in oklab, var(--bad) 45%, transparent); background: color-mix(in oklab, var(--bad) 7%, transparent); } | |
| 147 | + | |
| 148 | +/* ================= filters ================= */ | |
| 149 | +.filters { grid-column: span 12; display: flex; gap: 8px; flex-wrap: wrap; align-items: center; } | |
| 150 | +.filters select, .filters input, .filters button, .btn { | |
| 151 | + background: var(--surface); color: var(--ink); border: 1px solid var(--ring); | |
| 152 | + border-radius: 10px; padding: 8px 13px; font: inherit; font-size: 13px; | |
| 153 | + cursor: pointer; box-shadow: var(--shadow-1); min-height: 38px; | |
| 154 | +} | |
| 155 | +.filters input { cursor: text; } | |
| 156 | +.filters button:hover, .btn:hover { border-color: color-mix(in oklab, var(--s1) 45%, var(--ring)); } | |
| 157 | +.presets { display: flex; gap: 0; border: 1px solid var(--ring); border-radius: 10px; overflow: hidden; box-shadow: var(--shadow-1); } | |
| 158 | +.presets button { border: 0; border-radius: 0; box-shadow: none; background: var(--surface); min-height: 38px; } | |
| 159 | +.presets button + button { border-left: 1px solid var(--ring); } | |
| 160 | +.presets button[aria-pressed="true"] { background: var(--s1); color: #fff; font-weight: 700; } | |
| 161 | + | |
| 162 | +/* ================= tables ================= */ | |
| 163 | +.scroll { max-height: 420px; overflow: auto; border-radius: 8px; } | |
| 164 | +table { width: 100%; border-collapse: collapse; font-size: 12.5px; min-width: 520px; } | |
| 165 | +th { | |
| 166 | + text-align: left; color: var(--muted); font-weight: 700; padding: 7px 9px; | |
| 167 | + border-bottom: 1px solid var(--grid); position: sticky; top: 0; z-index: 1; | |
| 168 | + background: var(--surface); font-size: 11px; text-transform: uppercase; letter-spacing: .05em; | |
| 169 | +} | |
| 170 | +td { padding: 7px 9px; border-bottom: 1px solid color-mix(in oklab, var(--grid) 55%, transparent); | |
| 171 | + font-variant-numeric: tabular-nums; white-space: nowrap; } | |
| 172 | +tr:hover td { background: color-mix(in oklab, var(--s1) 5%, transparent); } | |
| 173 | +td.num, th.num { text-align: right; } | |
| 174 | +td.mono, .mono { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 11.5px; | |
| 175 | + color: var(--ink-2); max-width: 150px; overflow: hidden; text-overflow: ellipsis; } | |
| 176 | +.dot { display: inline-block; width: 7px; height: 7px; border-radius: 50%; | |
| 177 | + background: var(--s1); margin-right: 7px; box-shadow: 0 0 0 3px color-mix(in oklab, var(--s1) 15%, transparent); } | |
| 178 | +.mint { color: var(--good-text); font-weight: 700; } | |
| 179 | +.burn { color: var(--bad); font-weight: 700; } | |
| 180 | +.feed-wrap { max-height: 320px; overflow: auto; } | |
| 181 | + | |
| 182 | +/* ================= bars (magnitude) ================= */ | |
| 183 | +.bars { display: flex; flex-direction: column; gap: 2px; } | |
| 184 | +.bar-row { | |
| 185 | + display: grid; grid-template-columns: minmax(72px, 100px) 1fr minmax(72px, 96px); | |
| 186 | + align-items: center; gap: 8px; padding: 4px 6px; border-radius: 8px; min-height: 24px; | |
| 187 | +} | |
| 188 | +.bar-row:hover { background: color-mix(in oklab, var(--s1) 8%, transparent); } | |
| 189 | +.bar-label { color: var(--ink-2); font-size: 12.5px; font-weight: 500; white-space: nowrap; | |
| 190 | + overflow: hidden; text-overflow: ellipsis; } | |
| 191 | +.bar-label a { color: inherit; } | |
| 192 | +.bar-track { height: 13px; border-radius: 0 5px 5px 0; | |
| 193 | + border-left: 2px solid var(--baseline); position: relative; | |
| 194 | + background: color-mix(in oklab, var(--grid) 30%, transparent); } | |
| 195 | +.bar-fill { position: absolute; inset: 0 auto 0 0; | |
| 196 | + background: linear-gradient(90deg, color-mix(in oklab, var(--s1) 88%, #fff0), var(--s1)); | |
| 197 | + border-radius: 0 5px 5px 0; min-width: 2px; } | |
| 198 | +.bar-val { text-align: right; font-variant-numeric: tabular-nums; font-size: 12.5px; font-weight: 600; } | |
| 199 | + | |
| 200 | +/* ================= charts ================= */ | |
| 201 | +.chart { width: 100%; } | |
| 202 | +.chart svg { display: block; width: 100%; height: auto; } | |
| 203 | +.legend { display: flex; gap: 14px; flex-wrap: wrap; margin-top: 8px; font-size: 12px; color: var(--ink-2); } | |
| 204 | +.legend .sw { display: inline-block; width: 10px; height: 10px; border-radius: 3px; | |
| 205 | + margin-right: 5px; vertical-align: -1px; } | |
| 206 | +#tooltip { | |
| 207 | + position: fixed; pointer-events: none; background: var(--surface); | |
| 208 | + border: 1px solid var(--ring); border-radius: 10px; padding: 8px 11px; | |
| 209 | + font-size: 12px; box-shadow: var(--shadow-2); display: none; z-index: 60; | |
| 210 | + font-variant-numeric: tabular-nums; max-width: 260px; | |
| 211 | +} | |
| 212 | +#tooltip b { display: block; margin-bottom: 3px; } | |
| 213 | +.empty { color: var(--muted); font-size: 12.5px; padding: 14px 4px; } | |
| 214 | + | |
| 215 | +/* ================= api docs ================= */ | |
| 216 | +.docwrap { display: grid; grid-template-columns: 220px 1fr; gap: 18px; grid-column: span 12; align-items: start; } | |
| 217 | +@media (max-width: 900px) { .docwrap { grid-template-columns: 1fr; } } | |
| 218 | +.toc { position: sticky; top: 66px; display: flex; flex-direction: column; gap: 2px; | |
| 219 | + background: var(--surface); border: 1px solid var(--ring); border-radius: var(--radius); | |
| 220 | + padding: 10px; box-shadow: var(--shadow-1); } | |
| 221 | +@media (max-width: 900px) { .toc { position: static; flex-direction: row; flex-wrap: wrap; } } | |
| 222 | +.toc a { color: var(--ink-2); font-size: 12.5px; padding: 6px 10px; border-radius: 8px; } | |
| 223 | +.toc a:hover { background: color-mix(in oklab, var(--s1) 8%, transparent); text-decoration: none; } | |
| 224 | +.toc .group { font-size: 10.5px; text-transform: uppercase; letter-spacing: .07em; | |
| 225 | + color: var(--muted); font-weight: 700; padding: 10px 10px 3px; } | |
| 226 | +.docbody { display: flex; flex-direction: column; gap: 14px; min-width: 0; } | |
| 227 | +.ep { background: var(--surface); border: 1px solid var(--ring); border-radius: var(--radius); | |
| 228 | + padding: 16px 18px; box-shadow: var(--shadow-1); scroll-margin-top: 70px; } | |
| 229 | +.ep .head { display: flex; align-items: center; gap: 10px; flex-wrap: wrap; } | |
| 230 | +.method { font-size: 11px; font-weight: 800; letter-spacing: .05em; padding: 4px 9px; | |
| 231 | + border-radius: 7px; color: #fff; background: var(--s1); } | |
| 232 | +.method.ws { background: var(--s3); } | |
| 233 | +.ep .path { font-family: ui-monospace, Menlo, monospace; font-size: 13.5px; font-weight: 600; | |
| 234 | + word-break: break-all; } | |
| 235 | +.ep .desc { color: var(--ink-2); font-size: 13px; margin: 8px 0 0; } | |
| 236 | +.ep .params { margin-top: 10px; } | |
| 237 | +.ep .params table { min-width: 0; } | |
| 238 | +pre.code { | |
| 239 | + background: var(--surface-2); border: 1px solid var(--ring); border-radius: 10px; | |
| 240 | + padding: 12px 14px; font-family: ui-monospace, Menlo, monospace; font-size: 12px; | |
| 241 | + overflow-x: auto; margin: 10px 0 0; line-height: 1.55; position: relative; | |
| 242 | +} | |
| 243 | +pre.code .cm { color: var(--muted); } | |
| 244 | +.tryrow { display: flex; gap: 8px; margin-top: 10px; flex-wrap: wrap; align-items: center; } | |
| 245 | +.try-out { display: none; max-height: 280px; overflow: auto; } | |
| 246 | +.copybtn { margin-left: auto; font-size: 11px; padding: 4px 10px; min-height: 0; } | |
| 247 | +.hero { grid-column: span 12; padding: clamp(18px, 4vw, 34px) clamp(16px, 3vw, 28px); | |
| 248 | + border-radius: 18px; border: 1px solid var(--ring); | |
| 249 | + background: linear-gradient(140deg, color-mix(in oklab, var(--s1) 14%, var(--surface)), | |
| 250 | + var(--surface) 55%, color-mix(in oklab, var(--s3) 8%, var(--surface))); | |
| 251 | + box-shadow: var(--shadow-1); } | |
| 252 | +.hero h1 { margin: 0 0 6px; font-size: clamp(21px, 3.4vw, 30px); letter-spacing: -.02em; } | |
| 253 | +.hero p { margin: 0; color: var(--ink-2); max-width: 62ch; } | |
| 254 | +.hero .kpis { display: flex; gap: 18px; margin-top: 14px; flex-wrap: wrap; } | |
| 255 | +.hero .kpis b { font-size: 17px; font-variant-numeric: tabular-nums; } | |
| 256 | +.hero .kpis span { display: block; color: var(--muted); font-size: 11.5px; text-transform: uppercase; letter-spacing: .05em; } | |
| 257 | + | |
| 258 | +/* ================= asset icons ================= */ | |
| 259 | +.asset { display: inline-flex; align-items: center; gap: 6px; color: inherit; } | |
| 260 | +.asset span { line-height: 1; } | |
| 261 | +.asset:hover { text-decoration: none; } | |
| 262 | +.asset:hover span { text-decoration: underline; text-underline-offset: 2px; } | |
| 263 | +td .asset { color: var(--s1); } | |
| 264 | +.coin { | |
| 265 | + position: relative; display: inline-grid; place-items: center; | |
| 266 | + width: var(--sz, 16px); height: var(--sz, 16px); flex: 0 0 auto; | |
| 267 | + vertical-align: -3px; border-radius: 50%; | |
| 268 | +} | |
| 269 | +.coin i { | |
| 270 | + position: absolute; inset: 0; border-radius: 50%; font-style: normal; | |
| 271 | + display: grid; place-items: center; | |
| 272 | + font-size: calc(var(--sz, 16px) * 0.52); font-weight: 800; | |
| 273 | + box-shadow: inset 0 0 0 1px rgba(255,255,255,.18), 0 1px 3px rgba(0,0,0,.18); | |
| 274 | +} | |
| 275 | +.coin img { | |
| 276 | + position: absolute; inset: 0; width: 100%; height: 100%; | |
| 277 | + border-radius: 50%; z-index: 1; background: transparent; | |
| 278 | +} | |
| 279 | +.pagehead .coin { vertical-align: -5px; margin-right: 2px; } | |
| 280 | +.bar-label .asset { max-width: 100%; overflow: hidden; } | |
| 281 | + | |
| 282 | +footer { text-align: center; color: var(--muted); font-size: 12px; padding: 0 16px 32px; } | |
added
ui/chain.html
+122 −0
@@ -0,0 +1,122 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>chain — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1 id="title">Chain</h1><span class="sub" id="metaNote"></span></div> | |
| 13 | + <div class="filters"><select id="chainSel" aria-label="Chain"></select></div> | |
| 14 | + | |
| 15 | + <section class="card tile"><h2>Transfers indexed</h2> | |
| 16 | + <div class="big" id="tCount">—</div><div class="note">stablecoin transfers stored</div></section> | |
| 17 | + <section class="card tile"><h2>Volume · 24h</h2> | |
| 18 | + <div class="big" id="tVol">—</div><div class="note" id="tVolN"></div></section> | |
| 19 | + <section class="card tile"><h2>Indexer lag</h2> | |
| 20 | + <div class="big" id="tLag">—</div><div class="note" id="tLagN"></div></section> | |
| 21 | + <section class="card tile"><h2>Coverage floor</h2> | |
| 22 | + <div class="big" id="tFloor">—</div><div class="note">backfilled down to this block</div></section> | |
| 23 | + | |
| 24 | + <section class="card half"><h2>24h volume by token</h2><div class="bars" id="tokenBars"></div></section> | |
| 25 | + <section class="card half"><h2>Assets on this chain</h2> | |
| 26 | + <div class="scroll"><table> | |
| 27 | + <thead><tr><th>token</th><th>identifier</th><th>issuance</th><th class="num">decimals</th></tr></thead> | |
| 28 | + <tbody id="tokens"></tbody></table></div></section> | |
| 29 | + | |
| 30 | + <section class="card"><h2>Recent transfers</h2> | |
| 31 | + <div class="filters" style="padding:0 0 8px"> | |
| 32 | + <select id="symFilter"><option value="">all tokens</option></select> | |
| 33 | + <input id="minUsd" type="number" placeholder="min USD" style="width:110px"> | |
| 34 | + <button class="btn" id="apply">Apply</button> | |
| 35 | + </div> | |
| 36 | + <div class="scroll"><table> | |
| 37 | + <thead><tr><th>block</th><th>token</th><th class="num">amount</th><th>from</th><th>to</th><th>tx</th><th>when</th></tr></thead> | |
| 38 | + <tbody id="transfers"></tbody></table></div> | |
| 39 | + <div style="margin-top:8px"><button class="btn" id="more">Load more</button></div></section> | |
| 40 | + | |
| 41 | + <section class="card"><h2>RPC endpoints</h2> | |
| 42 | + <div class="scroll"><table> | |
| 43 | + <thead><tr><th>endpoint</th><th class="num">score</th><th class="num">ok</th><th class="num">fail</th><th class="num">latency</th><th class="num">cooldown</th></tr></thead> | |
| 44 | + <tbody id="rpc"></tbody></table></div></section> | |
| 45 | +</main> | |
| 46 | + | |
| 47 | +<script type="module"> | |
| 48 | +import { $, qs, j, fmtUsd, fmtN, fmtAmt, mountNav, hbars, addrLink, txLink, tokenLink, chainIcon, ago } from "/assets/app.js"; | |
| 49 | +mountNav("Chains"); | |
| 50 | +let CHAIN = qs.get("chain") || "ethereum"; | |
| 51 | +let beforeBlock = null; | |
| 52 | + | |
| 53 | +async function refresh() { | |
| 54 | + document.title = `${CHAIN} — coinexplorer`; | |
| 55 | + const [sum, status, health] = await Promise.all([ | |
| 56 | + j(`/v1/${CHAIN}/summary`), j("/v1/status"), j("/v1/rpc/health"), | |
| 57 | + ]); | |
| 58 | + $("title").innerHTML = `${chainIcon(CHAIN, 26)} ${CHAIN}`; | |
| 59 | + $("metaNote").textContent = `${sum.family}${sum.chain_id ? " · chain id " + sum.chain_id : ""} · ${sum.block_time_s}s blocks`; | |
| 60 | + $("tCount").textContent = fmtN(sum.indexed_transfers); | |
| 61 | + const vol24 = sum.volume_24h.reduce((a, r) => a + r.volume, 0); | |
| 62 | + $("tVol").textContent = fmtUsd(vol24); | |
| 63 | + $("tVolN").textContent = `${fmtN(sum.volume_24h.reduce((a, r) => a + r.transfers, 0))} transfers`; | |
| 64 | + const cur = status.cursors[CHAIN] || {}; | |
| 65 | + $("tLag").textContent = cur.lag != null ? fmtN(cur.lag) + " blocks" : "—"; | |
| 66 | + $("tLagN").textContent = cur.last_block ? `at block ${fmtN(cur.last_block)}` : "not indexing"; | |
| 67 | + $("tFloor").textContent = cur.backfill_block != null ? fmtN(cur.backfill_block) : "—"; | |
| 68 | + | |
| 69 | + hbars($("tokenBars"), sum.volume_24h.map((r) => | |
| 70 | + ({ k: r.symbol, v: r.volume, link: tokenLink(r.symbol), | |
| 71 | + extra: `<br>${fmtN(r.transfers)} transfers` }))); | |
| 72 | + | |
| 73 | + $("tokens").innerHTML = sum.tokens.map((t) => { | |
| 74 | + const id = t.address || t.id; | |
| 75 | + const cls = t.discontinued ? "discontinued" : t.native ? "native" : "bridged"; | |
| 76 | + return `<tr><td>${tokenLink(t.symbol)}</td><td class="mono" title="${id}">${id}</td> | |
| 77 | + <td><span class="badge ${cls}">${cls}</span></td><td class="num">${t.decimals ?? "—"}</td></tr>`; | |
| 78 | + }).join(""); | |
| 79 | + | |
| 80 | + $("symFilter").innerHTML = `<option value="">all tokens</option>` + | |
| 81 | + [...new Set(sum.tokens.map((t) => t.symbol))].sort().map((s) => `<option>${s}</option>`).join(""); | |
| 82 | + | |
| 83 | + $("rpc").innerHTML = (health[CHAIN] || []).map((e) => | |
| 84 | + `<tr><td class="mono" style="max-width:none" title="${e.url}">${e.url}</td> | |
| 85 | + <td class="num">${e.score}</td><td class="num">${fmtN(e.ok)}</td><td class="num">${fmtN(e.fail)}</td> | |
| 86 | + <td class="num">${e.latency_ms} ms</td><td class="num">${e.cooldown_s > 0 ? e.cooldown_s + "s" : "—"}</td></tr>`).join("") || | |
| 87 | + `<tr><td colspan="6" class="empty">no health data yet</td></tr>`; | |
| 88 | +} | |
| 89 | + | |
| 90 | +async function loadTransfers(append = false) { | |
| 91 | + const sym = $("symFilter").value, min = $("minUsd").value; | |
| 92 | + let url = `/v1/${CHAIN}/transfers?limit=50`; | |
| 93 | + if (sym) url += `&symbol=${sym}`; | |
| 94 | + if (min) url += `&min_amount=${min}`; | |
| 95 | + if (append && beforeBlock) url += `&before_block=${beforeBlock}`; | |
| 96 | + const rows = await j(url); | |
| 97 | + if (rows.length) beforeBlock = rows[rows.length - 1].block; | |
| 98 | + const html = rows.map((r) => | |
| 99 | + `<tr><td>${fmtN(r.block)}</td><td>${tokenLink(r.symbol)}</td> | |
| 100 | + <td class="num">${fmtAmt(r)}</td><td>${addrLink(r.from)}</td> | |
| 101 | + <td>${addrLink(r.to)}</td><td>${txLink(r.chain, r.tx_hash)}</td> | |
| 102 | + <td>${ago(r.timestamp)} ago</td></tr>`).join(""); | |
| 103 | + if (append) $("transfers").insertAdjacentHTML("beforeend", html); | |
| 104 | + else $("transfers").innerHTML = html || `<tr><td colspan="7" class="empty">nothing indexed yet</td></tr>`; | |
| 105 | +} | |
| 106 | + | |
| 107 | +j("/v1/chains").then((chains) => { | |
| 108 | + $("chainSel").innerHTML = Object.keys(chains).sort() | |
| 109 | + .map((c) => `<option ${c === CHAIN ? "selected" : ""}>${c}</option>`).join(""); | |
| 110 | +}); | |
| 111 | +$("chainSel").addEventListener("change", (e) => { | |
| 112 | + CHAIN = e.target.value; beforeBlock = null; | |
| 113 | + history.replaceState(null, "", `?chain=${CHAIN}`); | |
| 114 | + refresh(); loadTransfers(); | |
| 115 | +}); | |
| 116 | +$("apply").addEventListener("click", () => { beforeBlock = null; loadTransfers(); }); | |
| 117 | +$("more").addEventListener("click", () => loadTransfers(true)); | |
| 118 | +refresh(); loadTransfers(); | |
| 119 | +setInterval(refresh, 30_000); | |
| 120 | +</script> | |
| 121 | +</body> | |
| 122 | +</html> | |
added
ui/flows.html
+84 −0
@@ -0,0 +1,84 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>flows — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1>Issuance flows</h1> | |
| 13 | + <span class="sub">mints and burns detected on-chain (zero-address transfers, EVM + Tron)</span></div> | |
| 14 | + | |
| 15 | + <div class="filters"> | |
| 16 | + <select id="tokenSel"></select> | |
| 17 | + <div class="presets" role="group" aria-label="Window"> | |
| 18 | + <button data-w="24h">24h</button><button data-w="7d" aria-pressed="true">7d</button> | |
| 19 | + </div> | |
| 20 | + </div> | |
| 21 | + | |
| 22 | + <section class="card tile"><h2>Minted <span class="sub" id="w1"></span></h2> | |
| 23 | + <div class="big up" id="tMint">—</div></section> | |
| 24 | + <section class="card tile"><h2>Burned <span class="sub" id="w2"></span></h2> | |
| 25 | + <div class="big down" id="tBurn">—</div></section> | |
| 26 | + <section class="card tile"><h2>Net issuance <span class="sub" id="w3"></span></h2> | |
| 27 | + <div class="big" id="tNet">—</div></section> | |
| 28 | + <section class="card tile"><h2>Events <span class="sub" id="w4"></span></h2> | |
| 29 | + <div class="big" id="tEvents">—</div><div class="note">mint/burn transactions listed</div></section> | |
| 30 | + | |
| 31 | + <section class="card"><h2 id="chartTitle">Net issuance over time</h2><div id="netChart"></div> | |
| 32 | + <div class="legend"><span><span class="sw" style="background:var(--s1)"></span>net positive (minted)</span> | |
| 33 | + <span><span class="sw" style="background:var(--div-neg)"></span>net negative (burned)</span></div></section> | |
| 34 | + | |
| 35 | + <section class="card"><h2>Events</h2> | |
| 36 | + <div class="scroll" style="max-height:60vh"><table> | |
| 37 | + <thead><tr><th></th><th>chain</th><th>token</th><th class="num">amount</th><th>counterparty</th><th>tx</th><th>when</th></tr></thead> | |
| 38 | + <tbody id="rows"></tbody></table></div></section> | |
| 39 | +</main> | |
| 40 | + | |
| 41 | +<script type="module"> | |
| 42 | +import { $, j, fmtUsd, fmtN, fmtAmt, mountNav, columns, addrLink, txLink, chainLink, tokenLink, ago } from "/assets/app.js"; | |
| 43 | +mountNav("Flows"); | |
| 44 | +let TOKEN = "USDC", WINDOW = "7d"; | |
| 45 | +const ivl = () => ({ "24h": "1h", "7d": "6h" }[WINDOW]); | |
| 46 | + | |
| 47 | +async function load() { | |
| 48 | + const [flows, events] = await Promise.all([ | |
| 49 | + j(`/v1/stablecoins/flows?token=${TOKEN}&window=${WINDOW}&interval=${ivl()}`), | |
| 50 | + j(`/v1/stablecoins/mints-burns?token=${TOKEN}&window=${WINDOW}&limit=100`), | |
| 51 | + ]); | |
| 52 | + ["w1", "w2", "w3", "w4"].forEach((id) => $(id).textContent = "· " + WINDOW); | |
| 53 | + const pts = (flows.tokens[TOKEN] || []).map((b) => ({ t: b.t, v: b.net, minted: b.minted, burned: b.burned })); | |
| 54 | + const minted = pts.reduce((a, b) => a + b.minted, 0), burned = pts.reduce((a, b) => a + b.burned, 0); | |
| 55 | + $("tMint").textContent = "+" + fmtUsd(minted); | |
| 56 | + $("tBurn").textContent = "−" + fmtUsd(burned); | |
| 57 | + const net = minted - burned; | |
| 58 | + $("tNet").textContent = (net >= 0 ? "+" : "") + fmtUsd(net); | |
| 59 | + $("tNet").className = "big " + (net >= 0 ? "up" : "down"); | |
| 60 | + $("tEvents").textContent = fmtN(events.length); | |
| 61 | + $("chartTitle").textContent = `${TOKEN} net issuance — ${WINDOW} (${ivl()} buckets)`; | |
| 62 | + columns($("netChart"), pts); | |
| 63 | + $("rows").innerHTML = events.map((m) => | |
| 64 | + `<tr><td class="${m.direction}">${m.direction === "mint" ? "▲ mint" : "▼ burn"}</td> | |
| 65 | + <td>${chainLink(m.chain)}</td><td>${tokenLink(m.symbol)}</td> | |
| 66 | + <td class="num">${fmtAmt(m)}</td> | |
| 67 | + <td>${addrLink(m.direction === "mint" ? m.to : m.from)}</td> | |
| 68 | + <td>${txLink(m.chain, m.tx_hash)}</td><td>${ago(m.timestamp)} ago</td></tr>`).join("") || | |
| 69 | + `<tr><td colspan="7" class="empty">none in window</td></tr>`; | |
| 70 | +} | |
| 71 | + | |
| 72 | +j("/v1/tokens").then((t) => { | |
| 73 | + $("tokenSel").innerHTML = Object.keys(t).sort() | |
| 74 | + .map((s) => `<option ${s === TOKEN ? "selected" : ""}>${s}</option>`).join(""); | |
| 75 | +}); | |
| 76 | +$("tokenSel").addEventListener("change", (e) => { TOKEN = e.target.value; load(); }); | |
| 77 | +document.querySelectorAll(".presets button").forEach((b) => b.addEventListener("click", () => { | |
| 78 | + document.querySelectorAll(".presets button").forEach((x) => x.setAttribute("aria-pressed", "false")); | |
| 79 | + b.setAttribute("aria-pressed", "true"); WINDOW = b.dataset.w; load(); | |
| 80 | +})); | |
| 81 | +load(); | |
| 82 | +</script> | |
| 83 | +</body> | |
| 84 | +</html> | |
added
ui/index.html
+125 −0
@@ -0,0 +1,125 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>coinexplorer — stablecoins & major crypto</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1>Market overview</h1> | |
| 13 | + <span class="sub">on-chain data, indexed from free public RPCs</span></div> | |
| 14 | + | |
| 15 | + <div class="filters"> | |
| 16 | + <select id="tokenSel" aria-label="Token"></select> | |
| 17 | + <div class="presets" role="group" aria-label="Window"> | |
| 18 | + <button data-w="1h">1h</button><button data-w="24h" aria-pressed="true">24h</button><button data-w="7d">7d</button> | |
| 19 | + </div> | |
| 20 | + </div> | |
| 21 | + | |
| 22 | + <section class="card tile"><h2>Tracked supply</h2> | |
| 23 | + <div class="big" id="tSupply">—</div><div class="note" id="tSupplyN">all stablecoins, all chains</div></section> | |
| 24 | + <section class="card tile"><h2>Volume <span class="sub" id="tVolW"></span></h2> | |
| 25 | + <div class="big" id="tVol">—</div><div class="note" id="tVolN"></div></section> | |
| 26 | + <section class="card tile"><h2>Transfers indexed</h2> | |
| 27 | + <div class="big" id="tCount">—</div><div class="note">all tokens · all chains</div></section> | |
| 28 | + <section class="card tile"><h2>Chains live</h2> | |
| 29 | + <div class="big" id="tChains">—</div><div class="note" id="tChainsN"></div></section> | |
| 30 | + | |
| 31 | + <section class="card twothird"><h2 id="volTitle">Volume over time</h2><div id="volChart"></div></section> | |
| 32 | + <section class="card third"><h2>Assets by tracked supply (USD)</h2><div class="bars" id="supplyBars"></div></section> | |
| 33 | + | |
| 34 | + <section class="card half"><h2 id="chainVolTitle">Volume by chain</h2><div class="bars" id="chainBars"></div></section> | |
| 35 | + <section class="card half"><h2>Live transfers <span class="sub" id="wsState">connecting…</span></h2> | |
| 36 | + <div class="scroll"><table> | |
| 37 | + <thead><tr><th>chain</th><th>token</th><th class="num">amount</th><th>from → to</th><th>tx</th></tr></thead> | |
| 38 | + <tbody id="feed"></tbody></table></div></section> | |
| 39 | + | |
| 40 | + <section class="card"><h2>Largest transfers <span class="sub" id="whaleW"></span></h2> | |
| 41 | + <div class="scroll"><table> | |
| 42 | + <thead><tr><th>chain</th><th>token</th><th class="num">USD</th><th>from</th><th>to</th><th>tx</th><th>when</th></tr></thead> | |
| 43 | + <tbody id="whales"></tbody></table></div></section> | |
| 44 | +</main> | |
| 45 | + | |
| 46 | +<script type="module"> | |
| 47 | +import { $, j, fmtUsd, fmtN, fmtAmt, mountNav, hbars, lineChart, topSeries, | |
| 48 | + addrLink, txLink, chainLink, tokenLink, ago } from "/assets/app.js"; | |
| 49 | +mountNav("Overview"); | |
| 50 | +let TOKEN = "USDT", WINDOW = "24h"; | |
| 51 | +const ivl = () => ({ "1h": "5m", "24h": "1h", "7d": "6h" }[WINDOW]); | |
| 52 | + | |
| 53 | +async function refresh() { | |
| 54 | + const [supplyAll, vol, status, series] = await Promise.all([ | |
| 55 | + j("/v1/stablecoins/supply"), | |
| 56 | + j(`/v1/stablecoins/volume?token=${TOKEN}&window=${WINDOW}`), | |
| 57 | + j("/v1/status"), | |
| 58 | + j(`/v1/stablecoins/volume/series?token=${TOKEN}&window=${WINDOW}&interval=${ivl()}`), | |
| 59 | + ]); | |
| 60 | + const totalSupply = Object.values(supplyAll).reduce((a, s) => a + (s.total_usd ?? s.total), 0); | |
| 61 | + $("tSupply").textContent = fmtUsd(totalSupply); | |
| 62 | + $("tVol").textContent = fmtUsd(vol.total_volume); | |
| 63 | + $("tVolW").textContent = "· " + WINDOW; | |
| 64 | + $("tVolN").textContent = `${TOKEN} · ${fmtN(Object.values(vol.chains).reduce((a, c) => a + c.transfers, 0))} transfers`; | |
| 65 | + $("tCount").textContent = fmtN(Object.values(status.indexed_transfers).reduce((a, b) => a + b, 0)); | |
| 66 | + $("tChains").textContent = Object.keys(status.cursors).length; | |
| 67 | + const lagging = Object.values(status.cursors).filter((c) => c.lag > 100).length; | |
| 68 | + $("tChainsN").textContent = lagging ? `${lagging} catching up` : "all at head"; | |
| 69 | + | |
| 70 | + $("volTitle").textContent = `${TOKEN} volume — ${WINDOW}`; | |
| 71 | + lineChart($("volChart"), topSeries(series.chains, 3)); | |
| 72 | + | |
| 73 | + hbars($("supplyBars"), Object.entries(supplyAll) | |
| 74 | + .map(([k, s]) => ({ k, v: s.total_usd ?? s.total, link: tokenLink(k) })) | |
| 75 | + .sort((a, b) => b.v - a.v).slice(0, 11)); | |
| 76 | + | |
| 77 | + $("chainVolTitle").textContent = `${TOKEN} volume by chain — ${WINDOW}`; | |
| 78 | + hbars($("chainBars"), Object.entries(vol.chains) | |
| 79 | + .map(([k, c]) => ({ k, v: c.volume, link: chainLink(k), | |
| 80 | + extra: `<br>${fmtN(c.transfers)} transfers` })) | |
| 81 | + .sort((a, b) => b.v - a.v).slice(0, 12)); | |
| 82 | +} | |
| 83 | + | |
| 84 | +async function refreshWhales() { | |
| 85 | + const w = await j(`/v1/stablecoins/whales?window=${WINDOW}&limit=30`); | |
| 86 | + $("whaleW").textContent = "· " + WINDOW; | |
| 87 | + $("whales").innerHTML = w.map((x) => | |
| 88 | + `<tr><td>${chainLink(x.chain)}</td><td>${tokenLink(x.symbol)}</td> | |
| 89 | + <td class="num"><b>${fmtUsd(x.usd)}</b></td><td>${addrLink(x.from)}</td> | |
| 90 | + <td>${addrLink(x.to)}</td><td>${txLink(x.chain, x.tx_hash)}</td> | |
| 91 | + <td>${ago(x.timestamp)} ago</td></tr>`).join("") || | |
| 92 | + `<tr><td colspan="7" class="empty">none in window</td></tr>`; | |
| 93 | +} | |
| 94 | + | |
| 95 | +function connectWS() { | |
| 96 | + const ws = new WebSocket(`${location.protocol === "https:" ? "wss" : "ws"}://${location.host}/v1/stream/transfers?min_usd=500`); | |
| 97 | + ws.onopen = () => $("wsState").textContent = "· live"; | |
| 98 | + ws.onclose = () => { $("wsState").textContent = "· reconnecting…"; setTimeout(connectWS, 3000); }; | |
| 99 | + ws.onmessage = (ev) => { | |
| 100 | + const feed = $("feed"); | |
| 101 | + for (const r of JSON.parse(ev.data).slice(-25).reverse()) { | |
| 102 | + const tr = document.createElement("tr"); | |
| 103 | + tr.innerHTML = `<td>${chainLink(r.chain)}</td> | |
| 104 | + <td>${tokenLink(r.symbol)}</td><td class="num">${fmtAmt(r)}</td> | |
| 105 | + <td>${addrLink(r.from)} → ${addrLink(r.to)}</td><td>${txLink(r.chain, r.tx_hash)}</td>`; | |
| 106 | + feed.prepend(tr); | |
| 107 | + } | |
| 108 | + while (feed.rows.length > 60) feed.deleteRow(-1); | |
| 109 | + }; | |
| 110 | +} | |
| 111 | + | |
| 112 | +j("/v1/tokens").then((t) => { | |
| 113 | + $("tokenSel").innerHTML = Object.keys(t).sort() | |
| 114 | + .map((s) => `<option ${s === TOKEN ? "selected" : ""}>${s}</option>`).join(""); | |
| 115 | +}); | |
| 116 | +$("tokenSel").addEventListener("change", (e) => { TOKEN = e.target.value; refresh(); }); | |
| 117 | +document.querySelectorAll(".presets button").forEach((b) => b.addEventListener("click", () => { | |
| 118 | + document.querySelectorAll(".presets button").forEach((x) => x.setAttribute("aria-pressed", "false")); | |
| 119 | + b.setAttribute("aria-pressed", "true"); WINDOW = b.dataset.w; refresh(); refreshWhales(); | |
| 120 | +})); | |
| 121 | +refresh(); refreshWhales(); connectWS(); | |
| 122 | +setInterval(refresh, 30_000); setInterval(refreshWhales, 60_000); | |
| 123 | +</script> | |
| 124 | +</body> | |
| 125 | +</html> | |
added
ui/status.html
+75 −0
@@ -0,0 +1,75 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>status — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1>Indexer status</h1> | |
| 13 | + <span class="sub">per-chain progress, coverage and free-RPC endpoint health · | |
| 14 | + <a href="/metrics">Prometheus</a></span></div> | |
| 15 | + | |
| 16 | + <section class="card tile"><h2>Transfers indexed</h2><div class="big" id="tTotal">—</div></section> | |
| 17 | + <section class="card tile"><h2>Chains at head</h2><div class="big" id="tHead">—</div> | |
| 18 | + <div class="note">lag ≤ 100 blocks</div></section> | |
| 19 | + <section class="card tile"><h2>Healthy endpoints</h2><div class="big" id="tEp">—</div> | |
| 20 | + <div class="note">score ≥ 0.8</div></section> | |
| 21 | + <section class="card tile"><h2>Benched endpoints</h2><div class="big" id="tBench">—</div> | |
| 22 | + <div class="note">currently cooling down</div></section> | |
| 23 | + | |
| 24 | + <section class="card"><h2>Chains</h2> | |
| 25 | + <div class="scroll" style="max-height:55vh"><table> | |
| 26 | + <thead><tr><th>chain</th><th class="num">indexed</th><th class="num">cursor</th> | |
| 27 | + <th class="num">head</th><th class="num">lag</th><th class="num">floor</th></tr></thead> | |
| 28 | + <tbody id="chains"></tbody></table></div></section> | |
| 29 | + | |
| 30 | + <section class="card"><h2>RPC endpoints</h2> | |
| 31 | + <div class="scroll" style="max-height:55vh"><table> | |
| 32 | + <thead><tr><th>chain</th><th>endpoint</th><th class="num">score</th><th class="num">ok</th> | |
| 33 | + <th class="num">fail</th><th class="num">latency</th><th class="num">cooldown</th></tr></thead> | |
| 34 | + <tbody id="rpc"></tbody></table></div></section> | |
| 35 | +</main> | |
| 36 | + | |
| 37 | +<script type="module"> | |
| 38 | +import { $, j, fmtN, mountNav, chainLink } from "/assets/app.js"; | |
| 39 | +mountNav("Status"); | |
| 40 | + | |
| 41 | +async function load() { | |
| 42 | + const [status, health] = await Promise.all([j("/v1/status"), j("/v1/rpc/health")]); | |
| 43 | + $("tTotal").textContent = fmtN(Object.values(status.indexed_transfers).reduce((a, b) => a + b, 0)); | |
| 44 | + const cs = Object.entries(status.cursors); | |
| 45 | + $("tHead").textContent = `${cs.filter(([, c]) => c.lag != null && c.lag <= 100).length}/${cs.length}`; | |
| 46 | + const eps = Object.values(health).flat(); | |
| 47 | + $("tEp").textContent = `${eps.filter((e) => e.score >= 0.8).length}/${eps.length}`; | |
| 48 | + $("tBench").textContent = eps.filter((e) => e.cooldown_s > 0).length; | |
| 49 | + | |
| 50 | + $("chains").innerHTML = cs.sort((a, b) => | |
| 51 | + (status.indexed_transfers[b[0]] || 0) - (status.indexed_transfers[a[0]] || 0)) | |
| 52 | + .map(([c, cur]) => { | |
| 53 | + const lag = cur.lag; | |
| 54 | + const cls = lag == null ? "" : lag > 1000 ? "burn" : lag > 100 ? "" : "mint"; | |
| 55 | + return `<tr><td>${chainLink(c)}</td> | |
| 56 | + <td class="num">${fmtN(status.indexed_transfers[c] || 0)}</td> | |
| 57 | + <td class="num">${fmtN(cur.last_block)}</td> | |
| 58 | + <td class="num">${cur.head_block ? fmtN(cur.head_block) : "—"}</td> | |
| 59 | + <td class="num ${cls}">${lag != null ? fmtN(lag) : "—"}</td> | |
| 60 | + <td class="num">${cur.backfill_block != null ? fmtN(cur.backfill_block) : "—"}</td></tr>`; | |
| 61 | + }).join(""); | |
| 62 | + | |
| 63 | + $("rpc").innerHTML = Object.entries(health).flatMap(([c, list]) => list.map((e) => | |
| 64 | + `<tr><td>${chainLink(c)}</td> | |
| 65 | + <td class="mono" style="max-width:none" title="${e.url}">${e.url}</td> | |
| 66 | + <td class="num ${e.score >= 0.8 ? "mint" : e.score < 0.5 ? "burn" : ""}">${e.score}</td> | |
| 67 | + <td class="num">${fmtN(e.ok)}</td><td class="num">${fmtN(e.fail)}</td> | |
| 68 | + <td class="num">${e.latency_ms} ms</td> | |
| 69 | + <td class="num">${e.cooldown_s > 0 ? e.cooldown_s + "s" : "—"}</td></tr>`)).join(""); | |
| 70 | +} | |
| 71 | +load(); | |
| 72 | +setInterval(load, 15_000); | |
| 73 | +</script> | |
| 74 | +</body> | |
| 75 | +</html> | |
added
ui/token.html
+130 −0
@@ -0,0 +1,130 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>token — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1 id="title">Token</h1><span class="sub" id="issuerNote"></span></div> | |
| 13 | + <div class="filters"> | |
| 14 | + <select id="tokenSel" aria-label="Token"></select> | |
| 15 | + <div class="presets" role="group" aria-label="Window"> | |
| 16 | + <button data-w="24h" aria-pressed="true">24h</button><button data-w="7d">7d</button> | |
| 17 | + </div> | |
| 18 | + </div> | |
| 19 | + | |
| 20 | + <section class="card tile"><h2>On-chain supply</h2> | |
| 21 | + <div class="big" id="tSupply">—</div><div class="note" id="tSupplyN"></div></section> | |
| 22 | + <section class="card tile"><h2>Volume <span class="sub" id="tVolW"></span></h2> | |
| 23 | + <div class="big" id="tVol">—</div><div class="note" id="tVolN"></div></section> | |
| 24 | + <section class="card tile"><h2>Active senders <span class="sub" id="tActW"></span></h2> | |
| 25 | + <div class="big" id="tAct">—</div><div class="note">distinct sending addresses</div></section> | |
| 26 | + <section class="card tile"><h2>Net issuance <span class="sub" id="tNetW"></span></h2> | |
| 27 | + <div class="big" id="tNet">—</div><div class="note" id="tNetN"></div></section> | |
| 28 | + | |
| 29 | + <section class="card half"><h2 id="volTitle">Volume over time</h2><div id="volChart"></div></section> | |
| 30 | + <section class="card half"><h2 id="supTitle">Supply over time</h2><div id="supChart"></div></section> | |
| 31 | + | |
| 32 | + <section class="card third"><h2>Supply by chain</h2><div class="bars" id="supplyBars"></div></section> | |
| 33 | + <section class="card twothird"><h2>Largest transfers <span class="sub" id="whaleW"></span></h2> | |
| 34 | + <div class="scroll"><table> | |
| 35 | + <thead><tr><th>chain</th><th class="num">USD</th><th>from</th><th>to</th><th>tx</th><th>when</th></tr></thead> | |
| 36 | + <tbody id="whales"></tbody></table></div></section> | |
| 37 | + | |
| 38 | + <section class="card half"><h2>Mints & burns <span class="sub" id="mbW"></span></h2> | |
| 39 | + <div class="scroll"><table> | |
| 40 | + <thead><tr><th></th><th>chain</th><th class="num">amount</th><th>tx</th><th>when</th></tr></thead> | |
| 41 | + <tbody id="mintburn"></tbody></table></div></section> | |
| 42 | + <section class="card half"><h2>Deployments by chain</h2> | |
| 43 | + <div class="scroll"><table> | |
| 44 | + <thead><tr><th>chain</th><th>identifier</th><th>issuance</th><th class="num">decimals</th></tr></thead> | |
| 45 | + <tbody id="deploys"></tbody></table></div></section> | |
| 46 | +</main> | |
| 47 | + | |
| 48 | +<script type="module"> | |
| 49 | +import { $, qs, j, fmtUsd, fmtN, fmtAmt, mountNav, hbars, lineChart, topSeries, | |
| 50 | + addrLink, txLink, chainLink, coinIcon, ago } from "/assets/app.js"; | |
| 51 | +mountNav("Tokens"); | |
| 52 | +let TOKEN = (qs.get("symbol") || "USDT").toUpperCase(), WINDOW = "24h"; | |
| 53 | +const ivl = () => ({ "24h": "1h", "7d": "6h" }[WINDOW]); | |
| 54 | + | |
| 55 | +async function refresh() { | |
| 56 | + document.title = `${TOKEN} — coinexplorer`; | |
| 57 | + $("title").innerHTML = `${coinIcon(TOKEN, 26)} ${TOKEN}`; | |
| 58 | + const [supply, vol, volSeries, supSeries, flows, tokens] = await Promise.all([ | |
| 59 | + j(`/v1/stablecoins/supply?token=${TOKEN}`), | |
| 60 | + j(`/v1/stablecoins/volume?token=${TOKEN}&window=${WINDOW}`), | |
| 61 | + j(`/v1/stablecoins/volume/series?token=${TOKEN}&window=${WINDOW}&interval=${ivl()}`), | |
| 62 | + j(`/v1/stablecoins/supply/series?token=${TOKEN}&window=7d`), | |
| 63 | + j(`/v1/stablecoins/flows?token=${TOKEN}&window=${WINDOW}&interval=${ivl()}`), | |
| 64 | + j("/v1/tokens"), | |
| 65 | + ]); | |
| 66 | + const s = supply[TOKEN]; | |
| 67 | + $("tSupply").textContent = s ? fmtUsd(s.total_usd ?? s.total) : "—"; | |
| 68 | + $("tSupplyN").textContent = s ? `across ${Object.keys(s.chains).length} indexed chains` : "no snapshot yet"; | |
| 69 | + $("tVol").textContent = fmtUsd(vol.total_volume); | |
| 70 | + ["tVolW", "tActW", "tNetW", "whaleW", "mbW"].forEach((id) => $(id).textContent = "· " + WINDOW); | |
| 71 | + $("tVolN").textContent = `${fmtN(Object.values(vol.chains).reduce((a, c) => a + c.transfers, 0))} transfers`; | |
| 72 | + $("tAct").textContent = fmtN(Object.values(vol.chains).reduce((a, c) => a + c.senders, 0)); | |
| 73 | + const fl = (flows.tokens[TOKEN] || []); | |
| 74 | + const net = fl.reduce((a, b) => a + b.net, 0); | |
| 75 | + $("tNet").textContent = (net >= 0 ? "+" : "") + fmtUsd(net); | |
| 76 | + $("tNet").className = "big " + (net >= 0 ? "up" : "down"); | |
| 77 | + $("tNetN").textContent = `minted ${fmtUsd(fl.reduce((a, b) => a + b.minted, 0))} · burned ${fmtUsd(fl.reduce((a, b) => a + b.burned, 0))}`; | |
| 78 | + | |
| 79 | + $("volTitle").textContent = `${TOKEN} volume — ${WINDOW}`; | |
| 80 | + lineChart($("volChart"), topSeries(volSeries.chains, 3)); | |
| 81 | + $("supTitle").textContent = `${TOKEN} supply — 7d (hourly snapshots)`; | |
| 82 | + lineChart($("supChart"), topSeries(supSeries.chains, 3, "supply"), { area: false }); | |
| 83 | + | |
| 84 | + hbars($("supplyBars"), Object.entries(s ? s.chains : {}) | |
| 85 | + .map(([k, v]) => ({ k, v: v.usd ?? v.supply, link: chainLink(k) })) | |
| 86 | + .sort((a, b) => b.v - a.v).slice(0, 14)); | |
| 87 | + | |
| 88 | + const info = tokens[TOKEN] || { chains: {} }; | |
| 89 | + $("deploys").innerHTML = Object.entries(info.chains).map(([c, d]) => | |
| 90 | + `<tr><td>${chainLink(c)}</td><td class="mono" title="${d.id}">${d.id}</td> | |
| 91 | + <td><span class="badge ${d.discontinued ? "discontinued" : d.native ? "native" : "bridged"}"> | |
| 92 | + ${d.discontinued ? "discontinued" : d.native ? "native" : "bridged"}</span></td> | |
| 93 | + <td class="num">${d.decimals ?? "—"}</td></tr>`).join(""); | |
| 94 | +} | |
| 95 | + | |
| 96 | +async function refreshLists() { | |
| 97 | + const [whales, mb] = await Promise.all([ | |
| 98 | + j(`/v1/stablecoins/whales?token=${TOKEN}&window=${WINDOW}&limit=25`), | |
| 99 | + j(`/v1/stablecoins/mints-burns?token=${TOKEN}&window=${WINDOW}&limit=25`), | |
| 100 | + ]); | |
| 101 | + $("whales").innerHTML = whales.map((x) => | |
| 102 | + `<tr><td>${chainLink(x.chain)}</td><td class="num"><b>${fmtUsd(x.usd)}</b></td> | |
| 103 | + <td>${addrLink(x.from)}</td><td>${addrLink(x.to)}</td> | |
| 104 | + <td>${txLink(x.chain, x.tx_hash)}</td><td>${ago(x.timestamp)} ago</td></tr>`).join("") || | |
| 105 | + `<tr><td colspan="6" class="empty">none in window</td></tr>`; | |
| 106 | + $("mintburn").innerHTML = mb.map((m) => | |
| 107 | + `<tr><td class="${m.direction}">${m.direction === "mint" ? "▲ mint" : "▼ burn"}</td> | |
| 108 | + <td>${chainLink(m.chain)}</td><td class="num">${fmtAmt(m)}</td> | |
| 109 | + <td>${txLink(m.chain, m.tx_hash)}</td><td>${ago(m.timestamp)} ago</td></tr>`).join("") || | |
| 110 | + `<tr><td colspan="5" class="empty">none in window</td></tr>`; | |
| 111 | +} | |
| 112 | + | |
| 113 | +j("/v1/tokens").then((t) => { | |
| 114 | + $("tokenSel").innerHTML = Object.keys(t).sort() | |
| 115 | + .map((s) => `<option ${s === TOKEN ? "selected" : ""}>${s}</option>`).join(""); | |
| 116 | +}); | |
| 117 | +$("tokenSel").addEventListener("change", (e) => { | |
| 118 | + TOKEN = e.target.value; | |
| 119 | + history.replaceState(null, "", `?symbol=${TOKEN}`); | |
| 120 | + refresh(); refreshLists(); | |
| 121 | +}); | |
| 122 | +document.querySelectorAll(".presets button").forEach((b) => b.addEventListener("click", () => { | |
| 123 | + document.querySelectorAll(".presets button").forEach((x) => x.setAttribute("aria-pressed", "false")); | |
| 124 | + b.setAttribute("aria-pressed", "true"); WINDOW = b.dataset.w; refresh(); refreshLists(); | |
| 125 | +})); | |
| 126 | +refresh(); refreshLists(); | |
| 127 | +setInterval(refresh, 60_000); | |
| 128 | +</script> | |
| 129 | +</body> | |
| 130 | +</html> | |
added
ui/transfers.html
+80 −0
@@ -0,0 +1,80 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>transfers — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1>Transfer explorer</h1> | |
| 13 | + <span class="sub">browse the index chain by chain · newest first</span></div> | |
| 14 | + | |
| 15 | + <div class="filters"> | |
| 16 | + <select id="chainSel" aria-label="Chain"></select> | |
| 17 | + <select id="symSel"><option value="">all tokens</option></select> | |
| 18 | + <input id="minUsd" type="number" placeholder="min USD" style="width:120px"> | |
| 19 | + <input id="addr" placeholder="filter by address (from/to)" style="flex:1;min-width:220px"> | |
| 20 | + <button class="btn" id="apply">Apply</button> | |
| 21 | + </div> | |
| 22 | + | |
| 23 | + <section class="card"><h2 id="tableTitle">Transfers</h2> | |
| 24 | + <div class="scroll" style="max-height:70vh"><table> | |
| 25 | + <thead><tr><th>block</th><th>token</th><th class="num">amount</th><th>from</th><th>to</th><th>tx</th><th>when</th></tr></thead> | |
| 26 | + <tbody id="rows"></tbody></table></div> | |
| 27 | + <div style="margin-top:8px"><button class="btn" id="more">Load more</button> | |
| 28 | + <span class="sub" id="count" style="margin-left:8px"></span></div></section> | |
| 29 | +</main> | |
| 30 | + | |
| 31 | +<script type="module"> | |
| 32 | +import { $, qs, j, fmtUsd, fmtN, fmtAmt, mountNav, addrLink, txLink, tokenLink, ago } from "/assets/app.js"; | |
| 33 | +mountNav("Transfers"); | |
| 34 | +let CHAIN = qs.get("chain") || "ethereum"; | |
| 35 | +let beforeBlock = null, total = 0; | |
| 36 | + | |
| 37 | +async function load(append = false) { | |
| 38 | + const sym = $("symSel").value, min = $("minUsd").value, addr = $("addr").value.trim(); | |
| 39 | + let rows; | |
| 40 | + if (addr) { | |
| 41 | + rows = (await j(`/v1/address/${encodeURIComponent(addr)}/transfers?limit=200`)) | |
| 42 | + .filter((r) => r.chain === CHAIN || !CHAIN) | |
| 43 | + .filter((r) => !sym || r.symbol === sym) | |
| 44 | + .filter((r) => !min || parseFloat(r.value) >= +min); | |
| 45 | + $("more").style.display = "none"; | |
| 46 | + } else { | |
| 47 | + let url = `/v1/${CHAIN}/transfers?limit=100`; | |
| 48 | + if (sym) url += `&symbol=${sym}`; | |
| 49 | + if (min) url += `&min_amount=${min}`; | |
| 50 | + if (append && beforeBlock) url += `&before_block=${beforeBlock}`; | |
| 51 | + rows = await j(url); | |
| 52 | + if (rows.length) beforeBlock = rows[rows.length - 1].block; | |
| 53 | + $("more").style.display = rows.length ? "" : "none"; | |
| 54 | + } | |
| 55 | + total = append ? total + rows.length : rows.length; | |
| 56 | + $("count").textContent = `${fmtN(total)} shown`; | |
| 57 | + $("tableTitle").textContent = `Transfers — ${CHAIN}${sym ? " · " + sym : ""}${addr ? " · " + addr.slice(0, 12) + "…" : ""}`; | |
| 58 | + const html = rows.map((r) => | |
| 59 | + `<tr><td>${fmtN(r.block)}</td><td>${tokenLink(r.symbol)}</td> | |
| 60 | + <td class="num">${fmtAmt(r)}</td><td>${addrLink(r.from)}</td> | |
| 61 | + <td>${addrLink(r.to)}</td><td>${txLink(r.chain, r.tx_hash)}</td> | |
| 62 | + <td>${ago(r.timestamp)} ago</td></tr>`).join(""); | |
| 63 | + if (append) $("rows").insertAdjacentHTML("beforeend", html); | |
| 64 | + else $("rows").innerHTML = html || `<tr><td colspan="7" class="empty">no matches</td></tr>`; | |
| 65 | +} | |
| 66 | + | |
| 67 | +Promise.all([j("/v1/chains"), j("/v1/tokens")]).then(([chains, tokens]) => { | |
| 68 | + $("chainSel").innerHTML = Object.keys(chains).sort() | |
| 69 | + .map((c) => `<option ${c === CHAIN ? "selected" : ""}>${c}</option>`).join(""); | |
| 70 | + $("symSel").innerHTML = `<option value="">all tokens</option>` + | |
| 71 | + Object.keys(tokens).sort().map((s) => `<option>${s}</option>`).join(""); | |
| 72 | + load(); | |
| 73 | +}); | |
| 74 | +$("chainSel").addEventListener("change", (e) => { CHAIN = e.target.value; beforeBlock = null; load(); }); | |
| 75 | +$("apply").addEventListener("click", () => { beforeBlock = null; load(); }); | |
| 76 | +$("addr").addEventListener("keydown", (e) => { if (e.key === "Enter") { beforeBlock = null; load(); } }); | |
| 77 | +$("more").addEventListener("click", () => load(true)); | |
| 78 | +</script> | |
| 79 | +</body> | |
| 80 | +</html> | |
added
ui/tx.html
+59 −0
@@ -0,0 +1,59 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>tx — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1>Transaction</h1><span class="sub mono" id="hash" style="max-width:none"></span></div> | |
| 13 | + <section class="card" id="summary"><h2>Summary</h2><div class="empty" id="state">loading…</div> | |
| 14 | + <div class="scroll"><table id="kv" style="display:none;min-width:0"><tbody></tbody></table></div></section> | |
| 15 | + <section class="card"><h2>Stablecoin transfers in this transaction</h2> | |
| 16 | + <div class="scroll"><table><thead><tr><th>token</th><th class="num">amount</th><th>from</th><th>to</th></tr></thead> | |
| 17 | + <tbody id="rows"></tbody></table></div></section> | |
| 18 | +</main> | |
| 19 | + | |
| 20 | +<script type="module"> | |
| 21 | +import { $, qs, j, fmtUsd, fmtN, fmtAmt, mountNav, addrLink, chainLink, tokenLink, ago } from "/assets/app.js"; | |
| 22 | +mountNav("Transfers"); | |
| 23 | +const HASH = qs.get("hash") || ""; | |
| 24 | +let CHAIN = qs.get("chain") || ""; | |
| 25 | +$("hash").textContent = HASH; | |
| 26 | + | |
| 27 | +async function load() { | |
| 28 | + if (!HASH) { $("state").textContent = "no hash given"; return; } | |
| 29 | + if (!CHAIN) { | |
| 30 | + const s = await j(`/v1/search?q=${encodeURIComponent(HASH)}`); | |
| 31 | + if (s.chain) CHAIN = s.chain; | |
| 32 | + else { $("state").textContent = "transaction not found in the index — supply ?chain= to query a node directly"; return; } | |
| 33 | + } | |
| 34 | + let t; | |
| 35 | + try { t = await j(`/v1/${CHAIN}/tx/${encodeURIComponent(HASH)}`); } | |
| 36 | + catch { $("state").textContent = "not found on " + CHAIN; return; } | |
| 37 | + $("state").style.display = "none"; | |
| 38 | + const kv = $("kv"); | |
| 39 | + kv.style.display = ""; | |
| 40 | + const rows = [ | |
| 41 | + ["chain", chainLink(t.chain)], | |
| 42 | + ["block", t.block != null ? fmtN(t.block) : "pending"], | |
| 43 | + ["status", t.status === 1 ? "✓ success" : t.status === 0 ? "✗ reverted" : t.source === "index" ? "indexed" : "—"], | |
| 44 | + ]; | |
| 45 | + if (t.timestamp) rows.push(["time", new Date(t.timestamp * 1000).toLocaleString() + ` (${ago(t.timestamp)} ago)`]); | |
| 46 | + if (t.from) rows.push(["from", addrLink(t.from)]); | |
| 47 | + if (t.to) rows.push(["to", addrLink(t.to)]); | |
| 48 | + if (t.gas_used != null) rows.push(["gas used", fmtN(t.gas_used)]); | |
| 49 | + kv.tBodies[0].innerHTML = rows.map(([k, v]) => | |
| 50 | + `<tr><td style="color:var(--muted);width:120px">${k}</td><td style="max-width:none">${v}</td></tr>`).join(""); | |
| 51 | + $("rows").innerHTML = (t.stablecoin_transfers || []).map((x) => | |
| 52 | + `<tr><td>${tokenLink(x.symbol)}</td><td class="num"><b>${fmtAmt(x)}</b></td> | |
| 53 | + <td>${addrLink(x.from)}</td><td>${addrLink(x.to)}</td></tr>`).join("") || | |
| 54 | + `<tr><td colspan="4" class="empty">no stablecoin transfers decoded</td></tr>`; | |
| 55 | +} | |
| 56 | +load(); | |
| 57 | +</script> | |
| 58 | +</body> | |
| 59 | +</html> | |
added
ui/whales.html
+77 −0
@@ -0,0 +1,77 @@ | ||
| 1 | +<!doctype html> | |
| 2 | +<!-- Author: Simon-Pierre Boucher --> | |
| 3 | +<!-- Mail: contact@spboucher.ai --> | |
| 4 | +<html lang="en"> | |
| 5 | +<head> | |
| 6 | +<meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1"> | |
| 7 | +<title>whales — coinexplorer</title> | |
| 8 | +<link rel="stylesheet" href="/assets/style.css"> | |
| 9 | +</head> | |
| 10 | +<body> | |
| 11 | +<main> | |
| 12 | + <div class="pagehead"><h1>Whale watch</h1> | |
| 13 | + <span class="sub">largest stablecoin & crypto movements across all indexed chains — including native BTC / ETH / SOL / TRX / BNB whales</span></div> | |
| 14 | + | |
| 15 | + <div class="filters"> | |
| 16 | + <select id="tokenSel"><option value="">all tokens</option></select> | |
| 17 | + <select id="minSel"> | |
| 18 | + <option value="250000">≥ $250K</option> | |
| 19 | + <option value="1000000" selected>≥ $1M</option> | |
| 20 | + <option value="10000000">≥ $10M</option> | |
| 21 | + <option value="100000000">≥ $100M</option> | |
| 22 | + </select> | |
| 23 | + <div class="presets" role="group" aria-label="Window"> | |
| 24 | + <button data-w="1h">1h</button><button data-w="24h" aria-pressed="true">24h</button><button data-w="7d">7d</button> | |
| 25 | + </div> | |
| 26 | + </div> | |
| 27 | + | |
| 28 | + <section class="card tile"><h2>Transfers over threshold</h2> | |
| 29 | + <div class="big" id="tCount">—</div><div class="note" id="tCountN"></div></section> | |
| 30 | + <section class="card tile"><h2>Combined value</h2> | |
| 31 | + <div class="big" id="tSum">—</div><div class="note">sum of listed transfers</div></section> | |
| 32 | + <section class="card half"><h2>By chain</h2><div class="bars" id="chainBars"></div></section> | |
| 33 | + | |
| 34 | + <section class="card"><h2>Transfers</h2> | |
| 35 | + <div class="scroll" style="max-height:65vh"><table> | |
| 36 | + <thead><tr><th>chain</th><th>token</th><th class="num">USD</th><th>from</th><th>to</th><th>tx</th><th>when</th></tr></thead> | |
| 37 | + <tbody id="rows"></tbody></table></div></section> | |
| 38 | +</main> | |
| 39 | + | |
| 40 | +<script type="module"> | |
| 41 | +import { $, j, fmtUsd, fmtN, mountNav, hbars, addrLink, txLink, chainLink, tokenLink, ago } from "/assets/app.js"; | |
| 42 | +mountNav("Whales"); | |
| 43 | +let WINDOW = "24h"; | |
| 44 | + | |
| 45 | +async function load() { | |
| 46 | + const tok = $("tokenSel").value, min = $("minSel").value; | |
| 47 | + const url = `/v1/stablecoins/whales?window=${WINDOW}&min_usd=${min}&limit=200` + (tok ? `&token=${tok}` : ""); | |
| 48 | + const rows = await j(url); | |
| 49 | + $("tCount").textContent = fmtN(rows.length); | |
| 50 | + $("tCountN").textContent = `≥ ${fmtUsd(+min)} · last ${WINDOW}`; | |
| 51 | + $("tSum").textContent = fmtUsd(rows.reduce((a, r) => a + r.usd, 0)); | |
| 52 | + const byChain = {}; | |
| 53 | + for (const r of rows) byChain[r.chain] = (byChain[r.chain] || 0) + r.usd; | |
| 54 | + hbars($("chainBars"), Object.entries(byChain) | |
| 55 | + .map(([k, v]) => ({ k, v, link: chainLink(k) })).sort((a, b) => b.v - a.v).slice(0, 8)); | |
| 56 | + $("rows").innerHTML = rows.map((x) => | |
| 57 | + `<tr><td>${chainLink(x.chain)}</td><td>${tokenLink(x.symbol)}</td> | |
| 58 | + <td class="num"><b>${fmtUsd(x.usd)}</b></td><td>${addrLink(x.from)}</td> | |
| 59 | + <td>${addrLink(x.to)}</td><td>${txLink(x.chain, x.tx_hash)}</td> | |
| 60 | + <td>${ago(x.timestamp)} ago</td></tr>`).join("") || | |
| 61 | + `<tr><td colspan="7" class="empty">none in window</td></tr>`; | |
| 62 | +} | |
| 63 | + | |
| 64 | +j("/v1/tokens").then((t) => { | |
| 65 | + $("tokenSel").innerHTML = `<option value="">all tokens</option>` + | |
| 66 | + Object.keys(t).sort().map((s) => `<option>${s}</option>`).join(""); | |
| 67 | +}); | |
| 68 | +["tokenSel", "minSel"].forEach((id) => $(id).addEventListener("change", load)); | |
| 69 | +document.querySelectorAll(".presets button").forEach((b) => b.addEventListener("click", () => { | |
| 70 | + document.querySelectorAll(".presets button").forEach((x) => x.setAttribute("aria-pressed", "false")); | |
| 71 | + b.setAttribute("aria-pressed", "true"); WINDOW = b.dataset.w; load(); | |
| 72 | +})); | |
| 73 | +load(); | |
| 74 | +setInterval(load, 60_000); | |
| 75 | +</script> | |
| 76 | +</body> | |
| 77 | +</html> | |
| 78 | ||