SPB Git forge

spb/fetcha

Public
11commits 1branches 0releases
1.5 MBsize
maindefault branch
16 days agolast push
TypeScript 97.5% SQL 1.4% Python 0.8%
11.3 KB · 226 lines tsx
Raw Blame History
1import type { Metadata } from "next";2import { PLAN_LIMITS } from "@fetcha/core";3import { CodeBlock } from "@/components/ui/code-block";4import { Badge } from "@/components/ui/badge";5import { DocPage } from "@/components/docs/doc-page";6import { A, Code, H2, H3, Li, P, Strong, Table, TBody, Td, Th, THead, Tr, Ul } from "@/components/docs/prose";7import { Callout } from "@/components/docs/callout";8import { CodeTabs } from "@/components/docs/code-tabs";9import { ResponseExample } from "@/components/docs/response-example";10import { fetchTabs } from "@/components/docs/snippets";1112export const metadata: Metadata = {13  title: "Network Selection",14  description: "Network classes, what auto does, how routes are scored, per-domain intelligence, escalation, circuit breakers and which classes are available (all of them).",15};1617const CLASSES: Array<{ name: string; live: boolean; desc: string }> = [18  { name: "auto", live: true, desc: "Let the engine choose. Cheapest class first, escalating to more reliable classes on blocks. Resolves to residential today." },19  { name: "residential", live: true, desc: "Exit IPs assigned by consumer ISPs to households. Highest acceptance on sites that filter datacenter traffic. Geo-targetable." },20  { name: "datacenter", live: false, desc: "Exit IPs from hosting providers. Fastest and cheapest, most often blocked. Included, no live route yet." },21  { name: "isp", live: false, desc: "Static IPs registered to consumer ISPs but hosted in datacenters. Residential reputation with datacenter speed. Included, no live route yet." },22  { name: "mobile", live: false, desc: "Exit IPs from cellular carriers. Highest acceptance, highest cost. Included, no live route yet." },23];2425const ALL_NETWORKS = ["datacenter", "residential", "isp", "mobile"] as const;26const L = PLAN_LIMITS.unlimited;2728export default function NetworksPage() {29  return (30    <DocPage path="/docs/networks" eyebrow="Core API" title="Network selection" description="Fetcha routes requests across classes of networks rather than individual proxies. You pick a class, or let auto pick for you, and the engine handles scoring, escalation and health." status="Stable">31      <H2>Network classes</H2>32      <P>33        The <Code>network</Code> field of a fetch request accepts one of five values. Classes describe the <Strong>kind</Strong> of exit IP; the upstream networks behind each class are managed by34        Fetcha and never exposed.35      </P>36      <Table>37        <THead>38          <Tr>39            <Th>Value</Th>40            <Th>Status</Th>41            <Th>Description</Th>42          </Tr>43        </THead>44        <TBody>45          {CLASSES.map((c) => (46            <Tr key={c.name}>47              <Td mono>{c.name}</Td>48              <Td>49                <Badge variant={c.live ? "success" : "warning"} dot>50                  {c.live ? "Live" : "Coming soon"}51                </Badge>52              </Td>53              <Td>{c.desc}</Td>54            </Tr>55          ))}56        </TBody>57      </Table>58      <Callout variant="warning" title="What happens today">59        Only the <Code>residential</Code> class has live capacity. <Code>auto</Code> therefore resolves to residential and <Code>metadata.network</Code> reports <Code>{`"residential"`}</Code>. Requesting{" "}60        <Code>datacenter</Code>, <Code>isp</Code> or <Code>mobile</Code> explicitly returns <Code>400 NETWORK_UNAVAILABLE</Code> because the class has no live route yet — never because of your plan: all61        classes are included for every organization. When those classes launch, <Code>auto</Code> will start using them without any change on your side.62      </Callout>6364      <H2>What auto does</H2>65      <P>66        <Code>auto</Code> is the default and the recommended setting. For each request the routing engine builds an ordered list of candidate routes (a route is one upstream network serving one class),67        then executes them in order until one succeeds or the retry budget is spent.68      </P>69      <H3>1. Eligible classes</H3>70      <P>71        The engine starts from the escalation order <Code>datacenter → isp → residential → mobile</Code> (all four classes are available to every organization) and drops classes that have no72        configured, healthy route right now. If the domain has an admin-pinned <Code>force_network</Code> policy, only that class is considered.73      </P>74      <H3>2. Scoring</H3>75      <P>Every eligible route receives a score between 0 and 1, computed as a weighted sum of six signals:</P>76      <Table>77        <THead>78          <Tr>79            <Th>Weight</Th>80            <Th>Signal</Th>81            <Th>How it is measured</Th>82          </Tr>83        </THead>84        <TBody>85          <Tr>86            <Td mono>35 %</Td>87            <Td>Historical success</Td>88            <Td>89              Success rate of this route on this domain, Bayesian-smoothed toward a per-class prior (datacenter 70 %, isp 85 %, residential 93 %, mobile 95 %) so new domains start with sensible90              expectations.91            </Td>92          </Tr>93          <Tr>94            <Td mono>20 %</Td>95            <Td>Cost</Td>96            <Td>Price per GB of the route, normalised against the most expensive class (about $15/GB).</Td>97          </Tr>98          <Tr>99            <Td mono>15 %</Td>100            <Td>Latency</Td>101            <Td>Average latency observed on this domain (300 ms scores 1.0, 5 s scores 0). Without history, datacenter assumes 500 ms and other classes 1,200 ms.</Td>102          </Tr>103          <Tr>104            <Td mono>15 %</Td>105            <Td>Network health</Td>106            <Td>Health factor from the route&apos;s circuit breaker: 1.0 when closed and clean, degraded by recent failure ratio, 0.3 while half-open, 0 while open.</Td>107          </Tr>108          <Tr>109            <Td mono>10 %</Td>110            <Td>Geography</Td>111            <Td>Whether the route can serve the requested country, region and city.</Td>112          </Tr>113          <Tr>114            <Td mono>5 %</Td>115            <Td>Session stability</Td>116            <Td>Whether the route supports sticky sessions when the request carries one.</Td>117          </Tr>118        </TBody>119      </Table>120      <H3>3. Ordering and escalation</H3>121      <Ul>122        <Li>123          In <Code>auto</Code> mode, candidates are grouped by class in escalation order (cheap first) and sorted by score inside each class. Distinct routes are preferred before repeating one, so a124          retry usually changes both the exit IP and the route.125        </Li>126        <Li>127          <Strong>Per-domain intelligence:</Strong> if the domain profile shows a cheaper class being blocked more than 60 % of the time over at least 5 attempts, that class is skipped entirely for128          the domain. Residential is never skipped by this rule.129        </Li>130        <Li>131          The list is cut to your attempt budget: <Code>retries + 1</Code>, at most {L.max_retries + 1}. See <A href="/docs/retries">Retries</A>.132        </Li>133        <Li>134          With an explicit class (for example <Code>{`"network": "residential"`}</Code>), candidates are simply sorted by score; there is no cross-class escalation.135        </Li>136      </Ul>137      <H3>4. Learning</H3>138      <P>139        After every attempt, the domain profile is updated with the outcome (success, block, latency, cost). Profiles are shared across all Fetcha customers, so a site that starts blocking a class is140        learned once and avoided for everyone. Only aggregate statistics are stored; never your request or response content.141      </P>142143      <H2>Circuit breakers</H2>144      <P>145        Each route has an in-memory circuit breaker so that an upstream incident does not turn into a wave of failed requests:146      </P>147      <Ul>148        <Li>Outcomes are tracked in a rolling 120-second window.</Li>149        <Li>150          The circuit <Strong>opens</Strong> when at least 8 samples are present and 60 % or more of them failed. An open route is excluded from routing and scores 0 on the health signal.151        </Li>152        <Li>After a 60-second cooldown the circuit becomes half-open: one request probes the route; success closes the circuit, failure reopens it.</Li>153        <Li>154          Only network-side failures count (upstream authentication or gateway errors, connection failures, 5xx from the gateway). A target that blocks or times out does not penalise the route.155        </Li>156      </Ul>157      <P>158        When every candidate route is open or unconfigured, <Code>auto</Code> requests fail fast with <Code>503 PROVIDER_UNAVAILABLE</Code> and explicit-class requests with{" "}159        <Code>400 NETWORK_UNAVAILABLE</Code>. Neither counts as a target failure and both are safe to retry after a short pause.160      </P>161162      <H2>Availability</H2>163      <P>164        Fetcha has a single plan and no network class is gated: every organization may request any class. The only reason an explicit class is refused is the absence of a live route for it. The table165        reflects the platform configuration; remember that only residential is live today.166      </P>167      <Table>168        <THead>169          <Tr>170            <Th>Class</Th>171            <Th className="text-center">Included</Th>172            <Th className="text-center">Live route</Th>173          </Tr>174        </THead>175        <TBody>176          {ALL_NETWORKS.map((n) => {177            const live = CLASSES.find((c) => c.name === n)?.live ?? false;178            return (179              <Tr key={n}>180                <Td mono>{n}</Td>181                <Td className="text-center">{L.networks.includes(n) ? <span className="text-success">Yes</span> : <span className="text-fg-subtle">&mdash;</span>}</Td>182                <Td className="text-center">{live ? <span className="text-success">Yes</span> : <span className="text-fg-subtle">Not yet</span>}</Td>183              </Tr>184            );185          })}186        </TBody>187      </Table>188189      <H2>Choosing a class</H2>190      <P>191        Prefer <Code>auto</Code>. Pin <Code>residential</Code> when you need to guarantee that no cheaper class is ever tried on a sensitive target. Read the served class from <Code>metadata.network</Code> and, with <Code>debug: true</Code>, the per-attempt route aliases from <Code>metadata.debug.attempts</Code>.192      </P>193      <CodeTabs194        tabs={fetchTabs(195          { url: "https://example.com", network: "residential", country: "US", debug: true },196          {197            javascript: `console.log(data.metadata.network, data.metadata.debug.attempts);`,198            python: `print(data["metadata"]["network"], data["metadata"]["debug"]["attempts"])`,199          },200          ["curl", "javascript", "python"],201        )}202      />203      <ResponseExample204        status={400}205        statusText="Bad Request"206        title="400 Bad Request · explicit class not available"207        body={{ error: { code: "NETWORK_UNAVAILABLE", message: 'The "mobile" network has no live route right now.', request_id: "req_0a1b2c3d4e5f6g7h" } }}208      />209      <CodeBlock210        lang="json"211        title="metadata.debug.attempts — escalation across two routes"212        code={JSON.stringify(213          {214            attempts: [215              { provider: "network-a", network: "residential", country: "US", outcome: "blocked", status: 403, duration_ms: 1320 },216              { provider: "network-b", network: "residential", country: "US", outcome: "success", status: 200, duration_ms: 1710 },217            ],218          },219          null,220          2,221        )}222      />223    </DocPage>224  );225}226