spb/datacenterindex

Public

338.7 MBsize

maindefault branch

2 h agolast push

HTML 53.9% TypeScript 44.5% JavaScript 0.6% SQL 0.5%

954 B · 35 lines

Raw Blame History

1# DataCenterIndex — CRAWL node (BHS64b) secrets & settings.2# Copy to deploy/.env.crawl (gitignored); deploy/bin/deploy.sh crawl copies it to BHS64b:/srv/dci/app/deploy/.env.crawl.3# Store credentials MUST match deploy/.env.data (the workers connect to 10.68.0.1 over dci0).45DATA_PRIVATE_IP=10.68.0.16CRAWL_PRIVATE_IP=10.68.0.27TZ=America/Toronto8DCI_TAG=latest9TSX_VERSION=4.23.131011# ── same values as .env.data12POSTGRES_PASSWORD=13REDIS_PASSWORD=14CLICKHOUSE_PASSWORD=15CLICKHOUSE_DB=dci16S3_ACCESS_KEY=dci17S3_SECRET_KEY=18S3_BUCKET=dci-raw1920# ── crawling21SCRAPFLY_API_KEY=22FIRECRAWL_API_KEY=23DCI_SCRAPFLY_DAILY_BUDGET=40024DCI_FIRECRAWL_DAILY_BUDGET=20025DCI_USER_AGENT="DataCenterIndexBot/0.1 (+https://www.datacenterindex.io/bot; contact@spboucher.ai)"26DCI_CRAWL_CONCURRENCY=627DCI_MAINTENANCE_CONCURRENCY=128DCI_LOG_LEVEL=info29DCI_CLICKHOUSE_OPTIONAL=130PG_POOL_MAX=83132# ── resources (BHS64b: 8c/16t, 64 GB)33WORKER_MEM_LIMIT=12g34WORKER_CPUS=835