SPB Git forge

spb/home-ka

Public
10commits 1branches 0releases
793.0 KBsize
maindefault branch
19 days agolast push
Python 49.6% TypeScript 25.5% CSS 24.1%
ZIP tar.gz
NameLast commitUpdated
data Home-Ka v1 — US real-estate aggregator (Immo-Ka architecture):... 27 days ago
frontend Share/branding assets: Home-Ka og.png (1200x630, US tricolor theme),... 27 days ago
homeka quality: plausible price is a hard publication requirement; jsonld:... 27 days ago
scripts quality: plausible price is a hard publication requirement; jsonld:... 27 days ago
tests Home-Ka v1 — US real-estate aggregator (Immo-Ka architecture):... 27 days ago
vendor Home-Ka v1 — US real-estate aggregator (Immo-Ka architecture):... 27 days ago
.env.example Home-Ka v1 — US real-estate aggregator (Immo-Ka architecture):... 27 days ago
.gitignore Home-Ka v1 — US real-estate aggregator (Immo-Ka architecture):... 27 days ago
README.md chore(retrait cluster): synchro copie du nœud M4M36 + README « URL... 19 days ago
requirements.txt Home-Ka v1 — US real-estate aggregator (Immo-Ka architecture):... 27 days ago
run.py Home-Ka v1 — US real-estate aggregator (Immo-Ka architecture):... 27 days ago
README.md source

⚠️ URL live hors fonction pour le moment. L'application a été retirée du cluster MacLustr le 2026-09-04 (processus arrêtés, copie du nœud supprimée). Ce dépôt spbgit est désormais la seule source de vérité du projet. Ancienne URL : https://www.home-ka.com

Home·Ka

US real-estate aggregator — every home for sale, from direct feeds first.
www.home-ka.com · Groupe KA · US counterpart of Immo·Ka (Canada)

Home-Ka reuses the Immo-Ka architecture end to end — connectors, normalization, ingestion, property matching, deduplication, quality gate, search API, Ka Maps frontend — adapted to the US market: RESO Data Dictionary as the schema reference, USD/sq ft/USPS states, US Census geocoder, and a brokerage partnership pipeline designed to convert sources into direct feeds instead of crawling.

# Pipeline (same philosophy as Immo-Ka)

text
SOURCE → connector → raw ingestion → normalization → property matching
       → deduplication → canonical listing → search index → Home-Ka
  • PROPERTY ≠ LISTING — a Listing is one publication by one source; a Property (properties table, keyed by APN or normalized address) is the physical asset and keeps existing after the listing goes off-market. Every listing is matched on ingest (homeka/propertymatch.py).
  • Change detection — content hash per listing, price history (price_log), grace period (2 syncs) before deactivation, drift detection (volume/price collapse suspends removals).
  • Dedup — precomputed (dup_hidden/dup_of): same MLS number across sources, or same property at ±1% price; most authoritative source wins (sources.authority: direct feed < partner API < site < classifieds).
  • Quality gate — completeness score, quarantine below threshold (published=0).
  • Geocoding — US Census batch geocoder (free, 5k/req) + budgeted Nominatim fallback, persistent cache.

# Connector families (homeka/connectors/)

Family Module Use
reso reso/webapi.py RESO Web API (OData) — MLS Grid, Trestle, Bridge, Spark… the preferred feed
rets rets/client.py legacy RETS (login + DMQL2, COMPACT-DECODED)
xml xml/feed.py XML exports / syndication feeds
json json/feed.py JSON partner APIs, wp-json, internal search APIs
jsonld json/jsonld_site.py schema.org JSON-LD site crawler (bootstrap only)
csv csv/feed.py CSV exports
sftp sftp/drop.py SFTP drops (csv/xml), newest file by mtime
arcgis public_data/arcgis.py county assessor/GIS parcels → feeds properties, not listings
custom custom_broker/*.py one class per site when a family can't express it

A new source of a known family = one row in the sources table (config JSON: URL, field mapping, $ENV:NAME credentials), zero code. Custom connectors auto-register via source_id, exactly like Immo-Ka. Anti-bot policy: direct requests first; auto-escalation chain (Oxylabs → Scrapfly ASP → Bright Data, connectors/_resilient.py) only when blocked.

# Brokerage pipeline (/admin/brokerages)

text
discover brokerage → inspect website → detect IDX/feed/provider
→ find public contacts → classify connector → score → admin
  • Seed: 630 real US brokerages (data/brokerages_seed.json, python run.py seed).
  • homeka/discovery.py inspects each website (direct GET): IDX platform fingerprints (kvCORE, iHomefinder, IDX Broker, Sierra, REW, Delta Media, Ylopo…), RESO/RETS hints, MLS affiliations, contact/partnership emails; then scores feed_probability_score and priority_score (size × feed probability × coverage).
  • Partnership statuses: prospect → to-contact → contacted → in-discussion → feed-received → live (linked to a sources row).

# Run

bash
python run.py serve 8099          # API + frontend
python run.py sync [source ...]   # one-shot ingestion
python run.py watch 60            # sync loop + geocode + img audit + discovery
python run.py seed                # load brokerage seed
python run.py discover 25         # inspect next prospects
python run.py list | geocode | quality | dedup | rescore

Admin: /admin (overview), /admin/sources, /admin/connectors, /admin/brokerages — optional HOMEKA_ADMIN_TOKEN (.env) required as X-Admin-Token.

# Deployment

  • Node M4M36 (~/apps/home-ka), port 8099, PM2 home-ka-web + home-ka-sync + home-ka-ngrok (--url=www.home-ka.com).
  • Frontend: Vite build (frontend/, ka-maps vendored in vendor/ka-maps).
  • Git: origin = spbgit (gitsrv:srv/git/home-ka.git).

# Adding a RESO feed (example)

python
from homeka import db
con = db.connect()
db.upsert_source(con, "mlsgrid_actris", "MLS Grid — ACTRIS", "reso", config={
    "base_url": "https://api.mlsgrid.com/v2",
    "token": "$ENV:MLSGRID_TOKEN",
    "filter": "StandardStatus eq 'Active'",
    "expand": "Media", "originating_system": "ACTRIS",
}, authority=0, states=["TX"])