houses: nightly Kamernet + Vesteda scraper, dedupe by address, owner-only /trackers/houses
All checks were successful
Deploy production / deploy (push) Successful in 2m44s
All checks were successful
Deploy production / deploy (push) Successful in 2m44s
- scripts/house-scrape.ts: polite nightly scrape (honest UA, 6-10 s apart, stops a source on 403/challenge), ingest + price history + removals, ntfy alerts on a separate topic (max 5 + summary, first run seeds)
- address_key + duplicate_of: copies on another source link to one listing, manual wins
- /projects/house-tracker -> /trackers/houses (308), owner only (404 otherwise); filters for min m2, rooms, source, Gouda region default; notes, add-by-hand form, total rent, also-on links
- scripts/house-db.sh: run the house-listing CLI as the limited house_cli role
- deploy/systemd/house-scrape.{service,timer}
This commit is contained in:
5
drizzle/20261024140745_house_scrapers/migration.sql
Normal file
5
drizzle/20261024140745_house_scrapers/migration.sql
Normal file
@@ -0,0 +1,5 @@
|
||||
ALTER TABLE "house"."listings" ADD COLUMN "address_key" text;--> statement-breakpoint
|
||||
ALTER TABLE "house"."listings" ADD COLUMN "duplicate_of" integer;--> statement-breakpoint
|
||||
ALTER TABLE "house"."listings" ADD COLUMN "total_price_eur" integer;--> statement-breakpoint
|
||||
CREATE INDEX "listings_address_idx" ON "house"."listings" ("address_key") WHERE address_key IS NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "house"."listings" ADD CONSTRAINT "listings_duplicate_of_listings_id_fkey" FOREIGN KEY ("duplicate_of") REFERENCES "house"."listings"("id") ON DELETE SET NULL;
|
||||
16755
drizzle/20261024140745_house_scrapers/snapshot.json
Normal file
16755
drizzle/20261024140745_house_scrapers/snapshot.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user