stan/website
1
0
Fork 0
Code Issues 21 Pull Requests Actions Packages Projects Releases Wiki Activity

houses: Leefbaarometer liveability per listing (PDOK geocode + LBM 2024 grid/buurt), min-score filter and sort; store every regional listing (no price caps at scrape time)
All checks were successful
Deploy production / deploy (push) Successful in 2m33s
Details

Browse Source
This commit is contained in:
Stan Runge
2026-10-24 15:40:20 +00:00
parent e95b7da1e3
commit 113eb54cd9
13 changed files with 19150 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
scripts/house-scrape.ts
View File

@@ -12,6 +12,7 @@ import postgres from 'postgres';
import { alertMessage, matchesRule, type AlertRule } from '../src/lib/house-tracker/alerts';
import { citySlug, GOUDA_REGION } from '../src/lib/house-tracker/address';
import { dedupe, ingestListings } from '../src/lib/house-tracker/ingest';
import { scoreListings } from '../src/lib/house-tracker/liveability';
import { kamernetSearchUrl, parseKamernetSearch } from '../src/lib/house-tracker/kamernet';
import { isChallenge } from '../src/lib/house-tracker/pararius';
import type { ScrapedListing } from '../src/lib/house-tracker/scraped';
@@ -29,12 +30,14 @@ const USER_AGENT =
'stanrunge.dev-house-tracker/1.0 (personal use, one run per night; stan@stanrunge.dev)';
const KAMERNET_CITIES = (
process.env.HOUSE_KAMERNET_CITIES ??
'gouda,waddinxveen,reeuwijk,bodegraven,boskoop,schoonhoven,oudewater,moordrecht,zevenhuizen,nieuwerkerk-aan-den-ijssel'
'gouda,waddinxveen,reeuwijk,bodegraven,boskoop,schoonhoven,oudewater,haastrecht,moordrecht,zevenhuizen,nieuwerkerk-aan-den-ijssel'
)
.split(',')
.map((c) => c.trim())
.filter(Boolean);
const KAMERNET_MAX_PAGES = 5;
// Everything in the region is stored (no price/size caps); paging stops early once a page has
// none of our cities, so this is only a safety limit.
const KAMERNET_MAX_PAGES = 25;
// Gouda centre; Vesteda's own search uses km radius around a point.
const VESTEDA = {
lat: 52.0115,
@@ -250,6 +253,16 @@ try {
const { keyed, copies } = await dedupe(sql);
console.log(`dedupe: ${keyed} new address keys, ${copies} listings are copies of another`);
// Leefbaarometer score for new listings and changed addresses (PDOK + the LBM WFS).
try {
const lbm = await scoreListings(sql, { limit: 300 });
console.log(
`liveability: ${lbm.checked} checked, ${lbm.scored} scored, ${lbm.notFound} without a location, ${lbm.failed} failed`
);
} catch (error) {
console.error(`liveability: ${(error as Error).message}`);
}
const rules =
(await sql`SELECT id, name, enabled, offer, min_price AS "minPrice", max_price AS "maxPrice",
min_area AS "minArea", min_bedrooms AS "minBedrooms", cities, property_types AS "propertyTypes"