Agency Lead Miner

Local-first CLI for discovering and enriching local business leads using Google Places API (New), website audits, transparent opportunity scoring, contact extraction, intelligent dedupe, and HTML/CSV exports.

Stack

• Node.js 20 + TypeScript
• SQLite (file-based, leadminer.sqlite)
• HTTP via undici
• HTML parsing via cheerio
• Optional Playwright screenshots (--screenshots)

Setup

1. Install dependencies:

npm install

2. Configure environment variables:

export GOOGLE_MAPS_API_KEY="your_google_maps_key"
# Optional, only for --llm-pitches
export OPENAI_API_KEY="your_openai_key"

3. Build:

npm run build

4. Run:

npx leadminer run --lat 40.7128 --lng -74.0060 --radius-miles 20 --category dentist --enriched 100 --export-dir ./out

CLI

leadminer run \
  --lat 40.7128 \
  --lng -74.0060 \
  --radius-miles 20 \
  --category dentist \
  --enriched 100 \
  --discover-target 160 \
  --export-dir ./out \
  --max-concurrency 6 \
  --places-qps 5

Defaults

• --radius-miles 20
• --enriched 100
• --discover-target 160
• --category dentist
• center defaults to 40.7128,-74.0060

Flags

• --screenshots: enable browser fallback + thumbnails
• --llm-pitches: generate two LLM pitch variants for top leads
• --top-pitches N: defaults to 20
• --dry-run: uses cached Places data only (website fetches still run unless cached)
• --max-concurrency N: defaults to 6
• --places-qps N: defaults to 5

Category Examples

Dentist:

leadminer run --category dentist --export-dir ./out/dentist

Realtor:

leadminer run --category realtor --export-dir ./out/realtor

Auto repair:

leadminer run --category auto_repair --export-dir ./out/auto

Cost Control Tips

• Lower --discover-target to reduce Places detail calls.
• Lower --enriched to reduce website audits/contact crawls.
• Keep --places-qps moderate to avoid bursty request patterns.
• Use --dry-run to iterate on export/audit logic from cached Places data.
• Re-runs benefit from SQLite caches:
  • Places responses (places_raw)
  • Website-level audits/contacts (website_cache)
  • Place+website audit cache (leads table keyed by place_id + website_normalized)

Output

• out/leads.csv
• out/report.html
• out/screenshots/*.jpg (only with --screenshots)

Testing

npm test

Unit tests cover:

• website normalization
• realtor confidence/status
• opportunity scoring + tiers
• dedupe rules
• viewport/CTA fixture checks