Competitor Pricing and Market Signal Scrapers

THREE SITES — study each before writing a scraper, because the right
method is per-site, not universal.

PER-SITE LOAD ANALYSIS (do this first, always)
  Site A  Prices in server-rendered HTML
          → plain fetch + parse. Cheapest, most stable. Don't reach for
            a browser you don't need.
  Site B  Prices injected by JS after load
          → check the network tab first — there's usually a JSON API
            behind it. Hit that directly and skip the browser entirely.
          Fall back to headless only if the API is signed.
  Site C  Prices behind a "show pricing" interaction
          → headless, one click, then read. The interaction is the
            fragile part; pin it to a stable selector, not a position.

  The mistake this avoids: rendering all three in a headless browser
  because one needs it. That's 10× the cost and 10× the breakage.

STRUCTURE INTO BUSINESS FIELDS — not raw HTML
  competitor · plan_name · price · billing_period · seat_included ·
  captured_at
  Normalize on the way in: "$29/mo" and "$29 per month" become the same
  record, or your change-detection fires on formatting noise.

VALIDATE BEFORE STORING — the step that prevents false alarms
  • price parses to a number in a sane range
  • plan count matches last run ±1 (a page redesign drops to zero
    fields silently; catch it here, not in a 3am alert)
  • non-empty across all three sites
  A scraper without validation reports the competitor "removed all
  pricing" the first time they change a class name.

SCALE + SCHEDULE
  Pagination auto-followed where tiers span pages. Daily run,
  business hours. Diff against last snapshot, alert only on a real
  price or plan change.

COMPLIANCE
  Check robots.txt and ToS per site, throttle to human-rate, identify
  your agent. Aggressive scraping gets the IP blocked and the data
  stops — politeness is a reliability feature here, not just etiquette.

Give me the three URLs and I'll classify each by load type.