Job Scout

What it does

I was applying to co-ops by opening fifty career pages, reading them, and losing track. Job Scout is the version of that I automated.

It reads a list of target companies, scrapes each one’s applicant tracking system for intern and co-op postings, extracts what each posting is actually screening for, groups the postings into a handful of resume archetypes, and then tells me which of my own projects to feature on each variant.

The output I care about most is the last one, and it is the reason the project exists: a gap report listing the attributes that no project of mine currently demonstrates. It is a list of what to build next, derived from what the jobs I want are actually asking for. Some of the work described elsewhere on this site is on it.

The structural decision

Five stages, and each one writes its output to disk before the next one reads it.

companies.csv -> scout -> parse -> context -> cluster/match -> sync -> Notion

That sounds like an implementation detail and it is the main design decision. Scraping is slow and rate-limited, the extraction stage costs money per posting, and the Notion sync is the flakiest link because it depends on someone else’s API being up. If the pipeline were one process, a failed sync at the end would mean re-scraping fifty career pages and re-paying for every extraction to try again.

Because every stage is durable and independently runnable, a sync failure costs a sync retry. Each stage is also separately testable, which is most of why the test suite is any good.

Things that turned out to matter

Per-company work should be config, not code. Most company career pages are not bespoke, they are Greenhouse, Lever, Workday, Ashby, or Taleo with a logo on top. So the scrapers are thin adapters behind one Scraper protocol, and a company is usually a row in a CSV naming which platform it uses rather than a new Python file. Three platform adapters are built so far.

LLM output gets schema-validated or it is not output. Every extraction is parsed into a Pydantic model. A failure retries once with the validation error appended to the prompt, and if it fails again it is quarantined to a folder for me to look at rather than silently dropped or, worse, silently half-parsed into the next stage.

I built a way to distrust it. parse --spot-check 5 samples five parsed postings and prints each one next to the raw description it came from. Structured output looks correct in a way that unstructured output does not, and the whole point of a JSON schema is that it will happily validate a confidently wrong answer. This is the cheapest possible check against believing my own pipeline.

Nothing is ever deleted. The Notion sync upserts on (company_slug, job_id) and marks vanished postings as Closed instead of removing rows, because the tracker holds application status and notes that the scraper knows nothing about. A sync that can delete is a sync that can destroy the part a human wrote.

It runs on a subscription now

The original version needed a billed Anthropic API key, and parsing every new posting on every overnight run is exactly the sort of recurring cost that makes you quietly stop running a tool.

There are now three auth paths behind one interface: an API key, a Claude Code OAuth token, or a backend that shells out to claude -p headlessly and uses whatever subscription is already logged in. The CLI adapter mimics the SDK’s response shape, so no calling stage knows which one it is talking to, and a near-empty system prompt keeps each call a single-shot transform rather than a full agent boot.

Where it actually is

4,200 lines, 96 tests, CI on Python 3.11 and 3.12. The pipeline runs end to end and the CI also asserts that the suite passes with every credential blanked, because every test fakes its client and I want to know immediately if one starts reaching a real service.

It is not finished, and the honest gaps are these. Three of the five named ATS platforms have adapters. The archetype clustering is produced by a model and I have not measured whether the clusters are stable across runs or whether a different seed produces a different four, which for a project sitting next to an evaluation-first RAG system on this site is a gap I am aware of. And the whole thing is tuned to one person’s job search, which is mine.