Personal Learning Agent

What it does

I could not keep up with AI/ML content, and worse, I kept re-watching explanations of things I already understood because I had no record of what I had covered.

So pla ingests the videos from the channels I follow, tags them against a controlled vocabulary of 50 concepts and 45 tools, and answers my questions with citations that link to the exact second in the source video. Then it tracks what I have watched and stops recommending it.

You can drive it from a CLI, from a Discord bot by DMing it a question, or read the whole knowledge base as a linked note graph inside Obsidian.

The feature I actually use

pla plan <video> is the part that changed how I learn.

Given a video, it labels each timestamped segment by whether I already know that material, and tells me which spans to watch and which to skip. A sixty minute talk usually contains about twelve minutes I have not seen before, and this finds them. It logs partial watches, so --until 8:30 records where I stopped and picks up there next time.

That only works because coverage is modelled properly rather than as a watched flag. A rule-based knowledge-tracing layer moves every concept through new -> learning -> covered as I log what I watch, and covered concepts get deprioritised in answers as well as dropped from recommendations.

The measurement I am most glad I ran

I built the retrieval pipeline three times: once plain, once in LangChain, once in LangGraph. Not as an exercise, but because I wanted to know what the frameworks actually cost, and every discussion of it I could find was an opinion.

orchestration overhead, per query
  plain        2.12 ms
  LangChain    2.43 ms
  LangGraph    3.32 ms

end-to-end median, same questions
  plain        5.45 s     1497 output tokens
  LangGraph    5.97 s     1587 output tokens
  LangChain    6.24 s     1842 output tokens

The orchestration overhead is about one millisecond of a five and a half second query. It is noise. Anyone choosing between these on performance grounds is optimising something that does not matter, and I would not have believed that firmly enough to say it out loud without having measured it.

The difference that is real is the token count. LangChain’s prompting produced 23% more output tokens for the same questions, and on a metered API that is the actual bill. So the honest basis for choosing is legibility and token behaviour, not speed. I kept LangGraph, because an explicit state machine makes branching debuggable, and that is worth 0.5 seconds.

What retrieval scores

46 hand-labelled questions against a committed corpus, local bge-small embeddings on CPU, k=8:

recall@8      0.978
recall@3      0.924
recall@1      0.739
MRR           0.855
judge score   4.72 / 5

The number worth staring at is recall@1 at 0.739. Roughly one question in four does not put the right document first, which is fine for a system that reads eight of them before answering and would not be fine for one that reads one. Recall@8 being 0.978 is what makes the design work.

A regression guard runs these in CI and fails loudly if retrieval or answer quality drops, so the numbers cannot quietly rot.

It costs nothing to run

This was a constraint, not an accident. Embeddings are always local sentence-transformers on CPU, no key required. For generation the default provider shells out to Claude Code on an existing subscription rather than billing an API, and Ollama is a fully local alternative. There is also a fake provider that the tests run against, so the suite needs no network and no credentials.

That is also why the whole thing is local-first: the knowledge base is SQLite plus Chroma on my own disk, and the Obsidian export means the output survives the project.

Details that took more thought than expected

Canonicalization. “RAG” and “retrieval augmented generation” are the same concept, and if the tagger treats them as two nodes the graph quietly becomes useless. Every concept carries an alias list matched case-insensitively, and canonicalization runs before anything is written.

Batch safety. Ingest is idempotent and one failing video never kills the run. Polling a channel means processing things that will occasionally be malformed, private, or missing captions, and a pipeline that aborts on the first bad item is a pipeline you stop trusting to run unattended.

What is still open

The eval corpus is committed fixture documents rather than a sample of real ingested videos, so the retrieval numbers describe the retriever more than they describe the lived system. The judge is an LLM, unvalidated against human labels. And one of the 46 questions scores zero recall, which I have not yet diagnosed.

161 tests, CI green.