How Ari Actually Teaches You (Part 2): The Brain That Watches
This is Part 2 of three. In Part 1, we talked about teaching Ari to teach — the Socratic disaster, the wall of text problem, the mobile keyboard eating the screen. All of that works inside a single session. But the moment a student closes the tab and comes back tomorrow, everything resets.
We needed a second brain. One that watches the conversation after it ends and decides: what did this person actually learn? What approaches worked? What misconceptions showed up? We call it the Extractor. Building it nearly broke our server.
The Obvious Solution That Didn't Work
The first idea was simple. When a student finishes a session, run AI calls that read the entire conversation and extract the important stuff — mastery level, knowledge gaps, what teaching approaches worked, learning style observations. Save it to their profile. Next session, the Tutor reads that profile and adapts.
So we ran extraction at the end of every session. Three separate AI calls in parallel — one for topic progress, one for learning style, one for engagement patterns.
The server crashed. Not immediately — it worked fine with one user. But when multiple students were active simultaneously, those parallel calls stacked up. Each one takes 3-5 seconds. Three per student, multiple students — suddenly the web server is juggling a dozen long-running AI calls and running out of threads.
The Back Button Disaster
Users exposed another problem immediately. A student finishes Topic 3, extraction starts, and they hit the back button. Or they click into Topic 4. Now the system is extracting the old session while starting a new one — same user, same profile. Race conditions everywhere.
Some students would rapidly switch between topics. Click one, read the preview, go back, click another. Each click triggered extraction. The system was doing work nobody needed, burning compute, and occasionally writing stale data on top of fresh data.
Making It Synchronous Made It Slow
Next attempt: run extraction when the student opens their next session. Before Ari greets you, process your last conversation. No race condition — extraction happens at a controlled moment.
But now you click "Start Topic 4" and wait. Three extraction calls running. The Tutor can't greet you until the Extractor finishes. We added a loading screen, but 8-10 seconds of waiting before your tutor says hello is a terrible experience.
Sonnet Was Too Slow
Part of the problem was the model. We used Claude Sonnet for everything — teaching AND extraction. Sonnet is brilliant for real-time teaching because it needs nuance and creativity. But for extraction, you're asking for structured JSON output. "Read this conversation, output mastery level and misconceptions." That doesn't need Sonnet's full reasoning power.
We switched extractors to Haiku. Same structured output quality for this task. Response time dropped from 4-5 seconds per call to under 2 seconds. Still not instant, but no longer painful.
The Cron Job
The real fix came from changing our mental model. Extraction doesn't need to happen immediately after every session. It needs to happen before the data matters — which is the next time you open that same topic.
We wrote a script that runs every 8 hours on the server. It scans for idle sessions — no activity for 3 hours or more — moves them to a pending queue, and extracts them quietly. No user waiting. No server spikes. By the time you come back tomorrow, your profile is already updated.
But if you come back before the cron runs and start the same topic again, we still need fresh data. So when you start a new session on a topic you've studied before, the system extracts your previous session for that topic first — one call, blocking but fast on Haiku. Different topic? No extraction needed. Hit back or switch rapidly? We skip it. The cron picks it up later. No more race conditions.
What the Extractor Actually Learns
Two specialized extractors now run on every completed session.
The first looks at topic progress. Did the student demonstrate understanding, or just agree with Ari's explanation? What subtopics got covered? Are there misconceptions — things the student genuinely believes that are wrong? How many times has the same misconception appeared? What teaching approaches were tried, and which ones worked?
The second looks at the person. How does this student learn? Do they respond better to examples or theory? Do they engage with humor or prefer it direct? Do they catch their own mistakes? What are their interests — things from their real life we can connect lessons to?
Both write to the same profile. Both are additive — merging new observations with existing ones, never overwriting. After a few sessions, the profile is rich enough that the Tutor feels genuinely personal.
The Misconception Problem
One thing we're still refining: distinguishing real misconceptions from casual errors. If a student types 4 when the answer is 5, is that a fundamental gap in understanding, or did they just rush? Early versions flagged everything. The misconception list grew noisy.
Now we track frequency. A belief only becomes a flagged misconception if it shows up multiple times across sessions. One wrong answer is a mistake. The same wrong reasoning three sessions in a row is worth addressing.
The Loop
You chat with Ari. The Tutor teaches, adapts, asks good questions. You close the tab. Hours later, the Extractor reads your conversation silently — updates your mastery, logs what worked, notes what confused you. Next time you open Ari, the Tutor reads that updated profile. It knows what approach to start with. It doesn't repeat what failed last time.
Every session makes the next one better. Not because the AI is magically smarter — but because a dedicated brain is doing the analytical work between sessions, so the teaching brain can focus entirely on you in the moment.
But tracking your day-to-day progress still doesn't answer the ultimate, high-anxiety question every student eventually faces: are you actually ready to pass the exam?
That's Part 3.