The system map: one day of data

Lesson 0002 · ~20 minutes · companion reference: system map · builds on lesson 0001

Why this lesson: lesson 0001 zoomed into one seam. To judge any seam you need the whole map in your head — which layer owns what, and how a day of data actually moves. This is the step-back view, with X-64 pinned where it belongs. (Stepping back to the entity model before formalizing worked for you in the Telegram course; same move here, one level up.)

1 · The question that organizes everything

When the user taps Next on a teach card, or answers a gap — where does the app remember where they are? Not in the Node process: on Vercel the function instance may be recycled between any two taps. Not in the Telegram message: a message is pixels, and old messages must keep working. The answer, always, is: a Firestore document is the state; the server is a stateless translator; the surfaces are projections. You met this idea in the Telegram course ("code is not storage"); this codebase applies it everywhere, and it is why every flow below is legible as read doc → decide → write doc → repaint surface.

2 · Three layers, two fronts

Front A — Telegram (push + interact)

Teach cards, practice buttons, nudges. Bare text = Capture; everything else is an inline-keyboard callback carrying coordinates like ans:2:1:0 — never free text (Bot API: CallbackQuery).

Front B — Web (pull, read-only-ish)

Three plain Nuxt pages calling /api/* with useFetch. No state library, no client Firestore SDK. Practice never lives here — the Ladder page deep-links into Telegram.

▼ both fronts hit the same server ▼

Backend — one Nitro server, three doors

Webhook (/api/telegram/webhook/[secret]) · web API (/api/backlog, /api/ladder…) · cron (Vercel Cron → /api/cron/tick, docs). Inside: Telegram adapters (server/telegram/*), web pipeline (server/pipeline/ladder.ts), and the shared core — pipeline/tick.ts, exercises/* (X-64's grade.ts goes here), domain/*.

▼ firebase-admin, server-side only ▼

Storage — Firestore, everything namespaced by Scope (userId, language)

The state doc (scopes/{id}: dayCounter, tickArmed) · durable lexicon (lexemes, results) · episodic pipeline (backlogItems, portions, sessions/practice, sessions/teach-*). Idiom: document presence is state — a PracticeDay that exists means "day unfinished"; a scope doc that doesn't means "never bootstrapped". (Firestore data model)

Note what is not here: no queue, no cache, no ORM, no client database. One server, one database, two projections. Every architectural question in this repo reduces to "which module, which document, which direction".

3 · Flow A — one engagement day, traced

Red steps write; dark steps only read. Follow where the authority lives at each moment.

3:00 UTC: Vercel Cron GETs /api/cron/tick → maybeTick('cron'). Reads the scope doc and backlog count. The Gate: armed (yesterday completed) AND backlog ≥ 3 AND not ticked today. Fail any → silent no-tick; the pipeline just pauses.
Generation, before any write: Yandex gloss + one LLM call per new Portion + one gap-text call per Exercise. A failure here aborts with Firestore untouched — the user gets a 🔁 Retry button, not a half-ticked day.
write · atomic commitTick — one transaction: dayCounter+1, consume tickArmed, retire the age-8 Portion, create today's Portion + enriched Lexemes, consume BacklogItems, write the new PracticeDay with every Exercise pre-generated (answers all null). It re-checks the Gate inside the transaction, so racing duplicates commit exactly once (X-65).
write · side effects Post-commit: teach push (intro + one card message) + a TeachSession doc keyed by that message's id. If the push fails, the day stays ticked — side effects never un-tick.
The user walks the card: each teach:next callback loads the tapped message's own TeachSession, moves the cursor, edits the message in place. Old cards keep their own sessions — that's why last week's card still works.
write · per tap Practice: each ans:ex:gap:target tap loads the PracticeDay, validates the coordinates against it (stale message? already-filled gap? → safe no-op), writes one answer into answers[], repaints the single practice message.
write · atomic The last gap of the last Exercise: grading runs (X-64: wordStatus over each word's gaps) and commitPracticeCompletion writes Results + deletes the PracticeDay + sets tickArmed — one batch, with a delete-precondition so a double-fired tap can't write Results twice. The reveal doubles as the day's summary. Tomorrow, step 1 finds the Gate open.

4 · Flow B — the web read path

GET /api/ladder → buildLadderView(scope): read dayCounter → list Portions → derive ages (never stored) → keep rungs {0,1,2,3,5,7} → hydrate Lexemes → buildTeachCard each (X-62's shared model — the card is the wire shape). Then the status dot, and here is the X-64 hinge: if a PracticeDay exists, grade its live answers; if not, read today's Results (dayIndex === dayCounter). Same verdict, two sources, one moment of handover at step 7 above — which is exactly why both sides must be one function.

5 · X-64 on the map

Now the issue reads differently than it did in lesson 0001: it is not "move a function", it is a map correction. Grading is consumed by Flow A step 7 (Telegram, writes durable truth) and Flow B step 2 (web, displays live truth). A rule consumed by both fronts belongs in the shared core layer — below both, importing nothing from either. Today it lives inside Front A's adapter, and Front B reaches sideways into it. The refactor moves one band down the stack. That's the whole issue, seen from orbit.

6 · Check yourself

Q1. Mid-practice, the user's phone dies. Three days later they tap ✅ Practice → in Telegram and resume exactly where they left off. What made that work?

The Telegram message still contained the exercise state in its buttons The Node process kept the session in memory and was still warm The PracticeDay document still exists in Firestore — its presence is "day unfinished", and a fresh card is rendered from it

Yes. Buttons only carry coordinates; the process is disposable; the document is the state. Resume sends a fresh card rendered from the authoritative PracticeDay and re-points activeMessageId at it. Calendar gaps are invisible to the pipeline — "pause" isn't even a stored state, it's just the Gate not firing.

Neither the message nor the process can be trusted: callback data carries only coordinates (~20 bytes), and Vercel recycles function instances freely. The answer is the idiom from section 1: the PracticeDay document in Firestore is the state; its continued existence means "day unfinished", and resume just renders a fresh card from it.

Q2. The webhook route answers Telegram with HTTP 200 before processing the update, doing the work detached (X-65). What failure does the fast ack prevent?

Telegram redelivers unacknowledged updates — a slow tick-length handler got redelivered and ran 3–4 times in parallel Vercel bills by response time, so slow responses cost more Telegram drops the bot's webhook registration if responses exceed 10 seconds

Right. Telegram treats a slow response as a failed delivery and retries — so a ~30-second tick handler received the same update several times, in parallel. The fix has two halves: ack fast + run detached (waitUntil), and make commitTick a re-checking transaction so even racing duplicates commit exactly once. Notice the defense-in-depth: the transport fix reduces duplicates; the transaction makes them harmless.

Billing and deregistration aren't the mechanism. Telegram redelivers updates it considers unacknowledged — and a tick-length handler (~30s of LLM calls) outlives the answer window, so the same update ran 3–4 times in parallel (that was X-65). Fast ack + detached work reduces duplicates; the re-checking commitTick transaction makes any survivors commit exactly once.

Q3. Why are Results + delete-PracticeDay + set-tickArmed one atomic batch rather than three writes?

Performance — one round-trip is faster than three Consistency — a crash between separate writes could leave impossible states (Results written but day still "unfinished", or a consumed day that never armed tomorrow's tick), and the delete-precondition makes a double-fire unable to write Results twice Firestore requires related documents to be written together

Exactly. Enumerate the partial states and each is poisonous: Results without the delete = the Ladder could double-count "today"; delete without arming = tomorrow's tick never fires. And the precondition ("delete must find the doc") turns the batch into a natural idempotency lock — the second concurrent completion fails the precondition and writes nothing. Same design as commitTick: the batch is the unit of truth-change.

It's not speed and Firestore has no such requirement. Work through the partial states: Results written but PracticeDay alive → the day is both done and not done; PracticeDay deleted but tickArmed unset → the pipeline silently stops forever. The batch makes truth change all-or-nothing, and its delete-precondition doubles as an idempotency lock against double-fired completions.

Q4. GET /api/ladder finds no PracticeDay document. What does that mean, and where do the dots come from?

The scope was never bootstrapped — the page shows the empty state An error: the tick always writes a PracticeDay, so one must exist Today's practice is finished (completion deleted the doc) — dots come from Results filtered to dayIndex === dayCounter

Right. Absence of the PracticeDay is a meaningful state: completed. The dots switch source to the durable Results for today's dayIndex, and stay "today" until the next tick raises dayCounter. (Never-bootstrapped is signaled by a different absence — the scope doc itself.) This handover between two sources is precisely the moment X-64 protects.

Never-bootstrapped is signaled by the scope doc's absence, and it's no error — completion deletes the PracticeDay by design. Its absence means "today is done", and the dots switch to Results at dayIndex === dayCounter until the next tick. Two sources, one handover moment — the exact seam X-64 makes safe.

Q5. A future feature: a weekly e-mail digest of missed words. Using the map, which layer does its "which words were missed" logic belong to, and why?

Shared core (near grade.ts/domain) — a third front consuming the same verdicts; e-mail is just another projection A new server/email/ module with its own Results query and correctness logic, keeping e-mail concerns isolated The web layer — the Ladder already computes statuses, so the digest should call /api/ladder

Yes — and this is the map paying off. E-mail would be Front C: presentation of verdicts computed once in the shared core. Option 2 recreates X-64's disease (a fourth opinion about correctness); option 3 recreates its wrong-direction seam (one surface leaning sideways on another surface's endpoint). New surface → new adapter → same core.

Careful — option 2 is how the X-64 problem was born (another copy of the verdict logic in a surface module), and option 3 is the wrong-direction seam again, just pointing at the web instead of at Telegram. A digest is Front C: a new thin adapter over the same shared verdicts. New surface → new adapter → same core.

Ask your teacher. Good follow-ups: "walk me through commitTick's transaction re-check line by line", "why is age derived instead of stored", "what would break if practice moved to the web page?"

Reference: system map (keep this one — it's the compressed version) · architecture glossary · serverless bot state (previous course — section 1 is its sequel) · lesson 0001

Next: with the map in hand, candidate #3 (the Tick's Gate as a pure core inside an I/O shell) — you now know every document it reads.