Repo as canonical store — flip

Goal

Ship the user-visible architectural flip from plans/repo-as-canonical-store.md: locally-generated session IDs (<INITIALS>-YYMMDD-NN, atomic-create), Notion-silent /handoff and /resume, rich session content per the agreed schema, concurrency-hardened telemetry hook, and a one-shot interim manual sync script that bridges to Plan 2. Absorbs all the work from the superseded operator-aware-handoff-resume workstream.

Tasks

• [x] P0. Build scripts/lib/frontmatter.py (stdlib YAML read/write; deterministic content_hash) + 5 scripts/schemas/<kind>.schema.json files + scripts/validate_frontmatter.py. Wire as pre-commit hook (warning mode).
• [x] P1. Create .claude/operators.yaml (3 Roman emails + 1 Sergei email, with initials RT/SK). Build scripts/session.py with subcommands whoami + claim (consolidated resolver; cached operator slug; explicit YAML-template error message). Add iCloud-path warning + mkdir -p docs/sessions/. Atomic-create primitive writes a claim stub (---\nclaimed_at: …\noperator: …\nkind: session\n---\n) so abandoned claims are skip-able.
• [x] P2. Build scripts/hooks/session_state.py — PostToolUse hook, per-session JSON state file, fcntl.flock exclusive lock + unique per-invocation tmp filenames + orphan sweep. Canonical-key aggregation: every mcp__notion_offplan__* tool name folds into tool_calls_breakdown.mcp__notion_offplan. Register in committed .claude/settings.json (NOT .local.json). Unit-test the 50-parallel race + canonical aggregation.
• [x] P3. Rewrite .claude/commands/handoff.md per Plan 1 § Phase 3 spec — Steps 0 (resolve+claim) → 0.5 (operator-confirmation guardrail) → 0.7 (first-handoff hint) → 1 (interactive + --quick flag + auto-detect trivial) → 2 (read state; default {mcp__notion_offplan: 0} if missing; warn) → 3 (git-stats) → 4 (write markdown + loud WARN if invariant violated) → 4.5 (regen INDEX) → 5 (workstream Session Log; warn-only on missing workstream) → 6 (git add explicit pathspecs + git status --porcelain warn on dirty unrelateds) → 7 (commit + push with explicit failure messages) → 8 (move state file to .claude/state/archive/). Remove all blocking Notion calls.
• [x] P4. Rewrite .claude/commands/resume.md per § Phase 4 spec — Steps 0 (resolve operator) → 1 (glob both <INITIALS>-*.md AND CONV-*.md filtered by frontmatter operator: <slug>; merge sort; empty-result branch with explicit "no prior sessions for <slug>" message) → 2 (surface last session + resume prompt) → 3 (workstream partition by owner; cache result) → 4 (peer activity footer from merged scan) → 5 (assemble dashboard). Remove all Notion query call sites. Write .claude/state/resume-cache.json (gitignored) for <5s perf target.
• [x] P5. Lock the rich session-note schema per § Phase 5: mobile-reorderd frontmatter (title/operator/summary/workstream/tags at TOP; telemetry/sync-metadata at BOTTOM), body sections in agreed order, content_hash canonicalisation reference in scripts/lib/frontmatter.py, sync_status state machine documented in docs/conventions/frontmatter.md. Minimum-viable bad-day branch (--quick or auto-detect): Summary + Resume Prompt only.
• [x] P6. Create 4 Bash-driven variant wrappers: .claude/commands/{handoff-rt,handoff-sk,resume-rt,resume-sk}.md. Each invokes a literal OPERATOR=<slug> Bash command (NOT a prose instruction to the model). End-to-end verify /handoff-sk from a roman-account session produces SK-…md before any backfill commit.
• [x] P7. Schema upgrade for workstream + plan + ADR + learning templates. Update workstreams/TEMPLATE.md with owner: + sync-metadata + summary: + tags:. Document plan/ADR/learning conventions in docs/conventions/frontmatter.md. Create folders: docs/sync/.gitkeep, docs/learnings/.gitkeep, .claude/state/.gitkeep, .claude/state/archive/.gitkeep. Update .gitignore for state files.
• [x] P8. Write + run scripts/backfill_frontmatter.py — non-destructive, idempotent, preserves all existing non-standard keys (especially notion_page_id: pending-reconcile, notion_session_id_note, filename_note, date_started). Filename-derived external_id for sessions (3 distinct IDs for CONV-30 trinity). Add owner: to 11 existing workstreams (5 R / 6 S). legacy_notes: [] consolidation. Atomic single-commit + BACKFILL_LOCK. Run twice and git diff --quiet. Archive scripts/md_to_notion.py → scripts/_archive/. Mark v1 plan + workstream status: superseded.
• [x] P9. Build scripts/sync_to_notion_oneshot.py — interim manual sync script (Plan 2 reference impl). Rate-limited 3 req/s + 429 retry. --dry-run default. Writes docs/sync/LAST_SYNC.md audit log. Roman runs on demand for stakeholder Notion freshness.

What's Next

WORKSTREAM DONE 2026-05-11. All 10 phases (P0–P9) shipped in a single working day. 132/132 tests green throughout; idempotent backfill verified; /handoff + /resume Notion-silent at runtime (tool_calls_breakdown.mcp__notion_offplan = 0 falsifiable invariant); Phase 9 interim sync ready for Roman to invoke on demand (python3 scripts/sync_to_notion_oneshot.py --apply). Next P1: repo-as-canonical-store-vault — .obsidian/ baseline, INDEX.md MOCs, link normalisation; blocked-by was this workstream, now unblocked. Separate plan 2 follow-up: port scripts/sync_to_notion_oneshot.py to Cloudflare Worker + cron (TypeScript). No remaining work in THIS workstream.

Key Context

• Plan: repo-as-canonical-store (ratified 2026-05-11; score 9.6/10 after council review).
• Supersedes: the v1 workstream operator-aware-handoff-resume.md (P0, never built). All 12 v1 steps absorbed into P1–P8 above.
• Architecture invariant: every session file must show tool_calls_breakdown.mcp__notion_offplan = 0. Hook flattens all mcp__notion_offplan__* tool names into this single canonical counter. /handoff Step 4 prints a loud WARNING if non-zero. Self-policing.
• Hook concurrency: atomic-rename alone is INSUFFICIENT — flock is required. The 50-parallel test must pass before P2 is marked done.
• CONV-30 trinity: 3 legacy files (CONV-30.md, CONV-30-buyer-journey.md, CONV-30-cloudflare.md) get 3 distinct filename-derived external_ids. NEVER collapse them.
• Backfill must preserve notion_page_id: pending-reconcile on LOCAL-2026-05-11-1.md — load-bearing reconciliation marker.
• Variant wrappers are Bash-driven — env var must be literal in the command line, not a prose instruction. Verify end-to-end before backfill.
• Plan 2 platform pre-decided: Cloudflare Workers cron (not GitHub Actions — 60-day inactive-repo throttling is silent-failure).
• Phase 9 interim sync is the Plan 2 reference impl in Python; ports to TypeScript later.
• Realistic estimate: 2–3 /build sessions for this workstream. Don't compress.
• Council review findings (full disposition in plan): 5 CRITICAL + 19 HIGH fixed in plan v2; 10 MEDIUM acknowledged in Risks; 4 deferred (e.g. /resume caching beyond 200 sessions, hook-language ADR, Better Markdown Links install automation).

Session Log

• RT-260511-02 (2026-05-11): P4 + P5 + P6 + P7 + P8 + P9 SHIPPED — workstream CLOSED. All 10 phases (P0–P9) of repo-as-canonical-store-flip now done in a single working day. P4: /resume v3.0.0 Notion-silent + cache-first + 4 helper subcommands (scan-sessions, partition-workstreams, write-resume-cache, read-resume-cache) + 26 new tests; handoff.md Step 8 cache refresh (a.5). P5: docs/conventions/frontmatter.md 263-line canonical contract (11 H2s; lifts Phase 5 schema; documents content_hash recipe, sync_status state machine, body section order, nullability rules). P6: 4 Bash-driven variant wrappers (handoff-rt, handoff-sk, resume-rt, resume-sk) — literal OPERATOR=<slug> env prefix; prose-only would fall through to git-config. P7: workstreams/TEMPLATE.md + docs/sessions/TEMPLATE.md upgraded to Phase 5 shape; folder placeholders for docs/sync/, docs/learnings/, .claude/state/archive/; convention doc extended with plan-body ## Changelog + ADR external_id rule. P8: backfilled 102 files (19 workstreams + 41 sessions + 25 plans + 17 ADRs) in one atomic commit via scripts/backfill_frontmatter.py; idempotent on re-run (zero diff); CONV-30 trinity preserved as 3 DISTINCT external_ids; notion_page_id: pending-reconcile marker on LOCAL-2026-05-11-1.md kept verbatim; owner mapping derived from git log --diff-filter=A --follow original author (8 Roman + 9 Sergei — plan's prose count was outdated); scripts/md_to_notion.py → scripts/_archive/; .gitignore extended for .claude/state/archive/ + .claude/memory/ + BACKFILL_LOCK (surfaced by P8 pre-flight). P9: scripts/sync_to_notion_oneshot.py 776 lines stdlib urllib (no requests dep — 1:1 portable to Cloudflare Worker TS fetch()); rate-limit 3 req/s + 429 Retry-After honouring + exponential 5xx backoff; --dry-run default; last-chance POST dedupe (query by external_id even on action=post); writes notion_page_id back into local frontmatter on POST success; per-file errors stamp sync_status: error + sync_error: <msg> for next-pull visibility. 132/132 tests green throughout. 17 commits on top of 7454811. 6 sub-agents dispatched (parallel waves: P4 ×2, P5+P6 ×2, P8 ×1, P9 ×1). Surface area: status flipped active → done; next P1 unblocked is workstreams/repo-as-canonical-store-vault.md.
• RT-260511-01 (2026-05-11): P3 SHIPPED + first /handoff in new format. .claude/commands/handoff.md rewritten to be Notion-silent (v3.0.0). Old Steps 4 (Notion Sessions POST + CONV-N read-back) and 7 (Notion Learnings POST) removed entirely. New step sequence: 0 → 0.5 → 0.7 → 1 → 1.5 → 2 → 3 → 4 → 4.5 → 5 → 6 → 7 → 8. The high-leverage UX preserved verbatim: Resume Prompt accept/edit/skip dialogue (now Step 1.5 with stricter 80-150-word + Confidence-tag spec), live-progress ▶ Step X banners + closing checklist + Resume Prompt echo, transcript-archive to Dropbox (filename changed to <INITIALS>-YYMMDD-NN.jsonl). Step 4.5 (regen INDEX / paired HTML / CHANGELOG.md) lands as try-and-skip — regen_indexes.py + render_md.py arrive in W2 vault; the CHANGELOG append works today via git log --since. Helper: scripts/handoff_helpers.py (~700 lines) with 4 subcommands — git-stats (commits + lines + touched-workstream/plan/ADR slugs), telemetry-read (always emits mcp__notion_offplan as int, never null — invariant preserved), write-session (validates against session.schema.json before write; recomputes + embeds content_hash so the file is self-consistent on first write; emits canonical Phase 5 mobile-reorderd key order), archive-state (moves .json + .lock into .claude/state/archive/; idempotent). Tests: 23 new in tests/test_handoff_helpers.py + 1 new regression in tests/test_frontmatter.py = 106/106 across the suite. Three bugs caught + fixed by end-to-end smoke test, NOT by tests alone: (1) scripts/lib/frontmatter.py writer str()-stringified nested dicts (added block-style YAML mapping emit + reader peek-ahead for block-style dicts + flow-style {k:v} parser); (2) session.schema.json rejected legitimate-null fields on empty/quick sessions (relaxed first_commit_at, last_commit_at, active_min_estimate, handoff_write_min to ["type", "null"]); (3) git diff <iso>..HEAD is invalid syntax — git log --since=<iso> accepts ISO timestamps. Reviewer pass on the new handoff.md found 1 BLOCK + 7 CONCERNs; BLOCK + 3 CONCERNs fixed before commit, rest deferred. Parallelism: 3 sub-agents on reconnaissance (handoff_helpers builder + corpus survey + old-to-new step mapping) + 1 sub-agent on reviewer pass over the handoff.md. 3 logical commits on top of cde35f6.
• LOCAL-2026-05-11-4 (2026-05-11): P2 SHIPPED. scripts/hooks/session_state.py — PostToolUse hook, per-session JSON state at .claude/state/session-<id>.json. Concurrency model uses a separate lock file (session-<id>.lock) with fcntl.flock(LOCK_EX), NOT a lock on the state file itself (which would be lost on os.replace inode-swap — the original plan snippet would have lost updates under contention; corrected in implementation). Unique tmp filenames (session-<id>.json.<pid>.<ns>.tmp), atomic rename via os.replace, orphan tmp sweep >1h on every invocation. Canonical-key aggregation: every mcp__notion_offplan__* collapses into tool_calls_breakdown.mcp__notion_offplan (the falsifiable architecture invariant) with per-method forensics under tool_calls_breakdown_methods. subagents_dispatched increments on both Agent AND Task tool names. parallel_terminals_observed counts sibling session-*.json files modified within 1h (excluding self). Hook NEVER blocks the agent loop — every exception path logs to .claude/state/hook-errors.log and exits 0 unconditionally. Registered in committed .claude/settings.json under hooks.PostToolUse (timeout 3s, NOT .local.json). Tests: 22 new + 60 from P0+P1 = 82/82 pass — includes a 50-process subprocess race (tool_calls_count == 50 exactly), a 50-process canonical-aggregation race, poisoned-state-file recovery, garbage-stdin / empty-stdin / unwritable-state-dir all exit 0, orphan tmp sweep, parallel-terminals observation, session-id stdin→env→PPID fallback chain. End-to-end smoke test through the real hook script produced exactly the shape /handoff Step 4 will assert on. 4 logical commits on top of 153a54e.
• LOCAL-2026-05-11-2 (2026-05-11): Workstream created from ratified repo-as-canonical-store plan (9.6/10 council score). Absorbs all work from superseded v1 operator-aware-handoff-resume workstream. Plan amended through v1.2 (Phase 11c lifecycle hygiene + scale-forward; Phase 11b render_md.py <meta> emit spec; Phase 7 plan-body Changelog convention). No /build work yet — ready to start at P0 next session.
• LOCAL-2026-05-11-3 (2026-05-11): P0 + P1 SHIPPED. P0: scripts/lib/frontmatter.py (stdlib YAML subset; LF-normalised SHA-256 content_hash; flow-style canonical writer); 5 draft-07 JSON schemas under scripts/schemas/; scripts/validate_frontmatter.py (mini-validator: type/required/properties/enum/const/pattern/items, JSON-pointer error paths, --strict flag); scripts/hooks/pre-commit (warn-only during dogfooding; promotion to blocking on 2026-05-18); scripts/setup.sh Step 5 installs the hook. P1: .claude/operators.yaml (RT/SK); scripts/session.py (whoami + claim subcommands; resolution chain --email → $OPERATOR → $CLAUDE_USER_EMAIL → git config → halt; O_CREAT | O_EXCL atomic retry 1..99 with claim stub; mtime-busted JSON cache at .claude/state/operator.cache; iCloud / FileProvider path warning; hand-rolled YAML parser with line-numbered errors); CLAUDE.md iCloud subsection; .gitignore .claude/state/* patterns. Tests: 60/60 pass (26 frontmatter + 13 validate + 21 session, including a real 20-process subprocess race that asserts 20 distinct stdouts + 20 files on disk). Drift discovery: 97 files missing kind: (24 plans + 19 workstreams + 38 sessions + 16 ADRs) — zero non-kind violations anywhere, so Phase 8 backfill is purely mechanical. Reviews: general-purpose audit PASS; Anthropic /security-review PASS (no findings). Work parallelised heavily: 3 sub-agents on P0 (schemas/validator/hook); 1 sub-agent on P1 (operators+session.py+tests+CLAUDE.md) + 1 sub-agent on P1 review. 8 logical commits on top of e6f5f1e.