A warehouse that refuses to paraphrase

An accuracy-first knowledge system over university unit templates — served to Claude and any MCP client, and to a native macOS / iOS app. Passages come back verbatim or not at all.

Status
Livefly.io · syd
Commits
5221–24 jul
Source
31,310py + swift
Tests
559green
App
1.2.0signed dmg
[S1]

The premise

Design

ATHENA·PREMISE·general academic·single user

General-purpose models are confidently wrong about specific coursework. They will invent a week 6 topic, misattribute a reading, or paraphrase a statute into something that no longer means what it meant. For study, that failure mode is worse than no answer at all.

Athena inverts the arrangement. A unit offering is authored once as a structured FORM — topics per teaching week, learning outcomes, assessments, prescribed readings — then ingested into a dimensional star/snowflake warehouse where the fact grain is a single retrievable text chunk. Each unit becomes its own mini-warehouse; a federation layer fans a query across them.

The model is no longer the source of record. It is a reader with a well-indexed library and a hard rule against making things up. Only curated material is ingested, and every passage carries a reliability tier — so an official unit outline outranks a personal note by construction, not by hope.

README.md · docs/warehouse-design.md · CLAUDE.md

[S2]

The accuracy contract

Enforced in code

ATHENA·INVARIANTS·6 rules·asserted by the suite

Six load-bearing invariants. They live in deterministic Python and are asserted by tests — never left to a prompt, and never to a model's good behaviour.

Non-null citation

No chunk exists without attribution. Where no bibliographic reference is available, one is synthesized from unit + section at ingest. There is no unattributed path.

Verbatim returns

Retrieval never paraphrases. Passages return byte-for-byte with their offsets, and an empty result surfaces as an explicit no source found.

Authority travels

Tier and rank ride on every result: official_unit_outline › published › instructor › student › ai_generated, plus a verified flag.

Idempotent ingest

A source_nk checksum fast-exits unchanged sources; embeddings live in a content-addressed cache. Ingestion never runs inside a query.

No LLM in the core

Neither core/ nor ingestion/ imports a generation SDK — the embedder seam is the sole exception. Generation is confined to rag/.

Bounded payloads

Tool payloads stay under 25k tokens; chunks hit an 800-token ceiling; EMBEDDING_DIM ≤ 2000 for the pgvector HNSW cap, with a drift guard on the schema.

The chat surface adds a seventh, enforced after generation: a stdlib-only cite-integrity validator. It normalises loose [S#] markers, then fatally rejects an unknown reference, a zero-citation answer, or a fabricated verbatim quote of eight words or more — retrying once against a repair instruction before falling back to a purely extractive answer. The Swift client ships a byte-parity peer of the same validator, pinned to shared golden vectors.

rag/grounding.py · rag/chat.py · apple/Sources/AthenaKit/Grounding/CiteValidator.swift

[S3]

Six layers, strictly separated

✓ Live-verified

ATHENA·ARCHITECTURE·L0–L6·228 tracked files

The retrieval core is the durable centre; everything else is an adapter around it. One deployed service exposes both /mcp and /api/v1 over the same retriever — the app is a sibling adapter, never a fork.

ConsumersL6
claude.ai · Claude Desktop · any MCP client · the native macOS / iOS app.
mcp_server/L5 · adapter
Thin FastMCP wiring — five read-only tools, grounding descriptions, payload cap. Auth is none | token | github, the last being OAuth 2.1 with a per-user allowlist and durable Postgres-backed client storage that survives redeploys.
api_server/L5 · adapter
Ten REST routes mounted on the same app, with a fail-closed per-route auth guard and a generated OpenAPI 3.1 document. Bodies are byte-equal to the MCP tool outputs.
rag/L4 · edge
The only layer permitted to touch a generation model. Warehouse registry, federated fan-out, context-pack assembly, the grounding validator, and two independent generator seams — one for answer, one for chat.
core/L3 · retrieval
Stateless and LLM-agnostic. Hybrid vector + full-text search, RRF fusion at k=60, deterministic tie-breaks, the authority ladder, and the store protocol.
StoresL2
InMemoryStore is the executable reference and wins any disagreement; PostgresStore on pgvector is the production path, held to parity by 35 live checks; a persisted SQLite store backs the fully offline app.
ingestion/L1 · async
Off the request path and idempotent: parse → referential-integrity gate → transaction → purge → dims & bridges → tag inheritance → heading-aware verbatim chunking → cached embed → atomic replace-by-source.
InputsL0 · vault
Unit FORMs in YAML, markdown notes with frontmatter, and material ingest across pptx · pdf · html · url — all converging on the same chunk-and-embed tail.

docs/architecture.md · repo-structure.json · db/schema.sql

[S4]

What happens to a question

8 stages

ATHENA·RETRIEVAL·RRF k=60·deterministic

  1. rewrite

    Multi-turn queries are condensed deterministically. No model is invoked to decide what to retrieve.

  2. fan out

    The query goes to every selected mini-warehouse in parallel — cloud and on-device stores can be federated together.

  3. hybrid search

    Dense cosine against pgvector plus real English full-text, each with a relevance floor and over-fetched candidates.

  4. RRF merge

    Reciprocal rank fusion at k=60 across stores, tie-broken on score, authority, ordinal, then chunk id — the same query always returns the same ranking.

  5. rerank

    Authority-first ordering, with an optional Qwen3 cross-encoder pass and MMR diversification, both default-off and user-toggleable.

  6. pack

    Dedup, then fill to the token cap. Each passage gets an [S#] ref and an entry in the citation manifest.

  7. generate — or don't

    By default the pack is handed back extractively for the client model to quote. With a generator configured, synthesis happens server-side under the same rules.

  8. validate

    The cite-integrity ladder runs. It repairs once, then falls back to extractive rather than emit an ungrounded answer.

The ordering matters more than any single stage: retrieval decisions are made in code, generation is confined to the last step, and the last step can always be removed without the system losing its ability to answer.

core/search.py · rag/federated.py · rag/context.py · ingestion/rerank.py

[S5]

Surfaces

5 tools · 10 routes

ATHENA·SURFACES·read-only tools·one retriever

ToolReturns
search_notesFederated hybrid search across selected warehouses, filtered by week, outcome, assessment or tier.
answerA grounded context pack — reranked, deduped, token-capped, [S#]-tagged — with optional server-side synthesis.
get_sourceA verbatim window around a chunk, for reading around a hit without losing provenance.
get_unit_outlineThe unit's navigable structure: topics, weeks, outcomes, assessments, readings.
list_warehousesWhich units are loaded, and how many chunks each holds.

Alongside them, ten REST routes carry the same payloads to the app — reads for warehouses, outline, search, answer and sources; a server-mediated /chat; authed single-flight ingest for unit FORMs and notes; and a best-effort /parse/unit-outline that turns a PDF or markdown handbook into a FORM draft for editing.

mcp_server/tools.py · api_server/routes.py · api_server/openapi.py

[S6]

The app, and the offline problem

v1.2.0

ATHENA·APP·macOS + iOS·145 swift cases

A SwiftUI app for macOS and iOS shares one codebase across both: search with faceted filters, unit outlines, a verbatim source reader, an interactive FORM editor that accepts JSON, markdown or PDF upload, note capture, and chat.

The harder question was what happens without a network. The answer was to stop treating local as a global switch. Each warehouse is independently marked online or offline, and the app federates across both at once — a cloud unit and an on-device unit answer the same query together. The Python backend ships inside the app bundle via PyInstaller, code-signed and sandbox-clean, over a persisted SQLite store, with Ollama serving embedding, reranking and chat entirely on-device.

Chat engines are chosen decisively rather than silently: when a turn falls back from a higher-priority engine, the app says so in the transcript.

ConcernCloudOn-device
StoreSupabase Postgres + pgvectorPersisted SQLite, bundled
EmbeddingOllama baked into the imageqwen3-embedding:0.6b · 1024D
RerankAuthority-first (default)Qwen3-Reranker-0.6B, opt-in
ChatAnthropic or Moonshot seamgemma3:12b via Ollama
NetworkRequiredNone

The interface this page borrows is the app's own, locked after a live mockup review: Liquid Glass surfaces, a monospace terminal layer for structure and citations, New York serif for anything actually read, and Signal Red #FF0800 as the only saturated colour in the product — reserved for interactive accent, [S#] chips, and grounding-status banners. Destructive actions deliberately use the system's own red instead, so Signal Red never doubles as danger.

apple/DESIGN_SPEC.md · apple/Sources/AthenaApp/Theme · docs/local-backend.md

[S7]

Build record

12 phases

ATHENA·PHASES·P0–P11·OS-40…OS-64

PhaseScopeState
P0Scaffold, config, verified dev research● Shipped
P1Dimensional store — in-memory reference + pgvector, held to parity● Shipped
P2FORM ingestion, natural keys, deterministic offline embedder● Shipped
P3Markdown notes + guarded dimension sweep on re-ingest● Shipped
P4Hybrid retrieval over the star, RRF, metadata filters● Shipped
P5MCP adapter — verified live from claude.ai against the deployed server● Shipped
P6Remote deploy: Fly.io syd, Supabase, baked Ollama, GitHub OAuth● Shipped
P7LMS export ingestion — deliberately deferred past operation-ready○ Backlog
P8RAG federation: mini-warehouses, context packs, generator seam◐ In review
P9Native surface: REST adapter + macOS / iOS app◐ In progress
P10Operation readiness: real corpus, retrieval quality gate, ops hygiene◐ In progress
P11Multi-format material ingest — pptx, pdf, html, url● Shipped

Three decisions from the record are worth naming, because each one cost something.

A rejected feature. Conversation-aware retrieval was worth building; conversation-aware source rewriting was not. Letting chat silently re-embed the underlying data would have broken verbatim returns, provenance and determinism at once, and invited model collapse — a system slowly learning from its own output. Only the ranking-side forms were kept.

A parser humbled by reality. The outline parser passed its fixtures and then met real university handbooks, which render numbered topic lists on a single line and structure everything else three other ways. Hardening it against actual exports produced more fixes than the original build.

A bug that only existed in production. A fresh install timed out on first launch and worked on retry. The Fly machine suspends when idle; chat already had a 90-second allowance for cold start, but the read paths were still on 20 seconds — correct locally, wrong against a sleeping machine.

Linear · Olympus Studios · OS-40…OS-64 · git log

[S8]

Figures

✓ Measured

ATHENA·FIGURES·measured 24 jul 2026·not estimated

414Python tests

Green, fully offline. A further 55 are gated on a live Postgres.

145Swift cases

Headless via SwiftPM — no Xcode, no simulator.

31,310Lines

17,526 Python · 13,784 Swift, across 228 tracked files.

52Commits

21–24 July 2026, on the current history.

5 · 10Tools · routes

MCP tools and REST routes over one retriever.

1024Dimensions

Under the 2000 pgvector HNSW cap, with a schema drift guard.

The test suite is the load-bearing number. It runs entirely offline — in-memory store, deterministic local embedder, mocked HTTP — which means the accuracy invariants are checkable on any machine, in seconds, with no database and no API key. The live-Postgres parity checks are opt-in against a real instance, and the in-memory store remains the executable reference: where the two disagree, the reference wins.

pytest — 414 passed, 55 skipped · git ls-files · apple/Tests

[S9]

Open threads

5 open

ATHENA·OPEN·next up·tracked in linear

Real corpus and the quality gate

◐ In progress

The harness exists and passes on fixtures; what remains is the user's own units, an eval set per unit, and locking the embedding model and similarity floor once it clears — in-corpus recall at 90%, out-of-corpus refusal at 100%.

Retrieval optimisation

◐ In progress

Cross-encoder reranking, asymmetric query encoding and chunk-context expansion are built and default-off. Each needs measuring against the gate before it becomes the default, because enabling any of them re-baselines the pinned retrieval vectors.

Server-side synthesis

◐ Awaiting credit

Both generator seams are wired and tested behind mocked transports. Turning them on is a key and an environment variable — no code path changes. Extractive cited answers work without them, by design.

Zero-setup local AI

○ Backlog

The guided setup ships today; bundling the Ollama runtime so a fresh machine needs no manual install is the next step toward genuinely zero-configuration on-device chat.

LMS export ingestion

○ Backlog

Canvas, Blackboard and Moodle exports would plug into the identical resolve-chunk-embed tail. An accelerator, not a gate — the system is fully operational without it.

docs/go-live.md · docs/retrieval-quality.md · Linear OS-50 · OS-56 · OS-58

Athena — University Unit Knowledge Warehouse · Olympus Studios · current to 24 July 2026.
Interface ported from apple/DESIGN_SPEC.md (locked 2026-07-21): flat charcoal #0A0A0C, Signal Red #FF0800 as the sole saturated accent, monospace chrome, New York serif reading layer.
A study and research aid, not an authority: the official unit outline and the cited sources remain the source of truth.
Enquiries — ashden@apollo-industries.org