How the research agent works

When you ask the chat “what is reincarnation?”, a lot has to happen before a single word streams back. The answer must come from a closed, curated body of material — Śrīla Prabhupāda’s lectures, the scriptures, the commentaries, the letters — and every claim must point at something you can open and read for yourself. No improvising, no answering from what the model happened to memorise in training and passing it off with a straight face.

We have written before about the honesty rules this demands — the layered defense against invention, the answers that cite their sources, the citation a model cannot fake. This article is about how those rules actually work — the research agent between your question and the answer, and the handful of decisions that shape it.

To keep it concrete, we’ll follow one question all the way through: what is reincarnation? At each step you’ll see the actual numbers — what scores what, what survives, and what gets thrown out. (The values are representative — the shape of a real trace, not any one reader’s data.)

An agent, not a loop

The first version was a classic agent loop. The model would think, call a search tool, read the result, think again, search again — the pattern the field calls ReAct. It works. But for a job whose steps you can name in advance, that freedom is expensive: the model re-decides the obvious on every turn, latency piles up, and two identical questions can wander down two different paths.

So we took the autonomy away. As Anthropic argues in Building Effective Agents, an autonomous agent — one that picks its own next step — earns its keep only on open-ended tasks you can’t lay out in advance. When you already know the steps, a fixed workflow — model calls wired together by code you control — is more reliable and much cheaper. So the research turn is a deterministic graph (built on LangGraph) where the model is called only for the steps that genuinely need judgment — planning the search, breaking the answer into theses, writing the prose — and code runs everything in between.

flowchart TD
  START(["user turn"]) --> CC{{"deterministic classifiers<br/>before the LLM"}}
  CC -->|"simple, direct request"| SV["answer straight away"]
  CC -->|"needs research"| R{{"LLM router<br/>classify intent"}}
  R -->|"research, unknown"| RW["research worker"]
  R -->|"locate, catalog…"| OTHER["other workers"]
  RW --> SP["synthesis planner"]
  SP -->|"grounded plan"| SY["synthesizer streams the answer"]
  SP -->|"corpus empty"| CF["out-of-corpus fallback"]
  CF --> SY
  OTHER --> SY

  classDef grnd fill:#89b4fa,stroke:#6c7086,color:#1e1e2e;
  class RW,SP,CF grnd;

Two small decisions are already visible. A layer of cheap deterministic classifiers runs before the LLM router and skips the model entirely for simple, direct requests — a bare BG 2.13 is just shown as-is. Our question isn’t one of those: what is reincarnation? is classified as research and takes the top path through the graph. And an unknown intent never answers empty-handed: it falls through a light research pass, so a single misclassification never turns into a confident “not found.”

How hard should we look?

Not every question needs a full sweep of the corpus. So the agent first asks itself a cheap question: do we already have an answer an editor curated by hand? This borrows from two ideas: Corrective-RAG (check what you already have before retrieving more) and Self-RAG (judge whether the evidence is enough before answering). We do it with no extra model call at all, from signals already computed:

# research/sufficiency.py — decide the effort level from evidence in hand
def assess_sufficiency(question_matches, memory) -> str:
    if question_matches:                 # an editor pinned THIS answer to THIS question
        return CORRECT
    if memory_is_sufficient(memory):     # a strong curated note with ≥3 resolved refs
        return CORRECT
    return INCORRECT                      # nothing curated → do the wide sweep

The result is just an effort level. Lean takes the curator’s chosen sources plus a small extra search. Wide runs the full search across the corpus, up to two rounds. What is reincarnation? has no pinned answer and no strong memory match, so assess_sufficiency returns INCORRECT and it takes the wide path. Deciding this before the search matters: a question the curator already answered skips the wide sweep — ~194 sources and ~20 seconds — and resolves from ~56 sources instead. More on speed is a separate article: where the thirty seconds went.

Two ways to search, then a shortlist

When it does search, the agent searches two ways at once. Vector search matches by meaning: every passage is turned into an embedding and compared to the question’s over pgvector’s index. It is great at concepts but blind to exact strings — a verse number like BG 2.13 (numbers don’t embed), a Sanskrit term like yoga-kṣema, a short verse that scores low. So a second, lexical search — plain full-text matching — runs beside it, and its hits go into the pool no matter their vector score. The lexical search’s only job is to get the right passage into the running; whether it belongs is decided next.

For what is reincarnation? the planner fans out a few angles — the definition, the soul changing bodies, what happens at death — and the two searches come back with a mixed pool: a couple of lecture clips, the verses BG 2.13 and BG 2.22, a purport, a short verse the lexical lane forced in on the word punar-janma, and some noise.

Now, which of these belong? That’s decided by a reranker, and here the two kinds of model matter. A bi-encoder (Sentence-BERT) turns the question and each passage into vectors separately — you embed every passage once, up front, so searching is just comparing numbers, cheap enough to run across the whole corpus. That is the search step above. A cross-encoder instead reads the question and a passage together, in one pass: far more accurate, but because it depends on the specific question, you can’t precompute it over the corpus the way you can a bi-encoder — you can only run it on a shortlist. So you use both — the bi-encoder cheaply narrows the corpus to a shortlist, then the cross-encoder (Voyage’s rerank-2) re-scores just that shortlist. Here is the whole step for our question — the pool the searches returned, and what the reranker did with it:

q: "what is reincarnation?"   intent=research   path=wide

cosine  kind        passage                                  rerank  kept
 0.63   lecture     "the soul changes bodies like clothes"    0.94    yes
 0.71   verse       BG 2.13  dehino 'smin yathā dehe…         0.88    yes
 0.66   commentary  purport on BG 2.13                        0.81    yes
 0.34   verse       BG 2.20  na jāyate mriyate vā…  (lexical)  0.71    yes
 0.68   verse       BG 2.22  vāsāṁsi jīrṇāni…                  0.60    yes
 0.47   lecture     "temple management tips"                  0.21    no
 0.12   —           "quantum consciousness…"      below floor   —      no

Two things stand out in that block. The plain lecture ends up above the strong verse BG 2.13, because read together with the question it answers more directly — so the order changed. And the reranker overrides cosine both ways: it rescues BG 2.20, a dead-on verse that vector search had almost buried, and throws out “temple management,” which looked fine to vector search but is off topic. The off-topic chunk never even reaches the reranker — a permissive noise floor drops it first.

One bias is worth guarding against, though. A cross-encoder trained on prose tends to rank a chatty lecture transcript above a terse verse — that’s BG 2.20 in the block: dead-on, but with a cosine so low it could easily fall off the bottom. To stop scripture from being squeezed out, the cut keeps per-kind reserves: at least two verses and two book passages always survive, each still above a relevance floor so nothing off-topic is forced in. Which kinds make the cut and what order they appear in are two separate jobs:

flowchart LR
  RANK["reranker: best by relevance<br/>mostly lectures"] --> RES{"any verses and<br/>book passages?"}
  RES -->|"yes"| DONE["final set"]
  RES -->|"no, but some clear the floor"| PULL["pull them in"]
  PULL --> DONE

The full retrieve-and-discard story, with the numbers, is in answers that cite their sources.

Two scores, two jobs

Notice the block had two score columns, not one. That is deliberate: every candidate carries two scores, and they are never mixed, because they do two different jobs:

• Cosine — the vector similarity, on a stable [0,1] scale. This is the gate score: every yes/no decision about whether a passage is good enough reads cosine, so the same number means the same thing everywhere.
• The rerank score — the cross-encoder’s relevance. This is the ordering score: it decides what comes first within one answer, and nothing else.

Take BG 2.20 from the block: a low cosine, a high rerank score. The rerank score earns it a place in the ordering. But the coverage gate — the check that asks did we find enough to answer at all? — reads only cosine, where that low number counts for little; it looks instead at the top of the set, sees a confident hit plus two lectures, and stops after one round. Same passage, two numbers, two jobs.

Why keep two scores instead of one? Because rerank scores are not comparable between questions — a 0.7 on one question and a 0.7 on another don’t mean the same thing, so you can’t set a fixed threshold on them. Cosine you can. So every threshold reads cosine, ordering reads the rerank score, and the two are kept in separate tiers so they are never compared directly:

# research/pipeline.py — merge without ever comparing the two scales
def _tier_key(e):
    rs = e.get("rerank_score")
    if rs is None:
        return (1, e.get("score") or 0.0)   # tier 1: curated refs, by cosine
    return (0, rs)                           # tier 0: reranked chunks, by rerank score

What the editor already knows

The agent’s strongest signal isn’t search at all — it’s an editor who already decided the answer. Those decisions live in attributions, of three kinds: a pinned answer (a ready reply the editor tied to a specific question), a boost (a topic that lifts related material), and a memory (a background note — extra context that informs the answer but is never quoted). Our reincarnation question had none pinned, which is why it took the wide path; a question like what is the soul? often does, and skips most of the search.

That signal is powerful, so we double-check it. Handing back a curated “this is the source” answer on a weak match would be the worst thing this product can do — presenting a wrong answer as if a trusted source confirmed it. So a borderline match is re-checked by the cross-encoder against the user’s actual question, and if it doesn’t confirm, we drop it. We would rather miss a curated answer than fake one.

One more thing, about languages. Our embedder, text-embedding-3-small, scores about 62% on English retrieval but only ~44% on the multilingual MIRACL benchmark. That gap means the same question in Russian — что такое реинкарнация? — lands with a cosine handicap: its verses come back a few points lower than the English run. So the accept thresholds are set lower for a translated query to compensate — a pinned answer needs 0.85 cosine in English but 0.80 from a translation. How the same chat serves readers in languages the corpus was never written in is its own story: one chat, every language.

From notes to an answer

Retrieval hands over a slate of passages; grounding turns them into an answer where every claim is backed by a source. The agent plans first: a planner model breaks the question into a few theses — one clear claim each. Which note actually supports which thesis is decided by the reranker, not the planner, because embeddings judge relevance better than a model skimming headers, and it ranks against the thesis sentence itself, not the raw question:

thesis 1  the soul is eternal, distinct from the body   BG 2.13, lecture   strong
thesis 2  at death it moves to a new body               BG 2.22            strong
thesis 3  actions shape the next birth                  best note 0.41     thin

Thesis 3 comes back thin. That is the CRAG trigger: rather than write on weak evidence, the agent runs one more focused search for that thesis alone, re-ranks, and tries again. Deliberately conservative: per-thesis, only when needed, never chained. Meanwhile the answer’s opening paragraph — rewritten from the finished theses — is already being sent to the screen, so the reader gets the start of the answer seconds before the rest is done.

flowchart TD
  BO["planner: question → theses and draft notes"] --> RK["re-rank each thesis's notes<br/>anchor = the thesis sentence"]
  RK --> THIN{"thin thesis?<br/>weak best note, too few strong"}
  THIN -->|"no"| DONE["outline → synthesizer"]
  THIN -->|"yes"| AUG["one fresh search for that thesis<br/>re-rank and retry"]
  AUG --> DONE

What comes out reads like an ordinary paragraph, except every clause is tied to a source you can tap open:

The soul never dies with the body [^BG 2.20]; it simply moves on —
"as a person puts on new garments, giving up the old" [^BG 2.22].

When the corpus comes up empty

Sometimes the corpus genuinely has nothing. The old behaviour was a flat refusal — often the wrong answer, because the answer may well exist, just not in our corpus. So when the planner rejects every retrieved note, the agent calls a stronger model once, has it answer from general knowledge under a required disclaimer, and — in the spirit of HyDE — turns that answer into fresh search probes to try the corpus one more time, keeping only strong hits. The disclaimer is added by the server, not left to the model; there is no runtime “is this true?” judge (a model grading its own facts just flatters itself), so faithfulness is checked offline, on other models. The full behaviour is in answers that cite their sources.

What an answer costs

All this structure gives you something concrete: a grounded, cited answer for a fraction of a cent. The model is limited to the few steps that need judgment, and code runs the rest, so most of a turn never touches an expensive model. Our reincarnation turn — three theses, an intro and a conclusion, ~500 words citing a dozen sources — breaks down like this:

(Everything but retrieval runs on Gemini. Per-million-token rates: Gemini Flash-Lite $0.10 /$0.40 and Flash $0.30 /$2.50, Voyage rerank-2 $0.05, [text-embedding-3-small](https://openai.com/api/pricing/)$0.02. Token counts vary with question complexity; this is a mid-size turn.)

The most expensive step isn’t writing the answer — it’s the outline planner and the reranker, where we pay for judgment and relevance. The synthesizer that actually composes the prose runs on Gemini Flash-Lite at $0.10 /$0.40 per million tokens, so the 500-word answer itself costs about a tenth of a cent — less than the ranking that selected the sources it’s built from.

How much does that save? Suppose you skipped the pipeline and just handed what is reincarnation? and its retrieved context to one big model. Here is what that single call would cost on each, against our whole pipeline:

And even this is generous to the single call — it’s one shot, with no retrieval and no grounding of its own. A real agentic loop on one of these models makes five to seven such calls over a context that grows each turn — so multiply again, and an Opus loop lands near $0.30–0.40, fifty times our pipeline. Cheapness here isn’t a smaller model doing worse work; it’s an architecture choice — the same lesson as the un-fakeable citation: quality can come from the shape of the system, not the size of the model.

The one place a capable model does run is the out-of-corpus fallback (§ When the corpus comes up empty): a single Claude Sonnet call, about $0.01, which roughly doubles a turn — and it fires only on a genuine miss, where paying for a careful answer is exactly the right trade. Every ordinary question stays authoritative, refuses to invent, answers in seconds, and costs less than a cent.

In short

Strip the agent down and a few principles generated most of it:

• Determinism over judgment where it’s cheaper and just as good. The search fan-out replaced a ReAct loop; deterministic classifiers cut the LLM out entirely for simple requests; dedicated intro and conclusion writers replaced arguing with a prompt. “Fix it in code instead of fighting the model” is a literal line from our history.
• Two scores, two jobs. Cosine gates; the cross-encoder orders. Mix them and every threshold loses its meaning.
• Keep the mix, then order it. Reserves make sure each kind of source is represented in the answer; the reranker decides the order. Do only the second and a scripture assistant ends up quoting only lectures.
• Every threshold is a measured number, not a guess — each sits between two score clusters we actually observed: real matches on one side, false ones on the other.
• Degrade gracefully, with exactly one exception. Any stage can time out and fall through with partial results; the only error the agent refuses to swallow is a genuinely unavailable provider, which becomes an honest “the service is momentarily down” rather than a confident, ungrounded answer.

The result behaves like a careful librarian: it reaches first for what a human already curated, searches widely only when it must, defends the quiet verse against the talkative lecture, refuses to name a source it can’t stand behind, and — when the corpus really is empty — says so plainly before offering what it knows from memory.