13 Grounding, Citations, and Research Completion

Part III · Skills and the compositional research toolchain

Northstar deterministic trace + dated live workflow · reviewed reference snapshot 45a903c

Research is complete not when the model has written enough prose, but when the requested claims have the required kinds of evidence—or the remaining gaps are explicitly reported. The Northstar brief combines attached declarations, workbook measurements, generated image observations, deterministic calculations, and retrieved public guidance. Those evidence classes must never collapse into one generic “citation.”

Research completion is a task-specific decision that required questions have admitted evidence of the required class, applicable calculations and contradiction checks have run, unresolved gaps are disclosed, and the final answer preserves provenance and uncertainty. It is not a judgment that every sentence is universally true.

13.1 From tools to an evidence pipeline

Chapters 10–12 provide three different operations:

web_search
  discovers bounded candidate URLs and generated notes
        ↓ selection
read_source
  admits one screened, bounded source representation
        ↓ optional sourceId mount
code_interpreter
  computes over explicit invocation-local representations
        ↓
claim assembly, support checks, gaps, and completion decision

No earlier stage lends its authority to the next. A search candidate can fail retrieval. A successful retrieval can be irrelevant. A correct calculation can use the wrong input. A valid citation locator can point to a passage that does not support the claim.

The coordinator composes operations, but the server owns their limits and identity. The final answer must name what each operation established. A fluent synthesis is the product of this pipeline, not evidence that the pipeline completed.

13.2 Five Northstar evidence classes

The fictional Northstar task deliberately forces several evidence kinds into one short brief.

Evidence class	Example	What it can establish	What it cannot establish
Attached declaration	The supplied specification states a 72 Wh capacity and an 8–10 hour target	What the admitted document declares	Independent product truth, measured performance, or authenticity
Workbook measurement	The supplied XLSX represents Prototype A as 9 hours and B as 7.5 hours	Values in the admitted synthetic workbook	Real-world measurement integrity or general product performance
Generated image observation	The image model reports visible “NORTHSTAR” and Prototype A labels	Bounded model-generated observations about admitted pixels	Verified OCR, identity, provenance, authenticity, or truth
Deterministic calculation	Mean of 9 and 7.5 is 8.25 hours	Result under named values, formula, runtime, and missingness policy	Truth of the inputs or semantic applicability of the formula
Retrieved public guidance	Text obtained from selected official public HTTPS guidance	What the admitted representation contains at retrieval time	Universal applicability, future freshness, or automatic support for every answer claim

The labels belong in the answer because the user’s epistemic question differs by row. “The specification declares” is stronger than “the model guessed” in one respect and weaker than “independently measured” in another. A calculation can be exact while its premises remain fictional.

The Northstar fixture is intentionally synthetic. It is not third-party evidence, a real benchmark, or a product certification. The public guidance is real external material, but its presence does not make the fictional device real.

13.3 Identity, locator, and support are separate

A defensible citation relation has at least three independent questions.

13.3.1 1. Which representation?

Source identity names the exact admitted object: invocation-local source ID, canonical URL or attachment identity, retrieval/admission time, media type, extraction revision, and digest where retained. A URL alone is an address, not a fixed representation. A filename alone is user-controlled text, not proof that bytes were received.

13.3.2 2. Where in that representation?

A locator identifies a page, block, cell, row, character span, or other coordinate under a declared representation and unit. Locator validity asks whether the range exists and selects the quoted material. It does not ask whether the material is relevant.

Provider formats are not interchangeable. A PDF page range, an answer-text URL annotation, a custom block index, an XLSX row/cell, and a UTF-16 text range occupy different coordinate spaces. Converting among them requires retained parser/layout mappings; changing the integer label does not perform the conversion.

13.3.3 3. Does it support this claim?

Support is a relation among a claim, one or more premises, applicable qualifiers, and an inference method. A correct locator can select an unrelated passage. One cited measurement does not support a comparison between two prototypes. A source may faithfully support “the document says X” while failing to support “X is true.”

Verify identity and location mechanically. Assess support under an explicit task method. Preserve authority, freshness, and permission independently. Never collapse those checks into one citationValid boolean.

13.4 Representation-bound locators

The book’s independently authored evidence-locators.ts demonstrates a narrow mechanical boundary for application-held text. It binds a source digest, answer digest, exact half-open UTF-16 ranges, and quote. It rejects malformed ranges, split surrogate pairs, changed source/answer content, and stale quotes. It escapes the rendered card and rechecks the link scheme.

That design proves that a selected occurrence exists in named text and that the answer revision has not changed. It deliberately calls its output VerifiedLocator, not VerifiedClaim. Repeated identical text still needs occurrence identity; equal quote strings at different offsets are different locators.

The adversarial fixture selects a well-formed instruction-bearing passage that is irrelevant to the answer. Mechanical verification succeeds. Support fails. This counterexample is more valuable than a happy-path citation because it prevents structural integrity from being promoted into semantic verification.

The implementation defines UTF-16 units for its own admitted text. It does not claim to convert OpenAI or Anthropic native citation offsets. Native provider annotations require their own dated contract and conversion tests.

13.5 Multi-premise support: the Northstar comparison

The deterministic comparison requires two premises:

Prototype A has a measured runtime of 9 hours; and
Prototype B has a measured runtime of 7.5 hours.

The premises must also share compatible units, measurement kind, freshness, and observation time. The conclusion is then computed from their numeric relation. Neither premise alone supports the comparison.

export function assessNorthstar(measurements: readonly Measurement[], nowMs: number): string {
  const prototypeA = measurements.find((item) => item.prototype === "A");
  const prototypeB = measurements.find((item) => item.prototype === "B");
  if (
    measurements.length !== 2 ||
    !prototypeA ||
    !prototypeB ||
    !Number.isSafeInteger(nowMs) ||
    nowMs < 0 ||
    measurements.some(
      (item) =>
        item.kind !== "measurement" ||
        item.unit !== "hours" ||
        !Number.isFinite(item.runtimeHours) ||
        item.runtimeHours < 0 ||
        item.runtimeHours > 168 ||
        !Number.isSafeInteger(item.measuredAtMs) ||
        item.measuredAtMs < 0 ||
        item.measuredAtMs > nowMs ||
        nowMs - item.measuredAtMs > 60_000,
    ) ||
    Math.abs(prototypeA.measuredAtMs - prototypeB.measuredAtMs) > 5_000
  )
    throw new TypeError("Comparison lacks compatible measurements.");
  const relation =
    prototypeA.runtimeHours > prototypeB.runtimeHours
      ? "longer than"
      : prototypeA.runtimeHours < prototypeB.runtimeHours
        ? "shorter than"
        : "the same as";
  return `In the synthetic workbook, Prototype A's measured runtime is ${relation} Prototype B's (${prototypeA.runtimeHours} hours versus ${prototypeB.runtimeHours} hours).`;
}

The accepted answer is scoped:

In the synthetic workbook, Prototype A’s measured runtime is longer than Prototype B’s (9 hours versus 7.5 hours).

“In the synthetic workbook” and “measured” are part of the claim. Removing either widens what the evidence appears to establish. The task-specific assessor rejects missing prototypes, declarations substituted for measurements, non-finite values, stale/future observations, incompatible timestamps, mixed units, and a contradictory proposed answer.

This is exact support logic for one synthetic two-premise task. It is not a general natural-language entailment verifier. It does not prove that arbitrary passages support arbitrary prose, infer hidden premises, resolve scientific disagreement, or assess source authority.

13.6 Calculations preserve upstream provenance

The mean runtime, 8.25 hours, is deterministic over [9, 7.5]. Its record should retain:

both workbook measurement identities;
the accepted count and units;
the missing-value rule;
the formula or reviewed code;
interpreter/runtime revision; and
bounded output.

The calculation derives from the measurements; it does not replace them. If one cell is missing, silently inserting zero would create a different claim. If one value is a specification target rather than a measurement, averaging them would cross evidence classes.

This derivation also clarifies citation display. A calculated sentence may cite the workbook premises and identify the calculation method. It should not cite a public battery-guidance URL as if that page supplied the synthetic runtime values.

13.7 Generated image observations remain generated

The image bridge can be called by an Anthropic coordinator while a fixed OpenAI image route receives the admitted image and focused question. Its result includes generated: true and explicit recipient metadata.

A final answer must retain those facts. “The attached image visibly contains…” is still a model-generated observation unless an independent deterministic inspection established the text. It is not complete OCR, coordinate-preserving extraction, source identity proof, or authenticity verification.

Generated observations can guide the brief—for example, to describe a visible prototype label—but they should not become workbook measurements or attached declarations. If the observation conflicts with canonical extracted text or source metadata, the conflict is a research finding, not permission to choose the more convenient value silently.

13.8 Retrieved guidance: provenance without automatic truth

A successful read_source call supplies a representation and URL provenance. The answer can state that the retrieved official page says or recommends something within the scope actually read. It cannot infer that:

the page remains current after retrieval;
the guidance applies to every jurisdiction or device;
the selected representation contains the complete publication;
search notes are exact quotations;
publisher authority makes every claim true; or
a clickable URL proves the sentence follows from the page.

For recall-sensitive requests—such as “list all requirements in this document”—the representation must be sufficiently complete for the requested scope, and computation/search over the full canonical text may be necessary. A landing-page failure cannot support absence. The coordinator should seek an alternate admitted full-text representation or disclose the gap.

Retrieved instructions remain untrusted data. A relevant official page still cannot alter tools, recipients, credentials, quotas, or application policy.

13.9 When is the research run complete?

Completion begins with a claim plan, not a prose-length threshold. For each requested output, define:

required claim or decision—what must the answer establish;
required evidence class—declaration, measurement, generated observation, calculation, retrieved guidance, or another named class;
scope and freshness—which documents, dates, prototypes, jurisdictions, or versions count;
operation requirement—search, read, compute, image inspection, or no external operation;
support method—exact task rule, human review, documented evaluator, or explicitly unresolved judgment; and
failure behavior—abstain, narrow the claim, present conflict, or return an evidence gap.

A Northstar research run is complete when:

every admitted attachment has a known role and successful/failed status;
specification facts are labelled declarations;
workbook values are labelled measurements represented in the fixture;
the requested mean has run over the admitted values with units and missingness checked;
image observations carry generated status and recipient disclosure;
public guidance was discovered and the selected representation was actually read;
required comparisons have all premises and pass task-specific checks;
conflicts, unavailable evidence, truncation, and uncertainty are visible;
citations identify only retained/admitted provenance and valid locators; and
the final answer makes no broader claims than those records support.

A run can also finish honestly with insufficient evidence. Completion is a terminal research decision, not synonymous with a positive answer. Spending all call budget is neither necessary nor sufficient.

13.10 A completion matrix

A practical implementation can maintain a bounded matrix:

Required output	Evidence required	Current state	Completion action
Capacity/target	Attached specification declaration	Admitted	Report as declaration
A/B runtime comparison	Two compatible workbook measurements	Admitted and checked	Report scoped comparison
Mean runtime	Two finite hour values + code execution	Computed as 8.25	Report derivation
Visible product label	Generated image observation + recipient	Completed	Report as generated/unverified
Public guidance	Search candidate + successful read	Completed for selected URL	Report representation/date scope
Real product status	Independent external evidence	Absent	State fictional/unverified

The matrix is not itself truth. It is a control structure that prevents the answer from forgetting an evidence requirement. Its state must come from server-owned operation results, not model assertions that a step “was done.”

13.11 Editing invalidates relations

Citation and support relations bind to an answer revision. Inserting “not,” removing “synthetic,” changing a unit, or widening “the workbook reports” to “the device achieves” can invalidate support even when source offsets do not move.

The deterministic trace hashes the answer and rejects an old locator draft after editing. A production system should similarly republish the authoritative answer and evidence relations together. Streamed provisional text can show references for usability, but the final revision must reconcile them. Do not claim the user never saw unsupported text merely because the terminal snapshot was corrected.

13.12 Local-file citations and ephemeral identity

The current attachment system is invocation-local. An admitted src_… ID expires when the HTTP request ends. The original local path is not a durable server identity, and attachments must be resent on later turns. Pi session text can mention a filename without carrying the original bytes.

Shared ingestion retains canonical text for the invocation, but formal page/span mappings are incomplete across every format and extractor profile. A PDF page number may survive one extraction path; XLSX evidence may need sheet/row/cell coordinates; OCR may not preserve stable layout coordinates. Therefore the system must not invent file citations or page locators from model memory.

Current honest options are:

cite the attached document descriptively within the same run, with its evidence class;
retain verified representation identity and exact locators when the extractor supplies them;
provide the local model-independent inspection report as separate extraction evidence; or
state that durable formal local-file citation is unavailable.

A future durable source service needs authenticated ownership, retained bytes/representations, stable extraction revision, page/span mappings, deletion and retention policy, and cross-request source identity. Reusing the same filename does not satisfy those requirements.

13.13 Conflict and absence

Conflicting evidence remains conflict. A target in the specification and a measured value in the workbook are not contradictory merely because they differ; they have different kinds. Two measurements under ostensibly identical conditions that disagree require qualifiers or investigation.

Absence claims require scope. “The selected representation did not contain X under this complete-text search” is narrower than “the publisher never says X.” A failed read supports neither. Search failing to return a page does not prove nonexistence.

Research completion should record unresolved contradictions as first-class outcomes. It must not average incompatible evidence, choose the latest-looking snippet, or conceal a gap behind a generic caveat.

13.14 Evaluation without entailment overclaiming

Citation evaluation can measure structural validity, required-claim coverage, citation relevance, contradiction handling, source suitability, freshness disclosure, and uncertainty behavior. Automated natural-language-inference models can screen candidate relations, but their scores are measurements under a named evaluator—not deductive proof.

For Northstar’s numeric comparison, task-specific computation is stronger and simpler than general NLI. For public guidance, human review or a documented claim-evidence rubric may be needed. Use the narrowest reliable method for each claim type rather than assigning one universal “grounded” score.

A second model cannot certify its own judgment merely by returning confidence. Record evaluator/version, sample, threshold, and limitations whenever model-based support assessment influences publication.

13.15 Security and privacy

Evidence handling crosses disclosure, parser, instruction, and rendering boundaries. Search queries can leak private facts before retrieval. Retrieved pages can contain prompt injection. Citation links can expose sensitive query strings or document membership. Digests can be sensitive and are not anonymization.

Enforce source authorization before context insertion, execution authorization outside the model, and safe rendering at publication. A source never grants tool authority. A public answer must not reveal confidential evidence through its title, URL, quote, locator, or derived calculation.

13.16 How we test completion

The deterministic Northstar tests exercise:

required capability and authorization gates before execution;
two correlated measurements with compatible units and times;
missing, stale, future, non-finite, wrong-kind, and contradictory cases;
transcript reopen without redispatch;
unknown-effect/no-repeat recovery;
source snapshots and occurrence-preserving locators; and
answer-edit invalidation.

Separate reference tests cover search receipt/budget mechanics, public-source admission, source-ID mounting, interpreter execution, image recipient disclosure, and progress privacy. The published live workflow demonstrates one end-to-end result with dated provider/network metadata. None of these is a general natural-language entailment benchmark.

13.17 Failure modes

Discovery completion. Search returns URLs, so research stops. Read the selected source and evaluate support.

Evidence-class collapse. A declaration, measurement, observation, and calculation are all called “facts.” Preserve their labels and derivations.

One-premise comparison. One runtime is cited for a two-runtime claim. Require the complete premise set.

Locator certification. A valid range is treated as semantic support. Test the claim relation separately.

Filename citation. A remembered local name becomes a formal citation. Require retained representation identity and locators.

Completeness by verbosity. A long answer is considered finished despite a missing required operation. Use a claim/evidence matrix.

13.18 Pattern summary

Problem: Tool success and citation-like output can hide incomplete or unsupported research.

Pattern: Preserve evidence classes, connect discovery to admitted representations and computation, separate identity/locator/support, and decide completion against a task-specific claim plan.

Result: The answer can explain what was established, how, from which representation, and what remains unknown.

Cost: Evidence bookkeeping, task-specific support methods, conflict handling, revision-aware citations, and honest abstention remain necessary.

13.19 Exercises

Add a third Northstar runtime observation with an incompatible timestamp. Specify whether the comparison, mean, both, or neither can complete.
Create a locator that is structurally valid but irrelevant. Define the separate support decision.
Design durable local-file identity for a resent PDF without trusting its filename.
Write a completion matrix for a recall-sensitive inventory from a retrieved document. Include failed landing-page behavior.
Edit the accepted answer by removing “synthetic workbook.” Explain which evidence relation must be re-evaluated.

13.20 Chapter contract

Scope: evidence classes, source identity, locator validity, multi-premise support, derivation, and research-completion criteria.
Canonical claims: identity, locator validity, and claim support are separate; completion is task-specific and can end with an explicit evidence gap.
Migration inputs: deterministic Northstar composition, locator helper/tests, current research tools, and the dated end-to-end demonstration.
Evidence gates: admitted operation receipts, representation identity, task-specific support tests, revision binding, and explicit limitations.
Demonstration slots: Northstar supports the five-class pipeline and two-premise trace; it does not establish general entailment verification.
Source-claim ledger: every published claim records evidence class, representation/time, locator method where available, support method, and unresolved limits.
Exclusions: discovery belongs to Chapter 10; transport/representation admission belongs to Chapter 11; computation mechanics belong to Chapter 12.