1  A Portable Research Agent, End to End

Part I · The system in action

Demonstration baseline: reviewed 2026-10-22 · Reference snapshot: 45a903c

Start with the result, not a framework diagram. A user gives one agent an image, a fictional specification, and a workbook; asks it to research, calculate, and separate evidence classes; then asks for a PowerPoint deck. The visible workflow is compact. The contracts underneath it are the subject of this book.

1.1 The working task

Northstar Field Console is a synthetic device created for the demonstration. Its inputs are independently authored fixtures rather than production or confidential material.

Input Admitted representation What it can establish
northstar-console.png Reviewed image bytes and invocation-local image identity A generated observation about visible labels; not verified product truth
northstar-specification.pdf Bounded extracted text What the fictional specification declares: 72 Wh capacity, 1.8 kg mass, a 14-inch display, and an 8–10 hour target
northstar-measurements.xlsx Profiled workbook extraction Synthetic measured runtimes of 9 hours for Prototype A and 7.5 hours for Prototype B

The first recorded turn attaches all three inputs and asks:

Build a short evidence brief. Use image_understanding for the visible product label, inspect the attached specification and measurements, use web_search to find current official US guidance for portable-device battery capacity, use read_source on the most relevant official URL, and use code_interpreter to calculate the mean measured runtime. Separate generated image observations, attached declarations, measured values, retrieved public evidence, calculations, and anything still unverified. Do not invent citations or treat this fictional product as real.

The follow-up is deliberately short:

make a short pptx deck about those findings

That second turn does not reattach the original files. It asks a reviewed skill to organize findings already present in the conversation and a separately authorized tool to construct a bounded artifact. The distinction matters: prior conversational findings can be reused, while invocation-local file identities are not silently promoted into durable cross-request storage.

1.2 What the demonstration traverses

Pi interaction
  → loopback Open Responses request
  → ordered input admission
  → provider-neutral coordinator
  → server-owned tools
      image observation
      web discovery
      source reading
      bounded computation
  → evidence-labelled synthesis
  → reviewed presentation procedure
  → bounded presentation builder
  → digest-verified local save

The client presents the interaction. It does not own provider credentials, tool registration, document extraction, search budgets, source admission, Python execution, or presentation composition. Those decisions remain on the server side of the boundary.

The result deliberately uses different evidence labels:

  • generated observation for image-model output;
  • attached declaration for statements found in a supplied document;
  • measured value for values represented in the workbook;
  • retrieved public evidence for material admitted from a screened HTTPS source;
  • calculation for output from a fresh bounded worker; and
  • unverified when the available operations do not establish a claim.

A polished paragraph or slide must not erase these distinctions.

1.3 The portability problem

An application can call two models through similar TypeScript interfaces and still fail to be portable. Portability has at least five dimensions:

Dimension Question
Transport Can requests, streams, cancellation, and limits cross the connection correctly?
Provider Can the selected deployment be replaced without leaking provider objects into the application core?
Protocol Are request items, events, lifecycle, errors, and state interpreted consistently?
Feature Does the selected route support the tools, ordered inputs, output shape, and budgets required by this task?
Semantic Does the replacement still produce acceptable task outcomes under a published evaluation?

Structural compatibility is the narrowest claim. It does not prove account access, feature availability, conformance, safety, or semantic equivalence. Every replacement claim therefore names the environment, task profile, and meaning it is intended to preserve.

A portable agent system keeps application intent, trusted policy, tool authority, evidence identity, and terminal meaning stable while provider- or transport-specific code is confined to reviewed boundaries. Replacement remains an evidence-scoped claim, not a provider promise.

1.4 Five boundaries in one run

1.4.1 Client boundary

Pi owns terminal interaction and its local session. The reviewed launcher forces model traffic through portable-agents routes and disables Pi-local executable tools. Chapter 2 owns that client/server division.

1.4.2 Configuration boundary

agent.yaml selects only reviewed tool and skill identifiers plus bounded settings. It does not load arbitrary code, packages, paths, endpoints, credentials, or prompts. Chapter 3 owns static selection; invocation-time authorization remains Chapter 8.

1.4.3 Input boundary

Files are admitted in order, checked by format-specific limits, and assigned invocation-local identities. Text extraction is not citation support, and an image-model observation is not provenance. Chapter 4 owns ingestion.

1.4.4 Model boundary

The application sends one typed request to a selected deployment binding. Provider adapters translate that request and normalize selected output meanings. Chapter 5 owns translation; Chapters 6 and 7 own protocol lifecycle and stream-local state.

1.4.5 Tool and evidence boundary

Generated tool arguments are proposals. Server policy authorizes a call, validates input and output, enforces limits, and correlates one terminal result. Chapters 8–13 develop tools, skills, retrieval, computation, and research completion without transferring authority to model text.

1.5 Why “provider-neutral” does not mean provider-blind

Provider identity remains useful for construction, diagnostics, capability declarations, and evidence. The mistake is allowing it to determine application policy everywhere.

A provider adapter may need to:

  • construct the correct SDK client;
  • choose a provider protocol;
  • map ordered content and tool history;
  • preserve or reject provider-specific continuation state;
  • normalize finish reasons and usage without inventing absent values; and
  • sanitize failures before they cross the application boundary.

The application still decides which deployment alias is permitted, which capabilities the task requires, which tools are authorized, and whether a fallback changes the user-visible promise. Chapters 5 and 14 keep those responsibilities separate.

1.6 Failure modes visible in the case study

1.6.1 Fluent output without admitted evidence

A coordinator can write a plausible brief without reading the workbook or official source. Fluency is not completion. Chapter 13 makes required evidence operations and unresolved gaps explicit.

1.6.2 Tool names mistaken for authority

Mentioning code_interpreter in a prompt cannot enable it. Selection, current authorization, budgets, validation, and cancellation remain server-owned.

1.6.3 Search mistaken for source reading

Search produces bounded candidates and notes. A cited factual claim requires an admitted representation from read_source and a separate support judgment.

1.6.4 Attachments mistaken for durable memory

The three fixture files are invocation-local. A later request must resend them unless the application has a separately designed retained-source contract. Conversation text is not a trusted cross-request asset registry.

1.6.5 A successful deck mistaken for verified research

The presentation builder can produce a structurally valid artifact while the underlying claims remain incomplete or incorrectly labelled. Artifact integrity and evidence quality are different gates.

1.6.6 Similar APIs mistaken for conformance

The current server implements a useful Open Responses-compatible subset. Missing compaction and WebSocket lifecycle/state behavior keep the formal classification not conformant. Chapters 6 and 19 preserve that distinction.

1.7 Security considerations

Every external or generated representation remains untrusted: prompt text, attachments, extracted document text, image observations, search results, retrieved pages, Python output, skill resources, and generated presentation content. Labels inside those representations cannot grant authority.

The architecture therefore keeps these controls outside the model:

  1. trusted configuration selects reviewed capabilities;
  2. request admission enforces byte and structural limits;
  3. current policy authorizes each executable operation;
  4. credentials and recipients are server-owned;
  5. workers have explicit deadlines and output ceilings;
  6. source and artifact identities are integrity-bound; and
  7. diagnostics exclude sensitive content by default.

This reduces authority confused by prompt injection but is not a complete production security certification. Chapter 17 owns the final production gates, informed by OWASP guidance on prompt injection and excessive agency (OWASP, n.d.; “OWASP Top 10 for Agentic Applications 2026,” n.d.).

1.8 What the recording proves

The approved hero is dated live-provider and public-network evidence for its exact metadata, route, inputs, and visible behavior. It shows:

  • ordered attachment notices;
  • server-owned progress cards;
  • differentiated evidence labels;
  • a bounded calculation;
  • screened discovery and source reading;
  • progressive skill loading;
  • presentation construction; and
  • byte-count and SHA-256 verification before local save.

It does not establish deterministic output, broad semantic quality, provider equivalence, production suitability, complete privacy, or formal Open Responses conformance. The generated deck was not visually inspected during the recorded session. Those exclusions are part of the evidence, not footnotes to omit.

1.9 How to read the rest of the book

Part I separates the visible client, static configuration, and ingestion boundary. Part II develops the portable runtime. Part III composes skills and tools into a research workflow. Part IV handles substitution, recovery, observability, security, and evaluation. Part V places MCP and A2A at their proper boundaries and closes with an integrated case study and formal conformance status.

For every implementation statement, ask: which immutable release or source snapshot, which test or demonstration, which configuration, which date, and which limitation?

1.10 Pattern summary

Begin with one observable task. Keep interaction, configuration, input admission, model translation, tool authority, evidence, and artifact handling as separate contracts. Port only what the application can name and test; disclose every semantic reduction.

1.11 Exercises

  1. Classify each Northstar statement as generated observation, attached declaration, measured value, retrieved evidence, calculation, or unverified.
  2. Replace the coordinator deployment on paper. List which contracts should remain unchanged and which claims require new evidence.
  3. Remove read_source from the selected tools. Rewrite the task outcome without pretending that search candidates were read.
  4. Identify the exact point at which a generated tool call becomes an authorized execution.
  5. Explain why a digest-verified .pptx can still contain unsupported claims.

1.12 Further reading

Read the Open Responses specification as a protocol target rather than a synonym for one provider API (“Open Responses Reference 2026-04-24,” n.d.). Use the NIST Generative AI Profile for evaluation and source-review obligations (National Institute of Standards and Technology 2024), and OWASP’s prompt-injection and excessive-agency guidance for authority boundaries (OWASP, n.d.; “OWASP Top 10 for Agentic Applications 2026,” n.d.).

1.13 Chapter contract

  • Scope: the complete Northstar workflow and the portability problem it exposes.
  • Canonical claims: portability is layered; interfaces alone do not establish behavioral equivalence.
  • Migration inputs: former Chapter 1 portability definitions and the approved Northstar demonstration.
  • Evidence gates: dated demonstration metadata, immutable reference snapshots, and independently reproducible local examples.
  • Demonstration slots: the approved Pi research recording on the landing page.
  • Source-claim ledger: standards and implementation claims remain dated, versioned, and scoped.
  • Exclusions: detailed client, ingestion, runtime, tool, skill, recovery, and conformance mechanics belong to later chapters.
National Institute of Standards and Technology. 2024. “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1.” July. https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.600-1.pdf.
“Open Responses Reference 2026-04-24.” n.d. Accessed October 7, 2026. https://www.openresponses.org/reference/2026-04-24.
OWASP. n.d. “LLM Prompt Injection Prevention Cheat Sheet.” Accessed October 8, 2026. https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html.
“OWASP Top 10 for Agentic Applications 2026.” n.d. Accessed October 7, 2026. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/.