14  Capabilities, Fallbacks, and Mixed-Provider Workflows

Part IV · Portability in production

Deterministic admission tests + dated Northstar mixed-provider workflow · reviewed reference snapshot 45a903c

The preferred Northstar coordinator is unavailable. A second model can produce text, and its client has the same interface. Should the application switch? Only if that switch preserves the work the application promised. Returning a fluent brief without the required tools and evidence is not graceful degradation; it is a different task disguised as success.

A capability promise is a scoped assertion that a deployment can perform an operation under named conditions. Fallback selects another execution target; degradation explicitly reduces a task’s contract. Neither operation is justified merely by accepting the same request type.

14.1 The problem

Chapter 5 bound model construction to trusted deployment configuration. We now need a decision before sending a request: does this binding meet the task’s requirements? A constructor can succeed while a model lacks the required feature, the credential cannot access it, or the selected endpoint rejects the combination of settings.

Our synthetic Northstar comparison requires two runtime measurements obtained through a controlled tool. A fallback model that can only generate text may summarize measurements already supplied by a trusted upstream stage. It cannot satisfy a requirement to obtain those measurements itself. If the application changes from tool-assisted comparison to a clearly labelled explanation of what data is missing, the reduction in service must be visible in its outcome.

This is a task contract problem, not a larger switch statement. We must separate what the application requests, what the adapter declares, what a test observed, and what the product requires. The consequence of confusing them is often semantic: the HTTP request works and the answer is plausible, but the promised operation never occurred.

14.2 Context and forces

Availability without authority drift. A second provider can improve availability, but changing recipients, regions, tools, or principals also changes the security review. An outage is not permission to export the same data elsewhere.

Evidence that ages. A passing probe describes one configuration at one time. An adapter update, changed account, revised policy, or changed task profile can invalidate its use without changing the model’s name.

Combinations, not catalogues. Tools, streaming, structured output, citations, reasoning, and modalities interact. Independent positive feature checks do not prove their simultaneous use works.

Quality beyond syntax. A valid JSON object can contain the wrong city, a well-formed tool call can select the wrong operation, and a cited answer can contradict its source. Admission establishes necessary conditions, not the complete semantic result.

Reading rule. Ask which evidence authorizes this capability for this deployment, task stage, and policy revision—not whether a provider is generally described as supporting it.

14.3 Four different capability statements

Statement	Owner	Example	What it proves
Requested	Request construction	Tools are included in this call	The caller expressed an intention
Declared	Adapter configuration	This model is configured as tool-capable	A configured assertion
Observed	Test or operational measurement	A named fixture completed a correlated tool round trip	That observation, under its recorded conditions
Required	Application task policy	Fresh observations must precede comparison	A condition for admitting or accepting the task

The tagged CapabilityResolver returns an array of text, streaming, and tools for a model ID. declaredCapabilities(modelId) makes a new set from it. By default all three are declared. The model-scoped fixture changes the declaration for text-only and tool-model. It proves the configuration can vary by model; it neither contacts a provider nor enforces admission inside streamModel.

Part I’s book guard demonstrates rejection against declarations. For a trusted deterministic fixture, that is a useful admission rule. For a deployed service, choosing a stronger evidence requirement may be appropriate. Do not retrospectively relabel a declaration as a successful live probe merely because the model has generated some text.

MCP illustrates a different mechanism. Its pinned lifecycle exchanges protocol versions and client/server capabilities during initialization; operation must respect successfully negotiated capabilities (“Model Context Protocol 2025-11-25,” n.d.). That is actual protocol negotiation. The reference model port has no equivalent initialization handshake. Applying the idea of explicit capabilities does not make provider model features negotiated MCP capabilities, nor does an MCP server’s tools declaration prove that a model will choose its tools correctly.

14.4 A promise needs a scope

A useful evidence key includes deployment identity, provider model ID, adapter revision, task profile, and policy revision. The task profile represents the relevant combination of features and parameters, not just a marketing category. A profile might name a tool-assisted Northstar evidence brief with a particular schema set and generation budget.

This key is a design choice, not a standardized model capability format. An implementation may also need endpoint, account, region, dependency lock digest, model snapshot, prompt revision, and tool schema digests. The deployment identifier in this chapter assumes those construction details are managed elsewhere. If it does not change when those details change, the evidence key is incomplete.

Task requirements + permitted degradation
                   │
Trusted deployment ── scoped declaration/observation records
                   │
                   ▼
            Admission decision
      accept / explicit degradation / reject
                   │
                   ▼
       Execution and outcome evaluation

Expiry is an evidence reuse rule, not a guarantee that a model cannot change before that instant. Positive evidence can be revoked early by an incident or configuration change. Negative evidence can also be context-specific: a transient outage is not proof that a feature is permanently unsupported. Preserve the distinction between “not demonstrated for this request” and “known unsupported.”

A practical registry should keep the evidence reference and method, not just the result. Who ran the check? Was it a SDK seam mock, live protocol exchange, or task-quality experiment? What was the acceptance threshold? A trusted assertion may be adequate for a low-risk preview while an irreversible tool-bearing task demands stronger proof. These are policy decisions, not values supplied by a model-generated capability record.

14.4.1 Declaration is not dated evidence

The current reference has two deliberately different records. Static route configuration declares which adapter and feature profile may be constructed. A live demonstration record names the exact server revision, date, coordinator route, secondary recipients, operation sequence, and observed outcome. The declaration can authorize attempting a task; the dated record can support only the observed compatibility claim under its recorded conditions.

Do not write “model X supports image input” when the implemented path actually masks image bytes from the coordinator and sends one selected image to a separate OpenAI image route. Do not convert a deterministic synthetic transport test into a live-account claim. Conversely, a successful live run does not make the capability permanent: account access, endpoint policy, model behavior, dependencies, and tool configuration can change.

For example, the local evidence-method record resolves local-composition-fixture-v1 to scripted-local-composition, profile two-measurements-v1, the exact five-field scope, and named source/tests. A live-wire method would instead need account/endpoint class, request combination, producing revision, and observed terminal result; a task-quality method would additionally name the sample and acceptance rule. All can be observations, but they are not interchangeable evidence. The helper leaves method lookup to its trusted caller; its observed enum does not encode a production assurance level. In the composition fixture, the positive feature records are supplied synthetic policy inputs, not feature discovery or measurements of model/streaming performance. The observed property is the local policy transition and its task consequences.

14.5 An executable admission policy

The following implemented and deterministically tested book design is intentionally stricter than the reference declaration guard. Its types and scope comparison are in code/capability-admission.ts. The input evidence is trusted, already authenticated application data; this is not a decoder for untrusted JSON.

AdmissionRequest separates required features from optional features and selects a minimum evidence kind. Each evidence record contains a scope, feature, declaration/observation kind, boolean assertion, check/expiry times, and evidence ID. The book feature vocabulary includes citations and json-schema; these are not additions to the tagged reference’s capability union.

export function admitCapabilities(
  request: AdmissionRequest,
  evidence: readonly CapabilityEvidence[],
  now: number,
): { readonly mode: "accept" | "degrade" | "reject"; readonly missing: readonly Feature[] } {
  if (!Number.isSafeInteger(now) || now < 0) throw new TypeError("Invalid admission clock.");
  for (const record of evidence) {
    if (
      !Number.isSafeInteger(record.checkedAt) ||
      record.checkedAt < 0 ||
      !Number.isSafeInteger(record.expiresAt) ||
      record.expiresAt <= record.checkedAt ||
      record.evidenceId.trim() === ""
    ) {
      throw new TypeError("Invalid capability evidence.");
    }
  }
  const fresh = evidence.filter(
    (record) =>
      sameScope(record.scope, request.scope) && record.checkedAt <= now && now < record.expiresAt,
  );
  const supports = (feature: Feature): boolean => {
    const matching = fresh.filter((record) => record.feature === feature);
    if (matching.some((record) => !record.supported)) return false;
    return matching.some(
      (record) =>
        record.supported && (request.minimum === "declared" || record.kind === "observed"),
    );
  };
  const requiredMissing = [...new Set(request.required)].filter((feature) => !supports(feature));
  if (requiredMissing.length > 0) return { mode: "reject", missing: requiredMissing };
  const optionalMissing = [...new Set(request.optional)].filter((feature) => !supports(feature));
  return { mode: optionalMissing.length > 0 ? "degrade" : "accept", missing: optionalMissing };
}

export function mayFallback(
  reason: "unavailable" | "refusal" | "policy" | "authentication",
  outputStarted: boolean,
): boolean {
  return reason === "unavailable" && !outputStarted;
}

The half-open interval checkedAt <= now < expiresAt excludes observations from the future and treats the expiry instant as expired. Evidence for another scope cannot satisfy this request. Any fresh negative assertion vetoes the feature, even alongside a positive assertion. This conservative conflict rule avoids depending on input order; an operational registry needs a reviewed resolution process rather than accumulating contradictory observations forever.

The function returns feature names lacking adequate support. It does not assert that every missing feature is unsupported. Required gaps reject admission; optional gaps produce an explicit degradation result. The caller must map that result to an approved task variant, disclosure, and output contract before executing. A return value is not itself consent, authorization, or routing.

In the cross-chapter Northstar trace, missing tools rejects before database creation or tool execution: both counters remain zero. Missing only optional streaming selects a disclosed non-streaming comparison; it does not remove either observation requirement. A separate current-authorization denial still prevents all execution after admission succeeds. Those checks make the distinction between a capable target and an authorized operation observable.

A capability record admits a named operation under named conditions. It neither proves every feature combination nor authorizes the effects of executing that operation.

The helper does not launch probes, choose TTLs, authenticate records, refresh caches, enforce an account allowlist, or perform a provider call. Its synchronous scan is sufficient for small fixture registries; production bounds on registry size and request feature count remain necessary. The injected clock makes expiry tests deterministic and keeps time acquisition outside policy. In a distributed deployment, acceptable clock uncertainty must be specified rather than assumed away.

14.6 The implemented mixed-provider workflow

The reviewed Northstar run demonstrates composition across recipients rather than a monolithic provider feature set:

Pi request
  → portable-agents/anthropic/claude-opus-5-5 coordinator
      → local bounded Pyodide calculation
      → OpenAI gpt-5.4 image-understanding recipient
      → OpenAI gpt-5.4 hosted-search recipient
      → bounded public-HTTPS source reader
      → local bounded PptxGenJS presentation worker

The coordinator receives ordered attachment markers rather than the original image bytes. If it proposes image_understanding, the server sends only the selected admitted image plus a focused question to the fixed OpenAI route. The result names { provider: "openai", model: "gpt-5.4" } and marks observations as generated. Search similarly sends a focused query to a separate configured OpenAI search model, not the whole coordinator transcript.

This is a mixed-provider workflow because coordination and selected sub-operations have different recipients. It is not automatic provider fallback. Each recipient is statically reviewed, independently authorized, bounded, and disclosed. An Anthropic coordinator does not imply that Anthropic inspected the image or performed hosted search. A common application tool schema does not imply that OpenAI and Anthropic receive the same bytes, implement the same internals, or produce semantically equivalent results.

The dated recording metadata identifies the coordinator, each secondary recipient, exact server commit, Pi and Bun versions, review status, and limitations. It proves that one approved run visibly exercised those operations. Deterministic tests separately prove masking, fixed-recipient routing, recipient metadata, and progress non-disclosure with synthetic transport.

14.6.1 Recipient disclosure is part of admission

Routing policy must answer two questions before a secondary model receives data:

1. Is this recipient authorized for this data class and purpose?
2. Can the user and evidence record tell which recipient actually processed it?

A secondary operation should receive the minimum data needed: one image and question, or one focused search query—not the entire conversation by convenience. Its result should preserve generated status and recipient identity. Progress may name the reviewed recipient class without exposing payload. Logs and public errors remain bounded.

A user choosing an Anthropic coordinator has not necessarily consented to an OpenAI sub-operation. The product must disclose and authorize that route rather than rely on the coordinator label. Residency, retention, contractual permission, and cost can differ even when both operations share a TypeScript interface.

14.6.2 Evidence for a provider switch

A provider-switch claim requires an observable route change. A recording that runs two similar calculations without showing the selected portable-agents route does not prove a switch. Acceptable evidence should visibly show the model selection or route transition, preserve server-owned tool cards after the transition, and record both coordinator recipients in metadata.

No such recording should be published merely to fill a demonstration slot. Until an approved artifact visibly proves the switch, the chapter relies on deterministic route tests and the Northstar mixed-provider recording’s single Anthropic-coordinator path. That path proves mixed recipients; it does not prove live switching from one coordinator to another during the session.

14.7 Fallback task profiles

Fallback begins with a named task profile, not a generic requirement that the replacement “supports tools.” Useful Northstar profiles include:

Profile	Required operations	Permitted reduction
northstar-research-current-v1	attachments, fixed image recipient, search, read, local computation	none for a completed evidence brief
northstar-existing-evidence-v1	already admitted declarations/measurements and local computation	no fresh public-guidance claim
northstar-text-only-gap-v1	text generation only	explain unavailable evidence; no research completion claim
northstar-presentation-v1	reviewed skill plus separately authorized builder	return research answer without deck if build is optional and disclosed

The profile fixes required evidence classes, recipients, tools, freshness, and terminal disclosure. A fallback candidate must satisfy that exact profile or move to an explicitly approved reduced profile. Deleting a failed capability from the requirement list after routing has begun is not degradation policy; it is an unreviewed task rewrite.

14.8 Fallback is a policy transition

A fallback candidate must pass the same hard constraints as the original candidate: data destination, authorization, task requirements, remaining deadlines, and appropriate evidence. It is not enough to remove the capability that caused rejection.

Consider three Northstar outcomes:

1. Equivalent admitted execution: another authorized tool-capable deployment obtains the same required observations under the same tool policy. Its eventual answer still needs task evaluation.
2. Explicit task reduction: the service returns an unavailable-data explanation, or summarizes previously verified observations with their time clearly disclosed. The application labels the changed task and does not claim fresh retrieval.
3. Rejected substitution: the candidate lacks required tools or would send data to an unapproved destination. The application returns a bounded failure rather than fabricate a successful comparison.

The mayFallback example deliberately allows only an availability failure before visible output. This guard is necessary but not sufficient: it does not inspect effect history, debit a budget, or authorize a destination. A true result still needs the recovery gate in Chapter 15. It rejects policy, authentication, and refusal reasons. An operator can repair credentials and start an authorized attempt; an automated fallback should not reinterpret an access denial as a search for a provider that accepts the forbidden operation. Nor should a refusal be routed around to weaken safety protections.

Once output is visible, switching providers introduces an attempt-boundary problem. The second answer cannot safely be appended to the first as if both were one response. A product may support an explicit replacement attempt with a new identity and a notice that the earlier attempt was abandoned. That is a designed recovery protocol, not implemented by this helper; Chapter 15 develops the commitment boundary; it is not assumed to exist in the reference.

Fallback also needs a finite search space. A policy should bound candidates and total attempts, debit a shared deadline/cost budget, and record which candidate was rejected for which safe reason. Independent retries at the model, router, gateway, and client can turn a small candidate set into a large call fan-out. “Try another model” is not a budget policy.

14.9 Provider-specific variation: feature conjunctions

As rechecked in the 2026-10-08–09 review, OpenAI’s Structured model outputs guide distinguishes structured function arguments from structured response text (“Function calling vs response format”) and schema adherence from JSON mode (“Structured Outputs vs JSON mode”) (OpenAI, n.d.). Therefore a generic json capability is too imprecise for admission: it can mean syntactically valid JSON, constrained output, locally validated output, or strictly validated function arguments.

Anthropic’s guide, “Feature compatibility,” documents a concrete interaction: JSON outputs using output_config.format are incompatible with citations, which require interleaving citation information with text; enabling that combination returns a 400 error. Its “Invalid outputs” section describes refusal and token-limit cases where the desired schema may not be delivered (Anthropic, n.d.). The claim register records the specific sections and retrieval date. This is dated, mutable documentation evidence—not an execution result for the reference’s pinned packages.

This is why a feature vector alone is insufficient. If both citations and json-schema are declared true in isolation, the conjunction can still fail. Our task-profile scope must represent the actual combination being asserted. A production implementation should encode constraints directly or use profiles whose tests exercise the combined request. The book helper trusts the profile; it does not infer interaction constraints from provider names.

Neither structured-output configuration nor native citations are exposed in the tagged ModelRequest. For this reference, those requirements would need a reviewed contract extension or a separate stage. Do not promise them merely because the underlying SDK has a larger option surface.

14.10 Measure the semantic gap

NIST AI 600-1 recommends setting performance/assurance thresholds in deployment approval (GV-1.3-002) and avoiding extrapolation from narrow, non-systematic, anecdotal assessments (MS-2.5-001) (National Institute of Standards and Technology 2024). It is voluntary risk-management guidance, not a portability certification. Its useful architectural lesson is to state the measurement context and the decision it supports.

The historical tagged semantic evaluation script contains three synthetic exact-match tasks: a comparison from supplied facts, ordered name extraction, and ignoring a bracketed untrusted instruction. Its retained test-sequence summary is a sanitized aggregate compiled from owner-console observations. It records 12 passing attempts and explicitly withholds broad semantic portability; it is not the script’s complete machine-readable per-attempt report. The tasks do not execute external tools, measure citation entailment, or test a deployment’s response to a realistic adversarial distribution.

Exact match is appropriate when the task demands one fixed serialization. It can be a poor proxy for ordinary answer quality: a correct paraphrase fails, while a memorized short answer may pass without exercising the intended workflow. Define the construct being measured before selecting a metric.

For a replacement Northstar coordinator, a proposed evaluation protocol separates:

Gate	Example measurement	Acceptance role
Authority	No call outside authorized tools or destinations	Hard rejection, not averaged into quality
Procedure	Required observations obtained and correlated	Task contract
Factual result	Correct runtime comparison with units and measurement times	Semantic correctness
Abstention	Missing or conflicting observations are disclosed	Uncertainty behavior
Presentation	No partial result represented as complete	Client contract
Resources	Latency and measured usage under the same bounded scenario	Operational suitability

Use paired scenarios, documented model/configuration versions, repeated attempts, and disaggregated failures. Include unavailable tools, ambiguous city names, conflicting observations, stale data, and malicious tool text. Specify the sample construction and thresholds before inspecting the result. Do not collapse an authorization failure into an average quality score that can be offset by good prose.

A provider switch need not produce identical wording, tokenization, latency, or price. More importantly, a shared request type and successful terminal response do not establish semantic equivalence. Portability means preserving required meanings and constraints while measuring permitted variation. Unknown usage stays unknown; estimates and provider measurements must not be mixed into one supposedly observed cost figure.

14.11 Failure modes

Permanent booleans. A tools: true value without provenance becomes a stale promise after endpoint or policy changes. Invalidate on construction changes and define refresh or revocation behavior.

Optional by accident. Moving a hard requirement into the optional list merely to make a candidate pass is an undeclared product change. Review the reduced task separately.

Probe mismatch. A text probe does not prove tool continuation; a tool mock does not prove wire compatibility; a tiny exact-match test does not prove broad safety. Match evidence to the gate it exercises.

Fallback as laundering. A new provider must not acquire wider tool authority or receive data forbidden to the first. Keep policy identity stable while reviewing the new recipient.

Confidence from clean syntax. Schema-compliant output can still be factually wrong. Keep semantic verification outside the syntax gate.

14.12 Security considerations

Capability records influence execution and therefore belong to the control plane. A model, fetched document, or untrusted tool result must not declare itself capable and thereby enable a privileged operation. Verify record provenance, bind records to reviewed deployment identity, and authorize registry updates. An evidence ID is only a lookup label in this example; it is not a signature.

A fallback routing rule is also a data-flow rule. Do not assume that two deployments share retention, residency, or contractual permissions because their interfaces match. Avoid probing with sensitive prompts or privileged tools; use synthetic, bounded probes whenever possible. Account availability probes can themselves consume resources and expose information about deployment membership.

Conflict resolution and expiry should fail closed for required operations but return useful safe reasons. Public responses can identify an unmet requirement without disclosing internal endpoints, credentials, or the complete deployment registry.

14.13 Telemetry contract

Proposed fields include task/profile revision, selected deployment class, admission mode, unmet feature names, evidence method/age bucket, safe fallback reason, attempt count, and remaining budget. Evidence IDs and deployment identifiers may require redaction or controlled cardinality. Do not log prompts merely to explain a capability rejection.

Record degradation as a first-class task outcome, not as ordinary successful completion with a hidden flag. Separate admission rejection, provider failure, and task-quality failure; otherwise a dashboard can report excellent model uptime while required workflows never execute. The helper returns decisions; tracing their operational use belongs to its caller.

14.14 How we test portability

Seven network-free book tests cover fresh observed conjunctions, declaration-versus-observation policy, exact scope matching across all five fields, expiry/future times, order-independent conflict rejection, explicit optional degradation, malformed time/evidence fields, and refusal/partial-output fallback prevention. The fixtures use synthetic evidence; they do not certify the truth of a real capability assertion.

The reference’s model-scoped declaration fixture and retained 12-attempt evaluation remain separate proofs. The local composition additionally checks admission rejection, approved degradation, and authorization independently. No new paid provider probes were run. The reader archive runs these book-only checks with bun run test; from the authoring root:

bun test book/test/capability-admission.test.ts
bun run check

14.15 Portability and evidence ledger

Dimension	Evidence label and scope	Limitation
Transport	Not tested here: endpoint/account availability	Declaration and admission do not contact a wire endpoint
Provider	Implemented: model-scoped declarations in v0.3.0	Not live discovery; book admission remains separate
Protocol	Specified: MCP initialization negotiation	Not negotiation of model features
Feature	Implemented/tested: scoped book admission and explicit degradation	Trusts evidence/profile construction; no operational registry
Semantic	Locally tested: required observations survive optional degradation; retained exact-match aggregate is separate	No live substitution equivalence or broad safety assessment

Additional policy concern. The refusal/visible-output fallback guard is tested, but does not implement destination authorization, effect accounting, or a complete budgeted router.

14.16 Pattern summary

Problem: A replaceable interface can conceal an unsupported feature or a changed task.

Pattern: Define hard requirements and approved optional reductions, scope capability evidence to deployment and task configuration, invalidate stale claims, and admit each fallback under the same policy constraints.

Result: Substitution becomes an explicit, inspectable task decision rather than a silent downgrade.

Cost: Evidence maintenance, interaction testing, bounded candidate selection, and task-level evaluation remain necessary.

14.17 Exercises

1. Define two Northstar task profiles: obtaining admitted measurements and summarizing verified existing measurements. Explain when their required features differ without weakening the user’s promise.
2. Add a trusted early-revocation mechanism to the evidence design. Test a positive observation revoked before its expiry.
3. Design a joint citations/structured-output profile without assuming independent feature booleans compose. Identify which provider contract must be checked.
4. Write a paired replacement rubric with one non-negotiable authority gate and two quality thresholds. Specify how a tool-policy violation affects the decision.
5. Describe how an operator can repair an authentication failure without turning the automated fallback mechanism into an authorization bypass.

14.18 Further reading

Compare actual negotiated capabilities in MCP (“Model Context Protocol 2025-11-25,” n.d.) with documented provider feature constraints (OpenAI, n.d.; Anthropic, n.d.). Read NIST AI 600-1’s deployment-threshold and measurement-limit actions (National Institute of Standards and Technology 2024) alongside the reference evaluation script. None substitutes for evaluating the application’s own task distribution.

14.19 Chapter contract

• Scope: capability evidence, explicit degradation, fallback, and mixed-provider workflows.
• Canonical claims: availability and interface compatibility are necessary but insufficient for substitution.
• Migration inputs: former capabilities-and-fallbacks chapter.
• Evidence gates: scoped feature records and task-level semantic evaluation.
• Demonstration slots: deterministic fallback admission trace and the dated Northstar mixed-recipient workflow; no coordinator-switch recording is claimed until the route change is visibly proved.
• Source-claim ledger: observed, declared, and inferred capability evidence remain distinct.
• Exclusions: adapter mechanics belong to Chapter 5; retry policy belongs to Chapter 15.

Anthropic. n.d. “Structured Outputs.” Accessed October 9, 2026. https://platform.claude.com/docs/en/build-with-claude/structured-outputs.

“Model Context Protocol 2025-11-25: Lifecycle.” n.d. Accessed October 8, 2026. https://modelcontextprotocol.io/specification/2025-11-25/basic/lifecycle.

National Institute of Standards and Technology. 2024. “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1.” July. https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.600-1.pdf.

OpenAI. n.d. “Structured Model Outputs.” Accessed October 9, 2026. https://developers.openai.com/api/docs/guides/structured-outputs.