Structured Outputs and Tool Calling That Survive Production

Free-form model text is fine for drafts. Product features need objects: a ticket payload, a calendar event, a CRM update, a ranked list of next actions. Structured outputs and tool calling are how you turn probabilistic language into something your backend can trust — but only if you treat schema, validation, and failure as part of the product, not as afterthoughts.

Prose is not an API

If your UI parses a paragraph with regex or "usually the JSON is in a code fence," you do not have an integration. You have a demo that will break on the first edge case: a missing field, a string where a number belongs, or a helpful model that adds commentary around the object. Production systems need a contract: the model must return a shape your code already understands, or the request must fail in a way you designed.

• Define the output schema before you write the prompt
• Prefer provider-native structured output or constrained decoding when available
• Validate every response against the same schema your TypeScript types expect
• Never silently coerce bad fields into "good enough" defaults for money, access, or writes

Tool calling is orchestration, not magic

Tools are functions with names, arguments, and side effects. The model proposes a call; your server decides whether to run it. That boundary matters. A tool named `update_order` should not accept free-text "whatever looks right" — it should take typed arguments, check auth and tenancy, and return a result the model can continue from. If the model can invent tool names or skip required args, your feature will invent business actions too.

Keep the tool surface small

Five sharp tools beat twenty vague ones. Overlapping tools (`search_docs`, `find_document`, `lookup_kb`) teach the model to waffle. Prefer one retrieval tool with clear filters, one write tool with a narrow purpose, and explicit human-confirm steps for irreversible actions. Document each tool the way you would document a public API: when to use it, when not to, and what success looks like.

Fail closed, then retry with intent

Schema misses will happen. Plan for them: reject invalid JSON, log the failure with prompt and tool versions, and retry with a short repair prompt that includes the validation errors — not a full conversation dump. Cap retries. After the budget is gone, escalate to a safe UI state: ask the user, show a manual form, or queue for a human. Infinite "fix your JSON" loops burn tokens and still ship garbage.

• Validate on the server before any side effect
• Return field-level errors the model can correct in one retry
• Timeout and cancel hung tool chains instead of stacking partial writes
• Record which schema version produced each successful object

Design the product for partial success

A multi-step agent that booked the meeting but failed to send the invite is worse than a clear failure. Make tool sequences idempotent where you can, and make intermediate state visible when you cannot. Users should see which steps completed. Support should see which tool call failed and why. Structured outputs make that possible — but only if you persist the structured trail, not just the final chat bubble.

“If you cannot validate it, you cannot ship it. Structured output is how AI features earn the right to write to your systems.”

What we ship at Brutanix

At Brutanix Studios we start AI features from the object the product needs — not from a chat transcript. Schema, tools, auth, and eval cases for invalid shapes land in the same pull request as the UI. That keeps copilots and agents useful without turning your backend into a suggestion box the model can rewrite at will.