How to fix invalid JSON from an LLM API with structured outputs

To fix invalid JSON from an LLM API, first check whether you received a complete answer, a refusal, or only a streaming fragment. Then confirm that the selected model and API path enforce your schema: JSON mode alone does not. In AIVAX, response_schema enables gateway validation and bounded healing; response_format can use native provider enforcement or healing, depending on request and account settings.

The useful boundary is who checks the final answer before your code uses it. Native structured output constrains generation at the provider. AIVAX JSON Healing checks generated output and can ask the model to correct it. Neither mechanism proves that an identifier exists, a claim is true, or a database write is authorized.

Why does an LLM return invalid JSON with structured outputs?

“Invalid JSON” can describe three different failures. An unfinished object does not parse. A complete object with a missing required field parses but fails its schema. An object containing the wrong customer ID may pass both checks and still be unusable. Diagnose them separately before changing the prompt or retry budget.

For an OpenAI request, work through these checks:

1. Check the completion state. Handle a refusal separately from the requested object. A token limit or content filter can also prevent a usable final answer; OpenAI's official parsing helpers raise distinct errors for finish_reason: "length" and "content_filter". Do not repair a refusal into a successful business result.
2. Assemble the answer before parsing it. A streaming delta is a fragment, not necessarily a complete JSON value. Use the SDK's stream accumulator or collect the final answer before calling an ordinary JSON parser. Keep reasoning and tool events separate from answer content; our reasoning protocol guide explains why those channels should not be flattened into one string.
3. Check the mode and schema dialect. OpenAI's json_object mode targets valid JSON, not your required fields or enums. Its strict json_schema mode requires a supported schema, including additionalProperties: false on objects and every field listed in required. Follow the Structured Outputs guide, not just a prompt saying “return JSON.”
4. Check the actual API path. A setting in your client does not establish what a gateway sends upstream. For OpenAI directly, Chat Completions uses response_format; Responses uses text.format. In AIVAX, check whether healing is active before assuming the provider receives a native schema parameter.

If a complete, non-refusal response still fails validation under the intended native mode, preserve a redacted minimal request, the response, model identifier, completion state, and parser error for investigation. Do not assume that all reports of “strict mode failed” share one cause.

Should AIVAX validate the answer or use native provider enforcement?

Use response_schema when AIVAX should check the final JSON and retry recoverable failures. AIVAX places the schema in the model's instructions, extracts JSON from the generated answer, and validates the candidate. This path does not require native structured-output support, but it does not guarantee that every model can satisfy every schema.

With response_format.type: "json_schema", healing changes the path. Explicit response_format.json_schema.healing_options or the account's automatic JSON Healing setting enables gateway validation and correction. Automatic healing is enabled by default, so omitting healing_options does not by itself select native enforcement.

For the native path, use a compatible model, omit response_schema and explicit healing options, and ensure automatic JSON Healing is disabled for the account. Without healing, AIVAX builds a provider-native schema request rather than performing the healing validation loop. For integrated models, the outgoing strict setting follows the model's configured capability; do not treat a client-supplied strict: true as proof of the upstream setting.

Other response formats, such as json_object, do not activate schema healing by themselves. The AIVAX Structured Responses reference documents the parameters and supported validation features.

How does AIVAX repair invalid output?

Before asking the model again, AIVAX tries to extract JSON from the complete generated text, common delimiter-repaired variants, and JSON code blocks. Extraction can recover some formatting mistakes without another generation; it is not evidence that the recovered values are correct.

If the candidate fails schema validation, AIVAX adds validation feedback to the model's conversation and requests a corrected answer. Unparseable output receives a format-correction instruction. The retry budget is bounded, and repeated failure can end the request with an error rather than a usable result. Those correction messages are part of the healing process, not a promised public diagnostics field.

Healing targets the final schema-constrained answer, including an answer produced after tool results. It does not replace validation or authorization of tool-call arguments. Tool use still requires a compatible model or tool handler; removing the need for native JSON support does not remove other capability requirements.

Each retry is another model generation and can increase latency and cost. Fix an impossible schema, missing context, or ambiguous instruction before increasing the budget. Recovery is useful for formatting and shape errors, not for turning missing evidence into a trustworthy answer.

How do I enable bounded healing in an AIVAX request?

This request explicitly enables gateway healing through response_format. Send this JSON body to POST https://inference.aivax.net/v1/chat/completions with Content-Type: application/json and Authorization: Bearer YOUR_AIVAX_API_KEY. See the inference API guide for the endpoint contract.

Replace <AVAILABLE_MODEL_ID> with a text-generation model available to your account. This example uses gateway healing, not native OpenAI strict mode.

{
    "model": "<AVAILABLE_MODEL_ID>",
    "messages": [{ "role": "user", "content": "Return a short status object." }],
    "stream": false,
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "schema": {
                "type": "object",
                "properties": {
                    "status": { "type": "string" },
                    "message": { "type": "string" }
                },
                "required": ["status", "message"]
            },
            "healing_options": { "max_attempts": 5 }
        }
    }
}

The documented range for max_attempts is 1 to 10. This bounds healing retries; it should not be read as a cap on every model call in a tool-using workflow. On success, read the final answer from the chat completion envelope and parse its JSON content. The schema requires string-valued status and message fields; it does not define which status values your application should accept.

To use response_schema instead, move the object currently inside response_format.json_schema.schema to the top-level response_schema field and remove response_format. That selects the gateway-owned healing path rather than native provider enforcement.

How should I handle streaming and json_only?

json_only: true changes the response envelope and checks that the final content parses as JSON; it does not validate a schema by itself. With stream: false, a successful response contains the final JSON as application/json, without the normal choices, usage, and generation metadata. With stream: true, AIVAX sends the complete JSON as one SSE data event followed by [DONE].

Without json_only, do not assume that every event in a healing stream is JSON answer content. Reasoning, tool activity, and usage are separate concerns. Select and assemble the final answer channel, and handle error events or interrupted delivery before passing the result downstream.

How do I prevent valid JSON from causing an invalid action?

Define required fields explicitly, give arrays an items schema, and use supported constraints such as enum when the accepted values are known. For extraction, state which fields may be null when evidence is absent and which values must never be invented. Use the schema subset documented for the mode you selected rather than assuming every JSON Schema keyword works everywhere. In particular, AIVAX's healing validator does not enforce additionalProperties: false; validate unexpected keys in your application if rejecting them matters.

Then apply business checks in your own code: permissions, valid identifiers, allowed transitions, and account-specific rules. Google's Gemini documentation makes the same distinction: syntactically correct JSON still needs application-level value validation. A repaired object can be structurally acceptable and factually wrong.

For a workflow whose output is a routing decision rather than a generated document, see support triage with typed decisions. In either design, the application owns the policy that turns an accepted value into an action. When you swap the model behind a structured call, replay your accepted outputs first; the pin-or-alias guide lists what a replacement run should check.

Frequently asked questions

Why do I get invalid JSON even with strict: true?

Check for refusal, truncation, incomplete stream assembly, and whether native strict schema enforcement was actually used. OpenAI documents refusals separately from schema-conforming answers. In AIVAX, active healing uses schema instructions and validation instead of forwarding the native response format, so the client flag alone does not identify the enforcement path.

Is json_object the same as json_schema?

No. OpenAI JSON mode targets parseable JSON, with documented edge cases; it does not enforce your schema. Structured Outputs with a supported strict schema constrain its shape. In AIVAX, json_object alone does not select schema healing.