Structured output is a contract, not a request

The first thing we standardized when model calls entered our pipelines was the boundary. Every call whose output feeds a machine returns JSON against a schema, and the schema is enforced by an ordinary validator. Not by asking nicely in the prompt. By rejecting the output and making the model try again with the validator's error in its face.

The loop

async function extract(input, schema, tries = 3) {
  let feedback = '';
  for (let i = 0; i < tries; i++) {
    const raw = await llm({
      system: RULES + feedback,
      user: input,
      response_format: schema,   // constrained decoding where the API supports it
      temperature: 0,
    });
    const errors = validate(schema, raw);   // plain JSON Schema, same lib as our forms
    if (errors.length === 0) return raw;
    feedback = ' Previous output failed validation: ' + errors.join('; ');
  }
  return quarantine(input);   // a review queue. never a crash, never a guess.
}

Three details carry most of the weight:

• Enums beat free text. The single biggest reliability jump came from replacing string fields with closed enums. One extraction pipeline went from a 4.1% rejection rate to 0.2% the week we enum-ified its category fields. A model that must choose from seven values cannot invent an eighth.
• Feed the validator error back verbatim. A bare "try again" retries the same mistake. The exact message, "items[2].price must be integer", fixes it on the next attempt almost every time.
• Temperature 0 is necessary, not sufficient. Greedy decoding removes randomness, not wrongness. The validator is still the one telling the truth.

Quarantine is a feature

The item that fails three attempts is the interesting one. It goes into a review queue, a human labels it, and the labeled case joins the regression suite. Our schemas have been hardened by two years of their own rejects. Which points at the real lesson: the schema is half of the prompt. Most of our prompt engineering time is spent deleting fields, tightening types and closing enums, because every degree of freedom you remove from the output is a hallucination that can no longer happen.