Structured outputs

Structured outputs ensure that model responses match specific formats like JSON schemas, regex patterns, or predefined choices. Tinfoil uses vLLM’s guided decoding to constrain outputs by filtering next-token predictions, guaranteeing valid formats without post-processing.

Tinfoil supports structured outputs through vLLM. Use response_format with json_schema type for JSON outputs, or structured_outputs for choice and regex constraints. In Python, pass it via extra_body. In JavaScript, pass it directly on the request body (the OpenAI Node SDK does not support extra_body).

Benefits

Format Enforcement: Token-level filtering ensures outputs match your exact format
Type Safety: Works with Pydantic (Python), Zod (TypeScript), and native types in Go
No Post-Processing: Outputs are guaranteed valid
Deterministic: Next-token prediction is constrained to produce only valid tokens
Multiple Backends: Supports xgrammar and guidance backends

For complete documentation, see the vLLM Structured Outputs Guide and the vLLM blog post on structured decoding.

Quick Start

Here are basic examples for each structured output type:

Choice

Restrict output to a predefined list:

from tinfoil import TinfoilAI

client = TinfoilAI(api_key="<YOUR_API_KEY>")

response = client.chat.completions.create(
    model="<MODEL_NAME>",
    messages=[
        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
    ],
    extra_body={"structured_outputs": {"choice": ["positive", "negative"]}}
)

print(response.choices[0].message.content)

import { TinfoilAI } from 'tinfoil';

const client = new TinfoilAI({
  apiKey: process.env.TINFOIL_API_KEY
});

const response = await client.chat.completions.create({
  model: '<MODEL_NAME>',
  messages: [
    { role: 'user', content: 'Classify this sentiment: vLLM is wonderful!' }
  ],
  structured_outputs: { choice: ['positive', 'negative'] }
} as any);

console.log(response.choices[0].message.content);

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/openai/openai-go/v3"
    "github.com/openai/openai-go/v3/option"
    "github.com/tinfoilsh/tinfoil-go"
)

func main() {
    client, err := tinfoil.NewClient(
        option.WithAPIKey(os.Getenv("TINFOIL_API_KEY")),
    )
    if err != nil {
        log.Fatal(err)
    }

    response, err := client.Chat.Completions.New(
        context.TODO(),
        openai.ChatCompletionNewParams{
            Model: "<MODEL_NAME>",
            Messages: []openai.ChatCompletionMessageParamUnion{
                openai.UserMessage("Classify this sentiment: vLLM is wonderful!"),
            },
        },
        option.WithJSONSet("structured_outputs", map[string]interface{}{
            "choice": []string{"positive", "negative"},
        }),
    )
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(response.Choices[0].Message.Content)
}

use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("<MODEL_NAME>")
        .messages([json!(
            {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
        )])
        .structured_outputs_choice(["positive", "negative"]);

    let response = client.chat_relaxed().create(body).await?;
    println!("{}", response.content().unwrap_or(""));
    Ok(())
}

Regex

Enforce regex patterns for formatted outputs:

from tinfoil import TinfoilAI

client = TinfoilAI(api_key="<YOUR_API_KEY>")

response = client.chat.completions.create(
    model="<MODEL_NAME>",
    messages=[
        {"role": "user", "content": "Generate an example email address for Alan Turing, who works in Enigma. End in .com."}
    ],
    extra_body={"structured_outputs": {"regex": r"\w+@\w+\.com"}}
)

print(response.choices[0].message.content)

import { TinfoilAI } from 'tinfoil';

const client = new TinfoilAI({
  apiKey: process.env.TINFOIL_API_KEY
});

const response = await client.chat.completions.create({
  model: '<MODEL_NAME>',
  messages: [
    { role: 'user', content: 'Generate an example email address for Alan Turing, who works in Enigma. End in .com.' }
  ],
  structured_outputs: { regex: '\\w+@\\w+\\.com' }
} as any);

console.log(response.choices[0].message.content);

package main

import (
    "context"
    "fmt"
    "log"
    "os"

    "github.com/openai/openai-go/v3"
    "github.com/openai/openai-go/v3/option"
    "github.com/tinfoilsh/tinfoil-go"
)

func main() {
    client, err := tinfoil.NewClient(
        option.WithAPIKey(os.Getenv("TINFOIL_API_KEY")),
    )
    if err != nil {
        log.Fatal(err)
    }

    response, err := client.Chat.Completions.New(
        context.TODO(),
        openai.ChatCompletionNewParams{
            Model: "<MODEL_NAME>",
            Messages: []openai.ChatCompletionMessageParamUnion{
                openai.UserMessage("Generate an example email address for Alan Turing, who works in Enigma. End in .com."),
            },
        },
        option.WithJSONSet("structured_outputs", map[string]interface{}{
            "regex": `\w+@\w+\.com`,
        }),
    )
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(response.Choices[0].Message.Content)
}

use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("<MODEL_NAME>")
        .messages([json!(
            {"role": "user", "content": "Generate an example email address for Alan Turing, who works in Enigma. End in .com."}
        )])
        .structured_outputs_regex(r"\w+@\w+\.com");

    let response = client.chat_relaxed().create(body).await?;
    println!("{}", response.content().unwrap_or(""));
    Ok(())
}

JSON

Use response_format with json_schema type for reliable JSON generation:

from pydantic import BaseModel
from enum import Enum
from tinfoil import TinfoilAI

class CarType(str, Enum):
    sedan = "sedan"
    suv = "SUV"
    truck = "Truck"
    coupe = "Coupe"

class CarDescription(BaseModel):
    brand: str
    model: str
    car_type: CarType

client = TinfoilAI(api_key="<YOUR_API_KEY>")

json_schema = CarDescription.model_json_schema()

response = client.chat.completions.create(
    model="<MODEL_NAME>",
    messages=[
        {"role": "user", "content": "Output a JSON object with the brand, model, and car_type of the most iconic car from the 90's."}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "car-description",
            "schema": json_schema
        }
    }
)

print(response.choices[0].message.content)

import { TinfoilAI } from 'tinfoil';

const client = new TinfoilAI({
  apiKey: process.env.TINFOIL_API_KEY
});

const jsonSchema = {
  type: 'object',
  properties: {
    brand: { type: 'string' },
    model: { type: 'string' },
    car_type: { type: 'string', enum: ['sedan', 'SUV', 'Truck', 'Coupe'] }
  },
  required: ['brand', 'model', 'car_type']
};

const response = await client.chat.completions.create({
  model: '<MODEL_NAME>',
  messages: [
    { role: 'user', content: "Output a JSON object with the brand, model, and car_type of the most iconic car from the 90's." }
  ],
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'car-description',
      schema: jsonSchema
    }
  }
});

console.log(response.choices[0].message.content);

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "log"
    "os"

    "github.com/openai/openai-go/v3"
    "github.com/openai/openai-go/v3/option"
    "github.com/tinfoilsh/tinfoil-go"
)

type CarType string

const (
    Sedan CarType = "sedan"
    SUV   CarType = "SUV"
    Truck CarType = "Truck"
    Coupe CarType = "Coupe"
)

type CarDescription struct {
    Brand   string  `json:"brand"`
    Model   string  `json:"model"`
    CarType CarType `json:"car_type"`
}

func main() {
    client, err := tinfoil.NewClient(
        option.WithAPIKey(os.Getenv("TINFOIL_API_KEY")),
    )
    if err != nil {
        log.Fatal(err)
    }

    jsonSchema := map[string]interface{}{
        "type": "object",
        "properties": map[string]interface{}{
            "brand":    map[string]interface{}{"type": "string"},
            "model":    map[string]interface{}{"type": "string"},
            "car_type": map[string]interface{}{"type": "string", "enum": []string{"sedan", "SUV", "Truck", "Coupe"}},
        },
        "required": []string{"brand", "model", "car_type"},
    }

    response, err := client.Chat.Completions.New(
        context.TODO(),
        openai.ChatCompletionNewParams{
            Model: "<MODEL_NAME>",
            Messages: []openai.ChatCompletionMessageParamUnion{
                openai.UserMessage("Output a JSON object with the brand, model, and car_type of the most iconic car from the 90's."),
            },
        },
        option.WithJSONSet("response_format", map[string]interface{}{
            "type": "json_schema",
            "json_schema": map[string]interface{}{
                "name":   "car-description",
                "schema": jsonSchema,
            },
        }),
    )
    if err != nil {
        log.Fatal(err)
    }

    var car CarDescription
    json.Unmarshal([]byte(response.Choices[0].Message.Content), &car)
    fmt.Printf("Brand: %s, Model: %s, Type: %s\n", car.Brand, car.Model, car.CarType)
}

use serde::Deserialize;
use serde_json::json;
use tinfoil::Client;

#[derive(Debug, Deserialize)]
struct CarDescription {
    brand: String,
    model: String,
    car_type: String,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let json_schema = json!({
        "type": "object",
        "properties": {
            "brand":    {"type": "string"},
            "model":    {"type": "string"},
            "car_type": {"type": "string", "enum": ["sedan", "SUV", "Truck", "Coupe"]}
        },
        "required": ["brand", "model", "car_type"]
    });

    let body = client.chat_relaxed().request()
        .model("<MODEL_NAME>")
        .messages([json!(
            {"role": "user", "content": "Output a JSON object with the brand, model, and car_type of the most iconic car from the 90's."}
        )])
        .response_format_json_schema("car-description", json_schema);

    let response = client.chat_relaxed().create(body).await?;
    let content = response.content().unwrap_or("");
    let car: CarDescription = serde_json::from_str(content)?;
    println!("Brand: {}, Model: {}, Type: {}", car.brand, car.model, car.car_type);
    Ok(())
}

Prompt explicitly for JSON. While structured_outputs enforces valid JSON structure, the model produces more reliable results when your prompt explicitly requests JSON output and describes the expected fields. For example, use “Output a JSON object with…” rather than just “Generate…”

Advanced Features

Whitespace Pattern Override

Customize whitespace handling in JSON decoding by combining response_format with extra_body:

response = client.chat.completions.create(
    model="<MODEL_NAME>",
    messages=[...],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "my-schema",
            "schema": json_schema
        }
    },
    extra_body={
        "structured_outputs": {
            "whitespace_pattern": r"[ \t\n]*"
        }
    }
)

Complex Nested Schemas

Build complex nested structures with Pydantic:

Python

from pydantic import BaseModel
from tinfoil import TinfoilAI

class Address(BaseModel):
    street: str
    city: str
    state: str
    zip_code: str

class Employee(BaseModel):
    name: str
    age: int
    email: str | None
    addresses: list[Address]

class Company(BaseModel):
    name: str
    founded_year: int
    employees: list[Employee]
    headquarters: Address

client = TinfoilAI(api_key="<YOUR_API_KEY>")

response = client.chat.completions.create(
    model="<MODEL_NAME>",
    messages=[
        {"role": "user", "content": "Output a JSON object for a company profile. The company is named TechCorp and has 2 employees. Include the company name, founded_year, employees (each with name, age, email, and addresses), and headquarters address."}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "company",
            "schema": Company.model_json_schema()
        }
    }
)

import json
company_data = json.loads(response.choices[0].message.content)
print(f"Company: {company_data['name']}, Employees: {len(company_data['employees'])}")

Best Practices

Markdown-Wrapped Responses: Some models may wrap JSON responses in markdown code blocks (```json ... ```). Strip the formatting before parsing the JSON.

Use Low Temperature for Deterministic Outputs

response = client.chat.completions.create(
    model="<MODEL_NAME>",
    temperature=0.1,
    messages=[...],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "my-schema",
            "schema": schema
        }
    }
)

Validate Responses

from pydantic import ValidationError

try:
    parsed = MySchema.model_validate_json(response.choices[0].message.content)
except ValidationError as e:
    print(f"Validation failed: {e}")

Enable Streaming for Large Responses

response = client.chat.completions.create(
    model="<MODEL_NAME>",
    messages=[...],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "my-schema",
            "schema": schema
        }
    },
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Additional Resources

vLLM Structured Outputs Guide - Official documentation
BoundaryML - Reliable structured outputs from any model
Pydantic Documentation - Python schema validation
Zod Documentation - TypeScript schema validation

Getting Started

Model catalog

Tinfoil SDKs

Local Proxy

Tinfoil Containers

Guides

Verification & Attestation

Tutorials

Admin API

Resources

Structured outputs