Rust

Rust SDK

Rust SDK for Tinfoil’s secure AI inference API GitHub: tinfoil-rs

Overview

The Tinfoil Rust SDK is a thin wrapper around the async-openai crate that provides secure communication with Tinfoil enclaves. It uses the same types and request/response shapes as async-openai and adds automatic verification that the endpoint is running in a secure Tinfoil enclave, TLS certificate pinning, and attestation validation.

Installation

New to Rust? Start here - Project Setup

If you don’t have a Cargo project yet, create one:

cargo new my-tinfoil-app
cd my-tinfoil-app

The minimum supported Rust version is 1.87.

Add the SDK and a runtime to Cargo.toml:

cargo add tinfoil --git https://github.com/tinfoilsh/tinfoil-rs
cargo add tokio --features full
cargo add serde --features derive
cargo add serde_json futures-util base64
cargo add [email protected] --no-default-features --features json,rustls,stream,multipart

Migration from `async-openai`

Migrating from async-openai to Tinfoil is a small change — the SDK exposes the same chat / audio / embeddings handlers under tinfoil::Client:

// Before (async-openai)
- use async_openai::{Client, config::OpenAIConfig};
-
- let client = Client::with_config(
-     OpenAIConfig::new().with_api_key("OPENAI_API_KEY")
- );

// After (Tinfoil)
+ use tinfoil::Client;
+
+ let client = Client::new_default().await?;

tinfoil::Client derefs to async_openai::Client, so all OpenAI methods (client.chat(), client.embeddings(), client.audio(), …) work unchanged.

Usage

use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Create a client
    let client = Client::new_default().await?;

    // 2. Use client as you would async_openai::Client
    // see https://docs.rs/async-openai for API documentation
    Ok(())
}

The common request/response types live under tinfoil::chat, tinfoil::audio, and tinfoil::embeddings — all re-exports of the corresponding async_openai::types::* modules.

Model Examples

Below are specific examples for current Tinfoil model categories.

Chat Models

https://mintcdn.com/tinfoil/7kpqELCdP4WIVCil/images/model-icons/deepseek.png?fit=max&auto=format&n=7kpqELCdP4WIVCil&q=85&s=8b0e67d9e8e4077d980d15f7fe14ca56

DeepSeek V4 Pro

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("deepseek-v4-pro")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Summarize the key risks in a large technical migration plan.".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

https://mintcdn.com/tinfoil/GXlDUK5K0yNgB7O7/images/model-icons/zai.png?fit=max&auto=format&n=GXlDUK5K0yNgB7O7&q=85&s=36bb4bfa914e58deb8fbde1eba1af343

GLM-5.1

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("glm-5-1")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Solve this step by step: If a train travels 120 miles in 2 hours, \
                     and then increases its speed by 25% for the next 3 hours, how far \
                     does it travel in total?".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

https://mintcdn.com/tinfoil/7kpqELCdP4WIVCil/images/model-icons/qwen.png?fit=max&auto=format&n=7kpqELCdP4WIVCil&q=85&s=ceccb0e82dd42dae1af15f79d28fe557

Qwen3-VL 30B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestMessageContentPartImage,
    ChatCompletionRequestUserMessage, ChatCompletionRequestUserMessageContent,
    ChatCompletionRequestUserMessageContentPart, CreateChatCompletionRequestArgs, ImageUrl,
};
use tinfoil::multimodal::ImageUrlExt;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    // ImageUrlExt::from_path encodes the file as a base64 data URL.
    let image = ImageUrl::from_path("image.jpg")?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("qwen3-vl-30b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Array(vec![
                    ChatCompletionRequestUserMessageContentPart::Text(
                        "What's in this image?".into(),
                    ),
                    ChatCompletionRequestUserMessageContentPart::ImageUrl(
                        ChatCompletionRequestMessageContentPartImage { image_url: image },
                    ),
                ]),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Kimi K2.6

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("kimi-k2-6")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Review this codebase and suggest architectural improvements for \
                     better modularity and testability.".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

https://mintcdn.com/tinfoil/vfMAM73wcpT2SrpR/images/model-icons/gemma.png?fit=max&auto=format&n=vfMAM73wcpT2SrpR&q=85&s=7b20f1c96812bd1c9fd512c4fead47c7

Gemma 4 31B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("gemma4-31b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Explain the trade-offs between accuracy and latency in a production AI system.".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

https://mintcdn.com/tinfoil/7kpqELCdP4WIVCil/images/model-icons/llama.png?fit=max&auto=format&n=7kpqELCdP4WIVCil&q=85&s=0e9748643c3987fbf26b9b7531745bc9

Llama 3.3 70B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("llama3-3-70b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "What are the key differences between renewable and non-renewable \
                     energy sources?".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

https://mintcdn.com/tinfoil/7kpqELCdP4WIVCil/images/model-icons/openai.png?fit=max&auto=format&n=7kpqELCdP4WIVCil&q=85&s=1b9cbe9cd3cbfe37d282867e0d448c69

GPT-OSS 120B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("gpt-oss-120b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Analyze the trade-offs between different database architectures for \
                     a high-traffic e-commerce platform.".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Audio Models

https://mintcdn.com/tinfoil/vfMAM73wcpT2SrpR/images/model-icons/mistral.png?fit=max&auto=format&n=vfMAM73wcpT2SrpR&q=85&s=c629064347c0c847d788dc2b2e0c1e10

Voxtral Small 24B

Transcription

The transcribe shortcut on Client forwards to client.audio().transcription().create(request).

use std::path::PathBuf;
use tinfoil::async_openai::types::InputSource;
use tinfoil::audio::{AudioInput, CreateTranscriptionRequestArgs};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateTranscriptionRequestArgs::default()
        .file(AudioInput {
            source: InputSource::Path { path: PathBuf::from("meeting_recording.mp3") },
        })
        .model("voxtral-small-24b")
        .language("en")
        .prompt("This is a business meeting discussing quarterly results")
        .build()?;

    let transcription = client.transcribe(request).await?;
    println!("Transcription: {}", transcription.text);
    Ok(())
}

Audio Q&A

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestMessageContentPartAudio,
    ChatCompletionRequestUserMessage, ChatCompletionRequestUserMessageContent,
    ChatCompletionRequestUserMessageContentPart, CreateChatCompletionRequestArgs,
    InputAudio,
};
use tinfoil::multimodal::InputAudioExt;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let audio = InputAudio::from_path("audio.mp3")?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("voxtral-small-24b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Array(vec![
                    ChatCompletionRequestUserMessageContentPart::Text(
                        "What is the speaker talking about in this audio?".into(),
                    ),
                    ChatCompletionRequestUserMessageContentPart::InputAudio(
                        ChatCompletionRequestMessageContentPartAudio { input_audio: audio },
                    ),
                ]),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Embedding Models

https://mintcdn.com/tinfoil/7kpqELCdP4WIVCil/images/model-icons/nomic.png?fit=max&auto=format&n=7kpqELCdP4WIVCil&q=85&s=6568d64fd1a8b8110bf3354c1c8e78cd

Nomic Embed Text

The embed_batch shortcut takes a list of strings, preserves input order, and returns Vec<Vec<f32>>.

use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let documents = [
        "Artificial intelligence is transforming modern technology.",
        "Machine learning enables computers to learn from data.",
        "The weather today is sunny and warm.",
        "Deep learning uses neural networks with multiple layers.",
    ];

    let embeddings = client.embed_batch("nomic-embed-text", documents.iter().copied()).await?;

    let similarity = cosine_similarity(&embeddings[0], &embeddings[1]);
    println!("Similarity between first two AI-related documents: {:.3}", similarity);
    println!("Embedding dimension: {}", embeddings[0].len());
    Ok(())
}

fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
    let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
    let norm_a: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
    let norm_b: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
    if norm_a == 0.0 || norm_b == 0.0 { 0.0 } else { dot / (norm_a * norm_b) }
}

Vendor Extensions

Tinfoil exposes a few request/response fields the upstream OpenAI schema does not (for example web_search_options, pii_check_options, structured-output regex / choice constraints, and custom finish_reason values). To use them without async-openai rejecting the response, send the request through client.chat_relaxed():

use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("gpt-oss-120b")
        .messages([json!({ "role": "user", "content": "What's the latest on AI safety?" })])
        .web_search_with_options(json!({ "max_results": 5 }));

    let response = client.chat_relaxed().create(body).await?;
    if let Some(content) = response.content() {
        println!("{}", content);
    }
    Ok(())
}

Streaming uses the same builder and surfaces vendor-specific events (such as web_search_call) verbatim:

use futures_util::StreamExt;
use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("gpt-oss-120b")
        .messages([json!({ "role": "user", "content": "Explain quantum entanglement" })]);

    let mut stream = client.chat_relaxed().create_stream(body).await?;

    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        if let Some(delta) = chunk.delta_content() {
            print!("{}", delta);
        } else if chunk.is_vendor_event() {
            println!("[event] {:?}", chunk.event_type());
        }
    }
    Ok(())
}

Advanced Functionality

For advanced use cases requiring manual verification or direct HTTP access, use SecureClient:

use tinfoil::SecureClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Construct a secure client for manual verification and HTTP access
    let mut secure = SecureClient::new(
        "inference.tinfoil.sh",
        "tinfoilsh/confidential-model-router",
        std::env::var("TINFOIL_API_KEY")?,
    );

    // Verify the enclave attestation
    let ground_truth = secure.verify().await?;
    let enclave_fp = ground_truth.enclave_fingerprint.clone();
    let code_fp = ground_truth.code_fingerprint.clone();

    println!("Enclave: {}", secure.host());
    println!("Enclave fingerprint: {}", enclave_fp);
    println!("Code fingerprint: {}", code_fp);

    // Get the verified HTTP client for custom requests (multipart uploads,
    // unofficial endpoints like /v1/convert/file, etc.)
    let _http = secure.http_client()?;
    Ok(())
}

API Documentation

This library is a thin wrapper around the async-openai crate that can be used with Tinfoil. All methods and types are identical. See the async-openai documentation for complete API usage.

Guides

Web Search

Enable AI models to search the web with optional PII protection.

Tool Calling

Learn how to use function calling capabilities with AI models.

Structured Outputs

Use JSON schema validation for reliable data extraction.

Image Processing

Process images with multimodal AI models.

Document Processing

Upload and process documents securely.

Getting Started

Model catalog

Tinfoil SDKs

Tinfoil Containers

Guides

Verification & Attestation

Tutorials

Admin API

Resources

Rust SDK

Overview

Installation

Migration from `async-openai`

Usage

Model Examples

Chat Models

Audio Models

Transcription

Audio Q&A

Embedding Models

Vendor Extensions

Advanced Functionality

API Documentation

Guides

Web Search

Tool Calling

Structured Outputs

Image Processing

Document Processing

Getting Started

Model catalog

Tinfoil SDKs

Tinfoil Containers

Guides

Verification & Attestation

Tutorials

Admin API

Resources

Documentation Index

Rust SDK

​Overview

​Installation

​Migration from async-openai

​Usage

​Model Examples

​Chat Models

​Audio Models

​Transcription

​Audio Q&A

​Embedding Models

​Vendor Extensions

​Advanced Functionality

​API Documentation

​Guides

Web Search

Tool Calling

Structured Outputs

Image Processing

Document Processing

Overview

Installation

Migration from `async-openai`

Usage

Model Examples

Chat Models

Audio Models

Transcription

Audio Q&A

Embedding Models

Vendor Extensions

Advanced Functionality

API Documentation

Guides