Rust

View on GitHub

tinfoilsh/tinfoil-rs

Overview

The Tinfoil Rust SDK is a thin wrapper around the async-openai crate that provides secure communication with Tinfoil enclaves. It uses the same types and request/response shapes as async-openai and adds automatic verification that the endpoint is running in a secure Tinfoil enclave, TLS certificate pinning, and attestation validation.

Installation

New to Rust? Start here - Project Setup

If you don’t have a Cargo project yet, create one:

cargo new my-tinfoil-app
cd my-tinfoil-app

The minimum supported Rust version is 1.87.

Add the SDK and a runtime to Cargo.toml:

cargo add tinfoil --git https://github.com/tinfoilsh/tinfoil-rs
cargo add tokio --features full
cargo add serde --features derive
cargo add serde_json futures-util base64
cargo add [email protected] --no-default-features --features json,rustls,stream,multipart

Migration from `async-openai`

Migrating from async-openai to Tinfoil is a small change — the SDK exposes the same chat / audio / embeddings handlers under tinfoil::Client:

// Before (async-openai)
- use async_openai::{Client, config::OpenAIConfig};
-
- let client = Client::with_config(
-     OpenAIConfig::new().with_api_key("OPENAI_API_KEY")
- );

// After (Tinfoil)
+ use tinfoil::Client;
+
+ let client = Client::new_default().await?;

tinfoil::Client derefs to async_openai::Client, so all OpenAI methods (client.chat(), client.embeddings(), client.audio(), …) work unchanged.

Usage

use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Create a client
    let client = Client::new_default().await?;

    // 2. Use client as you would async_openai::Client
    // see https://docs.rs/async-openai for API documentation
    Ok(())
}

The common request/response types live under tinfoil::chat, tinfoil::audio, and tinfoil::embeddings — all re-exports of the corresponding async_openai::types::* modules.

Model Examples

Below are specific examples for current Tinfoil model categories.

Chat Models

GLM-5.2

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("glm-5-2")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Solve this step by step: If a train travels 120 miles in 2 hours, \
                     and then increases its speed by 25% for the next 3 hours, how far \
                     does it travel in total?".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Qwen3-VL 30B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestMessageContentPartImage,
    ChatCompletionRequestUserMessage, ChatCompletionRequestUserMessageContent,
    ChatCompletionRequestUserMessageContentPart, CreateChatCompletionRequestArgs, ImageUrl,
};
use tinfoil::multimodal::ImageUrlExt;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    // ImageUrlExt::from_path encodes the file as a base64 data URL.
    let image = ImageUrl::from_path("image.jpg")?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("qwen3-vl-30b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Array(vec![
                    ChatCompletionRequestUserMessageContentPart::Text(
                        "What's in this image?".into(),
                    ),
                    ChatCompletionRequestUserMessageContentPart::ImageUrl(
                        ChatCompletionRequestMessageContentPartImage { image_url: image },
                    ),
                ]),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Gemma 4 31B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("gemma4-31b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Explain the trade-offs between accuracy and latency in a production AI system.".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Llama 3.3 70B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("llama3-3-70b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "What are the key differences between renewable and non-renewable \
                     energy sources?".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

GPT-OSS 120B

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestUserMessage,
    ChatCompletionRequestUserMessageContent, CreateChatCompletionRequestArgs,
};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("gpt-oss-120b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Text(
                    "Analyze the trade-offs between different database architectures for \
                     a high-traffic e-commerce platform.".to_string(),
                ),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Audio Models

Voxtral Small 24B

Transcription

The transcribe shortcut on Client forwards to client.audio().transcription().create(request).

use std::path::PathBuf;
use tinfoil::async_openai::types::InputSource;
use tinfoil::audio::{AudioInput, CreateTranscriptionRequestArgs};
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let request = CreateTranscriptionRequestArgs::default()
        .file(AudioInput {
            source: InputSource::Path { path: PathBuf::from("meeting_recording.mp3") },
        })
        .model("voxtral-small-24b")
        .language("en")
        .prompt("This is a business meeting discussing quarterly results")
        .build()?;

    let transcription = client.transcribe(request).await?;
    println!("Transcription: {}", transcription.text);
    Ok(())
}

Audio Q&A

use tinfoil::chat::{
    ChatCompletionRequestMessage, ChatCompletionRequestMessageContentPartAudio,
    ChatCompletionRequestUserMessage, ChatCompletionRequestUserMessageContent,
    ChatCompletionRequestUserMessageContentPart, CreateChatCompletionRequestArgs,
    InputAudio,
};
use tinfoil::multimodal::InputAudioExt;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let audio = InputAudio::from_path("audio.mp3")?;

    let request = CreateChatCompletionRequestArgs::default()
        .model("voxtral-small-24b")
        .messages(vec![ChatCompletionRequestMessage::User(
            ChatCompletionRequestUserMessage {
                content: ChatCompletionRequestUserMessageContent::Array(vec![
                    ChatCompletionRequestUserMessageContentPart::Text(
                        "What is the speaker talking about in this audio?".into(),
                    ),
                    ChatCompletionRequestUserMessageContentPart::InputAudio(
                        ChatCompletionRequestMessageContentPartAudio { input_audio: audio },
                    ),
                ]),
                name: None,
            },
        )])
        .build()?;

    let response = client.chat().create(request).await?;
    println!("{}", response.choices[0].message.content.as_deref().unwrap_or(""));
    Ok(())
}

Embedding Models

Nomic Embed Text

The embed_batch shortcut takes a list of strings, preserves input order, and returns Vec<Vec<f32>>.

use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let documents = [
        "Artificial intelligence is transforming modern technology.",
        "Machine learning enables computers to learn from data.",
        "The weather today is sunny and warm.",
        "Deep learning uses neural networks with multiple layers.",
    ];

    let embeddings = client.embed_batch("nomic-embed-text", documents.iter().copied()).await?;

    let similarity = cosine_similarity(&embeddings[0], &embeddings[1]);
    println!("Similarity between first two AI-related documents: {:.3}", similarity);
    println!("Embedding dimension: {}", embeddings[0].len());
    Ok(())
}

fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
    let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
    let norm_a: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
    let norm_b: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
    if norm_a == 0.0 || norm_b == 0.0 { 0.0 } else { dot / (norm_a * norm_b) }
}

Vendor Extensions

Tinfoil exposes a few request/response fields the upstream OpenAI schema does not (for example web_search_options, pii_check_options, structured-output regex / choice constraints, and custom finish_reason values). To use them without async-openai rejecting the response, send the request through client.chat_relaxed():

use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("gpt-oss-120b")
        .messages([json!({ "role": "user", "content": "What's the latest on AI safety?" })])
        .web_search_with_options(json!({ "max_results": 5 }));

    let response = client.chat_relaxed().create(body).await?;
    if let Some(content) = response.content() {
        println!("{}", content);
    }
    Ok(())
}

Streaming uses the same builder and surfaces vendor-specific events (such as web_search_call) verbatim:

use futures_util::StreamExt;
use serde_json::json;
use tinfoil::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new_default().await?;

    let body = client.chat_relaxed().request()
        .model("gpt-oss-120b")
        .messages([json!({ "role": "user", "content": "Explain quantum entanglement" })]);

    let mut stream = client.chat_relaxed().create_stream(body).await?;

    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        if let Some(delta) = chunk.delta_content() {
            print!("{}", delta);
        } else if chunk.is_vendor_event() {
            println!("[event] {:?}", chunk.event_type());
        }
    }
    Ok(())
}

Advanced Functionality

For advanced use cases requiring manual verification or direct HTTP access, use SecureClient:

use tinfoil::SecureClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Construct a secure client for manual verification and HTTP access
    let mut secure = SecureClient::new(
        "inference.tinfoil.sh",
        "tinfoilsh/confidential-model-router",
        std::env::var("TINFOIL_API_KEY")?,
    );

    // Verify the enclave attestation
    let ground_truth = secure.verify().await?;
    let enclave_fp = ground_truth.enclave_fingerprint.clone();
    let code_fp = ground_truth.code_fingerprint.clone();

    println!("Enclave: {}", secure.host());
    println!("Enclave fingerprint: {}", enclave_fp);
    println!("Code fingerprint: {}", code_fp);

    // Get the verified HTTP client for custom requests (multipart uploads,
    // unofficial endpoints like /v1/convert/file, etc.)
    let _http = secure.http_client()?;
    Ok(())
}

API Documentation

This library is a thin wrapper around the async-openai crate that can be used with Tinfoil. All methods and types are identical. See the async-openai documentation for complete API usage.

Getting Started

Model catalog

Tinfoil SDKs

Local Proxy

Tinfoil Containers

Guides

Verification & Attestation

Tutorials

Admin API

Resources

View on GitHub

Overview

Installation

Migration from `async-openai`

Usage

Model Examples

Chat Models

Audio Models

Transcription

Audio Q&A

Embedding Models

Vendor Extensions

Advanced Functionality

API Documentation

View on GitHub

​Overview

​Installation

​Migration from async-openai

​Usage

​Model Examples

​Chat Models

​Audio Models

​Transcription

​Audio Q&A

​Embedding Models

​Vendor Extensions

​Advanced Functionality

​API Documentation

Overview

Installation

Migration from `async-openai`

Usage

Model Examples

Chat Models

Audio Models

Transcription

Audio Q&A

Embedding Models

Vendor Extensions

Advanced Functionality

API Documentation