​
Overview

Tinfoil uses confidential computing to provide a verifiably private runtime environment in the cloud. This verifiability property is realized through an attestation architecture which ensures that:

  1. The secure enclave environment is genuine and properly configured, as attested to by AMD and NVIDIA.

  2. Only immutable and publicly-auditable code is executed inside the secure enclave.

  3. Only static model weights are loaded into the inference engine.

​
Components

This attestation architecture is illustrated in Figure 1 and consists of the following components.

  • cvmimage: Confidential VM Image based on Ubuntu, with the CPU TEE compatible kernel, the vLLM inference server, our sev-shim and modelpack mount utils.
  • modelpack: Read only volume containing model weights or other immutable data
  • tinfoil-config.yml: Manifest for models, shim configuration, and dependency versions
  • sev-shim: Reverse proxy that runs inside the VM image and terminates TLS, enforces security policy, and serves the remote attestation document
  • pri-image-builder: Converts the tinfoil-config file into a deployment config and publishes a new Sigstore Bundle on the Sigstore transparency log
  • edk2 ovmf: UEFI boot firmware
  • Sigstore: Transparency log record containing source code measurements (i.e., a SHA256 hash of the compiled code)
  • Verifier (on Client Device): Checks that the source code and runtime measurements match and that the TLS connection matches the attested public key

Figure 1: Overview of Tinfoil’s attestation architecture.

​
Immutability

The Confidential VM (CVM) is inherently stateless. It has no persistent data whatsoever; all virtual disks are mounted as read-only. Consequently, we need a means to verify the integrity of the read-only disk images to ensure they haven’t been modified by an attacker on the host. We use dm-verity to create an attested measurement of the disk image which the CVM verifies at boot time. We use mkosi to build the rootfs and modelpack to create immutable disk images from huggingface model weights.

​
CPU-GPU Chain of Trust

Once the CVM verifies the integrity of the disks, it in turn queries the GPU to ensure it’s also configured correctly by NVIDIA. This creates a link between the CPU and GPU attestations. If the CPU fails to verify the GPU’s attestation, it aborts the boot process and returns an error.

​
Lifecycle

To run a model on Tinfoil, we first build the model into a deployment configuration, deploy it on our infrastructure, then verify it’s integrity on the client device.

​
Build-time

  1. Download model weights from huggingface or other model repository
  2. Use modelpack to create an immutable .mpk file of the weights (EROFS+dm-verity) and an info string ([root node hash]_[offset]_[block uuid]) that verifies the integrity of that file
  3. Create a tinfoil-config.yml with:
    • MPK info string for the model
    • sev-shim config (domains, path ACL, allowed CORS origins)
    • CVM image and OVMF firmware versions
    • Memory and vCPU core count
  4. Commit the config file and a GitHub Actions workflow for pri-build-action to a new repo
  5. Tag a release and pri-build-action publish the release including a measured deployment manifest
  6. Run the VM in QEMU

​
Runtime setup

  1. QEMU starts the VM
  2. When the CVM boots, our initialization process does the following:
    1. Creates a ramdisk for all ephemeral data
    2. Ensures the tinfoil-config file matches the attested hash provided in the kernel command line
    3. Checks the NVIDIA GPU attestation with NVIDIA’s local-gpu-verifier
    4. Uses modelpack to mount each model weight directory
    5. Applies sev-shim and vllm configurations from the attested config and starts each service

​
Connection-time verification

Before exchanging application data (e.g., chat completions) with an enclave, the verifier SDK completes the following checks:

  1. Fetches the attestation document from enclave which includes the signed runtime measurements
  2. Verifies the certificate chain in the attestation to the CPU’s hardcoded root certificate: (AMD)
  3. Fetches the Sigstore bundle from GitHub
  4. Verifies the Sigstore bundle to Sigstore’s root trust anchor
  5. Checks the measurement predicates to ensure the source code and runtime enclave measurements match
  6. Opens a TLS connection to enclave and ensures the public key offered by the remote server matches the public key included in the attestation document

​
Tinfoil Config File

The “tinfoil config file” is always called tinfoil-config.yml and placed at the root of a deployment repo. The private image builder action parses this file to create an attested deployment config and includes the SHA256 hash of the entire file as a kernel command line parameter to provide a cryptographic link to the running enclave.

For example, our DeepSeek R1 deployment:

cvm-version: 0.0.27
ovmf-version: 0.0.2
cpus: 8
memory: 32768

models:
  - name: "deepseek-r1-70b"
    repo: "casperhansen/deepseek-r1-distill-llama-70b-awq@a1ab7653aae77fbabc536cbcbac5bb2e2fb5354f"
    mpk: "8e39a53227ccb0c3cffbed1c0013d4d63c74c1e01541b953ff021e91cb158330_39785418752_efe58861-8b9c-5e64-b0ee-85d9169acb44"
vllm-args: --quantization awq_marlin --max-model-len 65536

shim: # The shim config is passed directly to sev-shim. See https://github.com/tinfoilsh/sev-shim
  domains:
    - deepseek-r1-70b-p.model.tinfoil.sh
  listen-port: 443
  upstream-port: 8080 # Port of internal service (vllm)
  control-plane: https://api.tinfoil.sh
  paths: # Path ACL
    - /v1/chat/completions
    - /metrics
  origins: # for CORS
    - https://tinfoil.sh
    - https://chat.tinfoil.sh