> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tinfoil.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Persistent storage with the buckets sidecar

> Store end-user data in S3 from a Tinfoil Container without exposing plaintext outside the enclave.

Tinfoil Containers have [no persistent disk](/containers/overview#limitations): the enclave filesystem is a ramdisk, and everything on it is lost on restart or redeploy. The standard way to persist data is object storage like Amazon S3, but writing plaintext to S3 would hand your users' data to the storage provider and break the enclave's privacy guarantees.

The [tinfoil-buckets-sidecar](https://github.com/tinfoilsh/tinfoil-buckets-sidecar) solves this. It runs as a second container inside your enclave and exposes an S3-compatible API on the enclave's internal network. On every write it encrypts the object with AES-256-GCM before forwarding it to S3; on every read it fetches the ciphertext, decrypts it, and verifies the authentication tag. Plaintext and keys only ever exist inside the secure enclave. S3 (and anyone with access to the bucket) sees only ciphertext.

```mermaid theme={"dark"}
flowchart LR
    user["End user<br/>(holds key)"] -->|request + key| app
    subgraph enclave["Secure enclave"]
        app["Your app"] -->|S3 API + key headers| sidecar["Buckets sidecar (:9000)<br/>encrypts / decrypts"]
    end
    sidecar -->|ciphertext only| s3[("Amazon S3")]
```

## End users hold the keys

You operate the enclave, but the data belongs to your end users, so each user's data is sealed under a key only that user holds. Keys cannot live inside the enclave: it has no persistent disk, so a key generated and kept there would be lost on the first restart, making everything stored under it unrecoverable.

Instead, each end user generates a 32-byte AES-256 key on their own device (or derives one from a passphrase using a key derivation function such as HKDF) and sends it along with each request to your app. Your app authenticates the user, then forwards the key to the sidecar in a per-request header. The key exists inside the enclave only for the lifetime of the request and is never persisted, so neither you as the operator nor Tinfoil can decrypt the stored data.

<Warning>
  Each user's key is the confidentiality boundary for their data. A user who loses their key permanently loses everything stored under it, so client applications must store keys durably (device keychain, passkey-wrapped backup, or similar).
</Warning>

## Tutorial

This walks through adding encrypted S3 storage to an existing container deployment. For a complete runnable deployment, see the [persistent storage example](https://github.com/tinfoilsh/tinfoil-persistent-storage-example).

### 1. Create a bucket and IAM credentials

Create an S3 bucket and an IAM user whose credentials the sidecar will use. Attach a policy scoped to the bucket(s) the sidecar should reach — IAM is the enforcement point for which buckets are accessible:

```json theme={"dark"}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": ["arn:aws:s3:::YOUR-BUCKET/*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads",
        "s3:GetBucketLocation"
      ],
      "Resource": ["arn:aws:s3:::YOUR-BUCKET"]
    }
  ]
}
```

Add the credentials as two [secrets](/containers/secrets-and-env-vars) in the Tinfoil dashboard (or via `tinfoil secret create`): `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. These credentials only ever touch ciphertext.

### 2. Add the sidecar to your config

Run the sidecar alongside your app in multitenant mode and put both on a shared [network](/containers/configuration#networking). The sidecar needs egress to reach S3; your app reaches it at `http://buckets:9000` by container name:

```yaml tinfoil-config.yml theme={"dark"}
cvm-version: 0.10.4
cpus: 4
memory: 8192

networks:
  storage:
    egress: open

containers:
  - name: "app"
    image: "ghcr.io/myorg/my-app:v1.0.0@sha256:abc123..."
    networks: [storage]
    env:
      - BUCKETS_URL: "http://buckets:9000"

  - name: "buckets"
    image: "ghcr.io/tinfoilsh/tinfoil-buckets-sidecar@sha256:def456..."
    networks: [storage]
    env:
      - PORT: "9000"
      - AWS_REGION: "us-east-2"
      - MULTITENANT: "true"
    secrets:
      - AWS_ACCESS_KEY_ID
      - AWS_SECRET_ACCESS_KEY

shim:
  upstream-port: 8080
  paths:
    - /health
    - /api/*
```

### 3. Have each end user generate a key

Client applications generate 32 random bytes per user, base64-encode them, and store the key durably on the user's side. The equivalent of:

```bash theme={"dark"}
openssl rand -base64 32
```

Alternatively, derive the key from a passphrase the user already holds, so there is nothing extra to store. The client sends this key to your app with each request that reads or writes stored data. The connection to the enclave is attested and encrypted end-to-end, so the key is only ever visible inside the enclave.

### 4. Make per-user requests from your app

For each authenticated request, your app forwards two headers to the sidecar: the user's key, and a tenant id your app derives from the user's verified identity (never from user input). Any S3 SDK works, configured with the sidecar as the endpoint, path-style addressing, and sequential single-request transfers; credentials can be any throwaway values, since SDKs refuse to send unsigned requests:

```python theme={"dark"}
import boto3
from botocore.config import Config
from boto3.s3.transfer import TransferConfig

s3 = boto3.client(
    "s3",
    endpoint_url="http://buckets:9000",
    region_name="us-east-2",
    aws_access_key_id="tinfoil-sidecar",
    aws_secret_access_key="tinfoil-sidecar",
    config=Config(s3={"addressing_style": "path"}),
)

# Attach the authenticated user's tenant id and key to every request
def tenant_headers(request, **kwargs):
    request.headers["X-Tinfoil-Tenant-Id"] = f"user-{verified_user_id}"
    request.headers["X-Tinfoil-Encryption-Key"] = user_key_base64

s3.meta.events.register("before-send.s3", tenant_headers)

# Force sequential multipart uploads and disable ranged-GET downloads
tc = TransferConfig(max_concurrency=1, multipart_threshold=5 * 1024**4)

s3.upload_file("./upload.bin", "YOUR-BUCKET", "docs/report.pdf", Config=tc)
s3.download_file("YOUR-BUCKET", "docs/report.pdf", "./roundtrip.pdf", Config=tc)
```

The bucket comes from the request path (`s3://bucket/key`), so one sidecar can serve multiple buckets; reachability is enforced by the IAM policy from step 1. Equivalent Java and AWS CLI setups are in the sidecar's [guides](https://github.com/tinfoilsh/tinfoil-buckets-sidecar/tree/main/guides).

## How multitenancy works

* Objects are transparently namespaced under `<tenantId>/` in the backing bucket, so one tenant can never address another's objects. Listing and reads within a tenant see plain object keys with the prefix stripped.
* The sidecar trusts the two headers — your app is the authentication layer, which is why the tenant id must come from the user's verified identity.
* Key and tenant id are decoupled, so a user can rotate keys without moving data. The caller tracks which key decrypts which object; the wrong key returns `400 DecryptionFailed`.

If the enclave stores only its own operational data rather than per-user data, the sidecar also runs in single-key mode: omit `MULTITENANT` and supply one `ENCRYPTION_KEY` secret instead. See the [sidecar README](https://github.com/tinfoilsh/tinfoil-buckets-sidecar#configure).

## Differences from normal S3

Because the sidecar maintains streaming AES-GCM cipher state, a few S3 behaviors differ:

| Behavior          | Constraint                                                                                                      |
| :---------------- | :-------------------------------------------------------------------------------------------------------------- |
| Addressing        | Path-style only (`forcePathStyle` or equivalent)                                                                |
| Multipart uploads | Sequential parts (`max_concurrency=1`); non-last parts must be multiples of 16 bytes                            |
| Ranged GETs       | Not supported — download whole objects                                                                          |
| GET responses     | Buffered in memory to verify the authentication tag; 1 GiB default, configurable via `BUFFER_SIZE` up to 64 GiB |
| ETags             | Synthetic — not usable for client-side integrity checks                                                         |

High-level SDK transfer managers satisfy the multipart constraints with the configuration shown above. For objects larger than the buffer cap, the sidecar supports a streaming mode (`DANGEROUS_DELAYED_AUTH`) that requires a verifying client; see the [design notes](https://github.com/tinfoilsh/tinfoil-buckets-sidecar#design-decisions-constraints) before using it.

## Relevant repositories

* [tinfoil-buckets-sidecar](https://github.com/tinfoilsh/tinfoil-buckets-sidecar) — the sidecar itself, with CLI and SDK guides
* [tinfoil-persistent-storage-example](https://github.com/tinfoilsh/tinfoil-persistent-storage-example) — minimal deployable example