# Migrate from Azure Blob Storage to Tigris Azure Blob Storage is not S3-compatible. Tigris shadow buckets require an S3-compatible source. You cannot point a Tigris shadow bucket at Azure Blob Storage directly. To migrate from Azure Blob Storage, you must put an S3-compatible shim in front of your Azure Blob account. This guide uses [s3proxy](https://github.com/gaul/s3proxy), an open-source proxy that exposes an S3 API and forwards requests to Azure Blob Storage. You point the Tigris shadow bucket at the s3proxy endpoint. Tigris then migrates your data with zero downtime. note This setup adds a component that you must run and operate. s3proxy sits between Tigris and Azure Blob Storage. For a native S3 source, such as AWS S3, Google Cloud Storage, Cloudflare R2, or MinIO, see the [provider-specific guides](/docs/migration/.md). Those sources need no shim. ## Why an S3 shim is required[​](#why-an-s3-shim-is-required "Direct link to Why an S3 shim is required") Tigris is [S3-compatible](/docs/api/s3/.md). Shadow bucket migration speaks the S3 API to your source bucket. Azure Blob Storage uses a different API and does not accept S3 requests. Tigris cannot read from it directly. s3proxy solves this. s3proxy presents an S3 endpoint to Tigris. s3proxy translates each S3 request into an Azure Blob Storage request. Tigris sees a standard S3 source. Azure Blob Storage stays unchanged. ## Migration approach[​](#migration-approach "Direct link to Migration approach") Tigris supports [lazy migration](/docs/migration/.md) with **shadow buckets**. Tigris does not copy all your data upfront. Instead, Tigris fetches objects from the s3proxy endpoint on demand. It caches them for future access. No downtime is required. You can also enable **write-through** mode, which syncs new writes back through s3proxy to your Azure Blob container. Your Azure Blob container stays up to date throughout the migration. You can take as long as you need before you complete the cutover. ## Prerequisites[​](#prerequisites "Direct link to Prerequisites") Before you start, verify that you have: * A [Tigris account](https://console.storage.dev) with a bucket created * An Azure Storage account and the container you want to migrate from * An Entra ID identity with the **Storage Blob Data Reader** role on that container (recommended), or the storage account access key * A host to run s3proxy that Tigris can reach over the public internet * s3proxy 4.0.0 or newer, and Java 17 or newer on that host * No blobs in the **Archive** access tier (see the next section) note This guide uses the configuration properties of s3proxy 4.0.0. Releases before 4.0.0 use a different Azure backend with different properties and support account keys only. ## Rehydrate archived blobs before you migrate[​](#rehydrate-archived-blobs-before-you-migrate "Direct link to Rehydrate archived blobs before you migrate") s3proxy cannot read blobs in the Azure **Archive** access tier. Archive-tier blobs are offline in Azure. When the migration reaches an archived blob, the `tigris buckets migrate` command fails with the message `an internal error occurred, try again`. The message does not name the archived blob as the cause. Containers with lifecycle management policies commonly contain archived blobs, because those policies move cold objects to the Archive tier automatically. No S3 shim reads archived blobs transparently. You must rehydrate them to the **Hot** or **Cool** tier before you migrate. A retry after a failed drain is expensive. The `tigris buckets migrate` command walks the full container again from the start on each run. Rehydrate all archived blobs first, then run the drain once. Rehydration changes blob data. Run the commands in this section with your own Azure identity, not with the read-only migration identity from the prerequisites. The commands use `--auth-mode login` to authenticate as the identity from `az login`. That identity must have the **Storage Blob Data Contributor** role, or a higher role, on the container. The **Storage Blob Data Reader** role can list blobs but cannot change their tier. To find the archived blobs in a container: ``` az storage blob list \ --account-name \ --container-name \ --auth-mode login \ --query "[?properties.blobTier=='Archive'].name" \ --output tsv ``` To rehydrate one blob, set its tier to `Hot` or `Cool`: ``` az storage blob set-tier \ --account-name \ --container-name \ --auth-mode login \ --name \ --tier Hot ``` To rehydrate every archived blob, pipe the list into a loop that runs the same command one time for each blob: ``` az storage blob list \ --account-name \ --container-name \ --auth-mode login \ --query "[?properties.blobTier=='Archive'].name" \ --output tsv | while IFS= read -r blob_name; do az storage blob set-tier \ --account-name \ --container-name \ --auth-mode login \ --name "$blob_name" \ --tier Hot done ``` The loop reads one blob name from each line and keeps spaces, quotes, and backslashes in the name intact. A blob name that contains a newline character is the one exception. Rehydrate such a blob with the single-blob command. caution Rehydration is not immediate. A standard-priority rehydration can take up to 15 hours. Wait until no blob reports `Archive` before you start the migration. ## Step 1: Configure s3proxy for Azure Blob Storage[​](#step-1-configure-s3proxy-for-azure-blob-storage "Direct link to Step 1: Configure s3proxy for Azure Blob Storage") s3proxy reads its settings from a properties file. Create a file named `s3proxy.conf` with the Azure Blob backend. s3proxy 4.0.0 authenticates to Azure Blob Storage in two ways. Entra ID is the recommended way because you can scope it to one container with a read-only role. The storage account access key is the fallback. caution Do not deploy the example credential values. Anyone with these public values can read your Azure Blob objects. The s3proxy endpoint must stay publicly reachable. Generate your own strong, unique S3 credentials for `s3proxy.identity` and `s3proxy.credential`. Use the same credential values in both the s3proxy config and the Tigris shadow-bucket command. The `s3proxy.identity` value must match the `--access-key` value. The `s3proxy.credential` value must match the `--secret-key` value. Replace the example credential values with your own secrets. * Entra ID (recommended) * Account key (fallback) Create a service principal with the **Storage Blob Data Reader** role. Scope the role to the one container that you migrate from: ``` az ad sp create-for-rbac \ --name s3proxy-migration \ --role "Storage Blob Data Reader" \ --scopes "/subscriptions//resourceGroups//providers/Microsoft.Storage/storageAccounts//blobServices/default/containers/" ``` Leave `jclouds.identity` and `jclouds.credential` empty in the properties file. s3proxy then authenticates with `DefaultAzureCredential` from the Azure SDK: ``` s3proxy.endpoint=http://0.0.0.0:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential s3proxy.read-only-blobstore=true jclouds.provider=azureblob jclouds.identity= jclouds.credential= jclouds.endpoint=https://.blob.core.windows.net ``` Give the service principal credentials to s3proxy through environment variables. Set them in the environment that starts s3proxy in Step 2: ``` export AZURE_CLIENT_ID= export AZURE_TENANT_ID= export AZURE_CLIENT_SECRET= ``` note `DefaultAzureCredential` also supports managed identities. If s3proxy runs on an Azure VM with a managed identity, assign the **Storage Blob Data Reader** role to that identity and set no environment variables. warning An Azure storage account access key cannot be scoped to one container and cannot be made read-only. Each key grants full control of every container in the storage account, and the holder can mint SAS tokens. If the key leaks through your public s3proxy endpoint, the whole storage account is exposed. Use Entra ID instead when you can, and always set `s3proxy.read-only-blobstore=true`. Set your Azure storage account name and account access key in the properties file: ``` s3proxy.endpoint=http://0.0.0.0:8080 s3proxy.authorization=aws-v2-or-v4 s3proxy.identity=local-identity s3proxy.credential=local-credential s3proxy.read-only-blobstore=true jclouds.provider=azureblob jclouds.identity= jclouds.credential= jclouds.endpoint=https://.blob.core.windows.net ``` The two credential pairs serve different roles: * `s3proxy.identity` and `s3proxy.credential` are the S3 access key and secret that Tigris uses to authenticate to s3proxy. You set them to values you choose. You enter these same values in the shadow bucket configuration in Step 3. * `jclouds.identity` and `jclouds.credential` are your Azure storage account name and account access key. When you leave them empty, s3proxy reads Entra ID credentials from its environment instead. Set `s3proxy.read-only-blobstore=true` in every setup. Migration only reads from the source. With this property, s3proxy rejects write and delete requests with a `NotImplemented` error, and read requests continue to work. This limits the damage if your s3proxy credentials leak. note Write-through mode (Step 6) writes back to Azure through s3proxy. If you plan to use write-through, remove `s3proxy.read-only-blobstore=true`, and give the Entra ID identity the **Storage Blob Data Contributor** role instead. ## Step 2: Run s3proxy[​](#step-2-run-s3proxy "Direct link to Step 2: Run s3proxy") Start s3proxy with your properties file. ``` java -jar s3proxy --properties s3proxy.conf ``` s3proxy now listens on the `s3proxy.endpoint` address. Test it with an S3 client before you continue. caution The example endpoint uses plain HTTP on port 8080. Tigris is a hosted service. Tigris must reach s3proxy over the public internet with a valid TLS certificate. Before you configure the shadow bucket, put s3proxy behind a public HTTPS endpoint. See the s3proxy [SSL support guide](https://github.com/gaul/s3proxy/wiki/SSL-support) for TLS options. ## Step 3: Configure the shadow bucket[​](#step-3-configure-the-shadow-bucket "Direct link to Step 3: Configure the shadow bucket") Point the shadow bucket at your s3proxy endpoint. Use the `s3proxy.identity` and `s3proxy.credential` values from Step 1 as the access key and secret key. * Dashboard * CLI 1. Go to the [Tigris Dashboard](https://console.storage.dev) 2. Click **Buckets** in the left menu 3. Select the bucket you want to migrate data into 4. Click **Settings** 5. Find **Enable Data Migration** and toggle it on 6. Enter your s3proxy connection details: * **Endpoint**: The public HTTPS URL of your s3proxy instance * **Region**: `auto` (s3proxy does not require a region) * **Access Key ID**: The `s3proxy.identity` value from Step 1 * **Secret Access Key**: The `s3proxy.credential` value from Step 1 * **Bucket**: The Azure Blob container name The CLI flow is two commands: configure the shadow source, then optionally drain it. **1. Configure the shadow bucket** with [`tigris buckets set-migration`](/docs/cli/buckets/set-migration/.md): ``` tigris buckets set-migration my-bucket \ --bucket azure-container-name \ --endpoint https:// \ --region auto \ --access-key local-identity \ --secret-key local-credential ``` Add `--write-through` for write-through mode, or `--disable` to clear the migration configuration. **2. Actively migrate.** Lazy migration copies objects only when a request touches them. To copy every remaining object server-side, run [`tigris buckets migrate`](/docs/cli/buckets/migrate/.md): ``` tigris buckets migrate my-bucket ``` The command runs in the foreground and reports progress as it runs. This step is required before you disable the shadow configuration and decommission Azure Blob Storage. If you keep Azure Blob Storage as the source of truth, this step is optional. note s3proxy maps each S3 bucket name to an Azure Blob container. Set the shadow bucket **Bucket** field to the name of the Azure Blob container that holds your data. ## Step 4: Update your application[​](#step-4-update-your-application "Direct link to Step 4: Update your application") Tigris exposes an S3-compatible API, not the Azure Blob Storage API. You must change your application to use the Tigris SDK or an AWS S3 SDK. The Azure Blob Storage SDK will not work against Tigris. Set the endpoint and credentials for Tigris in your S3 client configuration. With the AWS CLI: ``` aws s3 ls s3://your-bucket/ \ --endpoint-url https://t3.storage.dev \ --region auto ``` Or with boto3: ``` import boto3 s3 = boto3.client( "s3", endpoint_url="https://t3.storage.dev", region_name="auto", aws_access_key_id="your-tigris-access-key", aws_secret_access_key="your-tigris-secret-key", ) ``` Your bucket names and object keys stay the same after migration. You must change your application source code to use the Tigris SDK or an AWS S3 SDK, not the Azure Blob Storage SDK. ## Step 5: Verify the migration[​](#step-5-verify-the-migration "Direct link to Step 5: Verify the migration") Once your application points to Tigris, objects migrate on first access. To verify: 1. Request an object that exists in your Azure Blob container 2. Verify that it returns successfully through Tigris 3. Verify that later requests are served directly from Tigris ## Step 6: Enable write-through (optional)[​](#step-6-enable-write-through-optional "Direct link to Step 6: Enable write-through (optional)") To keep your Azure Blob container in sync during the migration, enable **write-through** in the shadow bucket settings. With write-through enabled: * New objects written to Tigris are also written through s3proxy to Azure Blob Storage * Deletes apply to both Tigris and Azure Blob Storage * Object listings include the full contents of the Azure Blob container This keeps your Azure Blob container current so you can fall back at any point. Write-through needs write access through s3proxy. Remove `s3proxy.read-only-blobstore=true` from `s3proxy.conf` and restart s3proxy. If you use Entra ID, give the identity the **Storage Blob Data Contributor** role on the container. ## Step 7: Complete the migration[​](#step-7-complete-the-migration "Direct link to Step 7: Complete the migration") caution Run `tigris buckets migrate` and verify that it completes before you disable the shadow bucket configuration. The command copies all objects to Tigris server-side. Lazy migration copies an object only on first access. Objects that no request ever touches stay only in Azure Blob Storage. If you disable the shadow configuration and decommission Azure Blob Storage before the full migration completes, Tigris cannot serve those objects. The data becomes unavailable. Once your workloads run well on Tigris, disable the shadow bucket configuration. Tigris becomes your primary object store. You can then shut down s3proxy and decommission your Azure Blob Storage. ## Access Tigris from your application[​](#access-tigris-from-your-application "Direct link to Access Tigris from your application") Tigris exposes an S3-compatible API, not the Azure Blob Storage API. After you move to Tigris, your application must talk to Tigris through the Tigris SDK or an AWS S3 SDK. The Azure Blob Storage SDK will not work against Tigris. Set the endpoint to `https://t3.storage.dev` and the region to `auto`. The SDK reads your credentials from the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. The examples below configure an S3 client against the Tigris endpoint and list the objects in a bucket. * JavaScript * Python * Go ``` import { S3Client, ListObjectsV2Command } from "@aws-sdk/client-s3"; const S3 = new S3Client({ region: "auto", endpoint: "https://t3.storage.dev", s3ForcePathStyle: false, }); const response = await S3.send( new ListObjectsV2Command({ Bucket: "your-bucket" }), ); for (const object of response.Contents ?? []) { console.log(object.Key); } ``` ``` import boto3 from botocore.client import Config svc = boto3.client( "s3", endpoint_url="https://t3.storage.dev", region_name="auto", config=Config(s3={"addressing_style": "virtual"}), ) response = svc.list_objects_v2(Bucket="your-bucket") for obj in response.get("Contents", []): print(obj["Key"]) ``` ``` package main import ( "context" "fmt" "log" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/service/s3" ) func main() { ctx := context.Background() sdkConfig, err := config.LoadDefaultConfig(ctx) if err != nil { log.Fatalf("Couldn't load default configuration: %v", err) } svc := s3.NewFromConfig(sdkConfig, func(o *s3.Options) { o.BaseEndpoint = aws.String("https://t3.storage.dev") o.Region = "auto" o.UsePathStyle = false }) resp, err := svc.ListObjectsV2(ctx, &s3.ListObjectsV2Input{ Bucket: aws.String("your-bucket"), }) if err != nil { log.Fatalf("Unable to list objects: %v", err) } for _, obj := range resp.Contents { fmt.Println(aws.ToString(obj.Key)) } } ``` ## Azure-specific considerations[​](#azure-specific-considerations "Direct link to Azure-specific considerations") * **s3proxy is an extra component.** You must run, secure, and monitor it for the length of the migration. If s3proxy goes down, Tigris cannot reach objects that are not yet migrated. * **Public reachability and TLS.** Tigris must reach s3proxy over the public internet with a valid TLS certificate. Do not expose the plain HTTP example endpoint to production traffic. * **Container-to-bucket mapping.** s3proxy maps an S3 bucket name to an Azure Blob container. Use the container name as the shadow bucket **Bucket** value. * **Credential handling.** The `s3proxy.conf` file and the s3proxy environment hold your Azure credentials. Restrict access to both. Tigris never receives the Azure credentials. Tigris authenticates to s3proxy with the separate `s3proxy.identity` and `s3proxy.credential` values. * **Least privilege.** Prefer an Entra ID identity with the **Storage Blob Data Reader** role on one container. An account access key grants full control of the whole storage account and cannot be restricted. Set `s3proxy.read-only-blobstore=true` in every setup that does not use write-through. * **Authentication support.** s3proxy 4.0.0 supports account keys and Entra ID through `DefaultAzureCredential`, which includes service principals and managed identities. Releases before 4.0.0 default to an older backend that supports account keys only. * **Archived blobs block the migration.** s3proxy cannot read blobs in the Archive access tier. Rehydrate them before you migrate. See [Rehydrate archived blobs before you migrate](#rehydrate-archived-blobs-before-you-migrate). ## FAQ[​](#faq "Direct link to FAQ") ### Can Tigris read from Azure Blob Storage directly?[​](#can-tigris-read-from-azure-blob-storage-directly "Direct link to Can Tigris read from Azure Blob Storage directly?") No. Azure Blob Storage is not S3-compatible. A Tigris shadow bucket needs an S3-compatible source. You must front Azure Blob Storage with an S3 shim such as s3proxy. ### Why does `tigris buckets migrate` fail with "an internal error occurred, try again"?[​](#why-does-tigris-buckets-migrate-fail-with-an-internal-error-occurred-try-again "Direct link to why-does-tigris-buckets-migrate-fail-with-an-internal-error-occurred-try-again") A blob in the Azure Archive access tier is a common cause. s3proxy cannot read archived blobs, and the error message does not name them as the cause. List the archived blobs in the container and rehydrate them, then run the drain again. See [Rehydrate archived blobs before you migrate](#rehydrate-archived-blobs-before-you-migrate). Each retry walks the full container again, so rehydrate all archived blobs before you retry. ### Does migration require downtime?[​](#does-migration-require-downtime "Direct link to Does migration require downtime?") No. Shadow bucket migration happens transparently. Your application continues to serve requests while objects migrate on first access. ### Do I need to change my application code?[​](#do-i-need-to-change-my-application-code "Direct link to Do I need to change my application code?") Yes. You need to change your application code. Tigris is not compatible with the Azure Blob Storage API. You must change your application source code to use the Tigris SDK or S3 API instead of the Azure Blob Storage API. ### Can I roll back to Azure Blob Storage?[​](#can-i-roll-back-to-azure-blob-storage "Direct link to Can I roll back to Azure Blob Storage?") Yes. If you enable write-through mode, your Azure Blob container stays in sync with all new writes. You can revert your endpoint configuration to switch back at any point. ### What happens to objects I do not access?[​](#what-happens-to-objects-i-do-not-access "Direct link to What happens to objects I do not access?") They remain in your Azure Blob container. Tigris only copies objects when they are first requested. Objects that are never accessed are never transferred. ### Can I remove s3proxy after the migration?[​](#can-i-remove-s3proxy-after-the-migration "Direct link to Can I remove s3proxy after the migration?") Yes. s3proxy is only required while Tigris migrates from Azure Blob Storage. Once you disable the shadow bucket configuration, you can shut down s3proxy.