# Configuration reference TAG can be configured via a YAML configuration file and/or environment variables. Environment variables take precedence over file configuration. ## Configuration precedence[​](#configuration-precedence "Direct link to Configuration precedence") 1. Command line flags (highest priority) 2. Environment variables 3. Configuration file 4. Default values (lowest priority) ## Environment variables[​](#environment-variables "Direct link to Environment variables") | Variable | Description | Default | | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | | `AWS_ACCESS_KEY_ID` | Tigris access key (TAG's own credentials, not client credentials) | (required) | | `AWS_SECRET_ACCESS_KEY` | Tigris secret key | (required) | | `TAG_UPSTREAM_ENDPOINT` | Upstream S3 endpoint URL | `https://t3.storage.dev` | | `TAG_UPSTREAM_REGION` | Upstream region for SigV4 signing scope | `auto` | | `TAG_TRANSPARENT_PROXY` | Transparent proxy mode; set `false`/`0` for signing mode | `true` | | `TAG_MAX_IDLE_CONNS_PER_HOST` | HTTP connection pool size per upstream host | `100` | | `TAG_MAX_INFLIGHT_REQUESTS` | Max concurrent S3 requests before shedding with 503 SlowDown | `1024` | | `TAG_CACHE_DISK_PATH` | Path to cache data directory | `/var/tmp/tag` | | `TAG_CACHE_MAX_DISK_USAGE` | Max disk usage in bytes (0 = unlimited) | `0` | | `TAG_CACHE_EVICTION_POLICY` | Eviction order when the disk cap is hit: `lru` or `fifo` (oldest-written first) | `lru` | | `TAG_CACHE_COMPACTION_BPS` | Read budget in bytes/second for background compaction of the cache's on-disk storage. `0` leaves it unthrottled | `0` | | `TAG_CACHE_TTL` | Default TTL for cached objects (Go duration, e.g. `12h`, `30m`) | `24h` | | `TAG_CACHE_DISABLED` | Disable caching (`true` or `1`) | `false` | | `TAG_CACHE_WARM_ON_WRITE` | Warm the cache after a successful write via a background fetch (`true`/`false`) | `false` | | `TAG_CACHE_MAX_CONCURRENT_WRITES` | Max concurrent cache-populate operations | `256` | | `TAG_CACHE_MAX_POPULATE_MEMORY` | Aggregate memory budget (bytes) for all cache buffering — populate + block-serve staging (one honest total) | `2147483648` (2 GiB) | | `TAG_CACHE_BLOCK_CACHING_ENABLED` | Cache large objects as fixed-size blocks (RFC 0001) so a range read fetches only the covering blocks (`true`/`false`) | `true` | | `TAG_CACHE_BLOCK_SIZE` | Block granularity and the whole-vs-block boundary (bytes); size to your workload's read granularity | `1048576` (1 MiB) | | `TAG_CACHE_META_ON_WRITE` | Cache an object's metadata when TAG proxies its write, so the first read does not spend a round trip discovering it. Requires block caching | `false` | | `TAG_CACHE_PARQUET_OPTIMIZATION` | Cache a parquet object's metadata blocks ahead of the reader, on read and on write. Requires block caching. See [Parquet optimization](/docs/acceleration-gateway/parquet-optimization/.md) | `false` | | `TAG_CACHE_WARM_ON_WRITE_RESERVED_FRACTION` | Fraction of the populate memory budget reserved for warm-on-write so it is not starved by read-miss warms (only when `warm_on_write` is on) | `0.5` | | `TAG_CACHE_DELETE_BATCH_SIZE` | File deletions processed per deletion-queue batch | `1000` | | `TAG_CACHE_RECOVERY_WORKERS` | Parallel workers for startup file recovery | `16` | | `TAG_HTTP_PORT` | HTTP listen port | `8080` | | `TAG_LOG_LEVEL` | Log level: `debug`, `info`, `warn`, `error` | `info` | | `TAG_LOG_FORMAT` | Log format: `json` or `console` | `json` | | `TAG_TLS_CERT_FILE` | Path to TLS certificate file (PEM format) | (none) | | `TAG_TLS_KEY_FILE` | Path to TLS private key file (PEM format) | (none) | | `TAG_CACHE_GRPC_ADDR` | Address for gRPC server | `:9000` | | `TAG_CACHE_GRPC_AUTH` | Authenticate cross-node gRPC cache traffic. Fail-closed: only `false` or `0` disables it, so a typo leaves auth on | `true` | | `TAG_AUTHZ_CACHE_TTL` | How long a per-bucket authorization decision is cached (Go duration) | `10m` | | `TAG_CACHE_NODE_ID` | Unique node identifier for cluster mode (clustering) | (none) | | `TAG_CACHE_CLUSTER_ADDR` | Address for memberlist gossip (clustering) | `:7000` | | `TAG_CACHE_ADVERTISE_ADDR` | Address advertised to other nodes (clustering) | (defaults to gRPC addr) | | `TAG_CACHE_SEED_NODES` | Comma-separated seed nodes for cluster discovery (clustering) | (none) | | `TAG_PPROF_ENABLED` | Enable pprof endpoints (`true` or `1`) | `false` | `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` are TAG's own Tigris credentials with read-only access to all buckets accessed through TAG (required). Clients use their own credentials directly. ## Configuration file[​](#configuration-file "Direct link to Configuration file") The configuration file uses YAML format. Specify the path with the `--config` flag: ``` ./tag --config /etc/tag/config.yaml ``` ### Full configuration reference[​](#full-configuration-reference "Direct link to Full configuration reference") ``` # Server configuration server: # HTTP port for the S3 API # Default: 8080 http_port: 8080 # IP address to bind to # Default: "0.0.0.0" (all interfaces) bind_ip: "0.0.0.0" # Enable pprof profiling endpoints # Default: false (disabled for security) pprof_enabled: false # Path to TLS certificate file (PEM format) # When both tls_cert_file and tls_key_file are set, TAG serves HTTPS # Default: "" (TLS disabled, serves HTTP) tls_cert_file: "" # Path to TLS private key file (PEM format) # Must be set together with tls_cert_file # Default: "" (TLS disabled, serves HTTP) tls_key_file: "" # Upstream Tigris configuration upstream: # Tigris S3 endpoint URL # Default: "https://t3.storage.dev" endpoint: "https://t3.storage.dev" # AWS region for request signing # Default: "auto" region: "auto" # HTTP connection pool size per upstream host # Higher values improve throughput for cache-miss scenarios # Default: 100 max_idle_conns_per_host: 100 # Transparent proxy mode forwards client requests as-is with proxy headers # (Tigris only). Set to false for signing mode, which validates and re-signs # requests and works with any S3-compatible endpoint. # Default: true transparent_proxy: true # Cache configuration cache: # Enable caching # Default: true enabled: true # Default TTL for cached objects # Default: 24h ttl: 24h # Maximum object size to cache (in bytes) # Objects larger than this are not cached # Default: 1073741824 (1GB) size_threshold: 1073741824 # Path to cache data directory # /var/tmp/tag works on both macOS and Linux without root # Default: /var/tmp/tag disk_path: "/var/tmp/tag" # Max disk usage in bytes (0 = unlimited) # Default: 0 max_disk_usage_bytes: 0 # Eviction order when the disk cap is reached: "lru" (default) or "fifo". # "fifo" evicts oldest-written objects first — better for write-once workloads # (e.g. dated parquet) where a rare read of an old object should not keep it # resident at the expense of newer, hotter data. # NOTE: eviction only runs when max_disk_usage_bytes > 0. With no disk cap the # cache is never evicted and this setting has no effect. # Default: lru eviction_policy: lru # Read budget in bytes/second for background compaction, which consolidates # cached objects into larger files and reclaims the space left by objects that # were overwritten, deleted, or expired. Unthrottled compaction can saturate a # throughput-capped cloud volume and stall reads. Set this to a small fraction # of the volume's throughput limit, for example 33554432 (32 MiB/s) on a volume # capped at 240 MB/s. Populate writes are never throttled. # Default: 0 (unthrottled) compaction_bytes_per_second: 0 # Warm the cache after a successful write (PutObject / CompleteMultipartUpload / # CopyObject) by triggering a background full-object fetch, so a read soon after a # write hits cache. This is cache-warm-on-write (write-around plus async warming), # NOT strict write-through: the write still invalidates, and the warm is a # separate, best-effort background fetch — deduplicated and shed under the cache # populate budget. It costs one extra upstream GET per write, so it defaults off. # An anonymous write warms with an unsigned fetch, so the object is cached as # public-read only if it is genuinely publicly readable. # Default: false warm_on_write: false # Max concurrent cache-populate operations (upstream fetch + streaming write). # When saturated, objects are still served from upstream, just not cached. # Default: 256 (0 or unset = default; negative = disabled) max_concurrent_writes: 256 # Aggregate memory budget for concurrent cache-populate buffering. Each populate # reserves its object size, capped at the per-populate buffer ceiling, so many # small objects populate concurrently while a burst of large objects is throttled # to keep buffered memory bounded. This, rather than the raw count above, is what # bounds populate memory; both limits apply. # Default: 2147483648 (2 GiB) (0 or unset = default; negative = disabled). One honest # total: block-serve staging draws from this same budget, capped at half of it. max_populate_memory_bytes: 2147483648 # Block-aligned caching for large objects (RFC 0001). On by default: any object at or # above block_size is cached as fixed-size blocks, so a range read populates and serves # only the blocks it touches. Set false to cache whole objects instead. block_caching_enabled: true # Block granularity AND the whole-vs-block boundary. Size it to your workload's read # granularity — an oversized block over-fetches on every miss. 0/unset = default (1 MiB). block_size: 1048576 # Unique node identifier for cluster mode # Required for multi-node deployments node_id: "tag-node-1" # Address for memberlist gossip protocol # Default: :7000 cluster_addr: ":7000" # Address for gRPC server (cache cluster routing) # Default: :9000 grpc_addr: ":9000" # Address advertised to other nodes # Defaults to grpc_addr if not specified advertise_addr: "tag-node-1:9000" # Seed nodes for cluster discovery # List of cluster addresses for other nodes # or can also be DNS name of headless service in Kubernetes (e.g. tag-headless-svc:7000) seed_nodes: - "tag-node-1:7000" - "tag-node-2:7000" - "tag-node-3:7000" # Broadcast configuration (request coalescing) broadcast: # Streaming chunk size in bytes # Default: 65536 (64 KiB) chunk_size: 65536 # Buffer size per listener in chunks # Total buffer per listener = chunk_size × channel_buffer # Default: 32 (~2 MiB with default chunk size) channel_buffer: 32 # Logging configuration log: # Log level: debug, info, warn, error # Default: "info" level: "info" # Log format: json (fast) or console (human-readable) # Default: "json" format: "json" ``` ## Additional notes[​](#additional-notes "Direct link to Additional notes") ### TLS[​](#tls "Direct link to TLS") When both `tls_cert_file` and `tls_key_file` are set, TAG serves HTTPS. See [TLS/HTTPS](/docs/acceleration-gateway/tls/.md) for certificate setup across Docker, Kubernetes, and native deployments. ### Endpoint validation[​](#endpoint-validation "Direct link to Endpoint validation") The upstream endpoint must be a well-formed absolute `http://` or `https://` URL with a host. In **transparent proxy mode** (the default) the host must also match one of `localhost`, `*.tigris.dev`, or `*.storage.dev`; TAG exits at startup otherwise. **Signing mode** does not apply this allowlist and accepts any S3-compatible endpoint — see [Using TAG with other S3-compatible services](#using-tag-with-other-s3-compatible-services). ### Using TAG with other S3-compatible services[​](#using-tag-with-other-s3-compatible-services "Direct link to Using TAG with other S3-compatible services") Signing mode re-signs requests with standard AWS SigV4, so it works against any S3-compatible endpoint (AWS S3, MinIO, Ceph, etc.), not just Tigris: ``` upstream: transparent_proxy: false endpoint: "https://s3.us-east-1.amazonaws.com" # Set the backend's region; the default "auto" only works with Tigris. AWS and # other region-sensitive services reject signatures whose credential-scope # region does not match the endpoint. region: "us-east-1" ``` * Transparent proxy mode and its zero-config, no-double-auth experience are Tigris-only. Third-party backends are supported on a best-effort, community-supported basis. * The `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` credentials are the credentials for that backend; clients must present the same credentials, and they need permissions covering whatever operations clients perform (read-only is insufficient if clients write). ### Cluster mode[​](#cluster-mode "Direct link to Cluster mode") For multi-node deployments, configure each node with a unique `node_id`, the same `seed_nodes` list, and an `advertise_addr` reachable from other nodes. | Port | Protocol | Purpose | | ---- | -------- | --------------------------------------- | | 8080 | TCP | HTTP API (S3-compatible) | | 7000 | TCP | Gossip protocol for cluster discovery | | 9000 | TCP | gRPC for inter-node cache communication | macOS port conflict On macOS, port 7000 is used by AirPlay Receiver. Use ports 17000 (gossip) and 19000 (gRPC) instead: ``` cache: cluster_addr: ":17000" grpc_addr: ":19000" seed_nodes: - "node1:17000" ``` ## Profiling[​](#profiling "Direct link to Profiling") TAG exposes pprof endpoints for performance profiling when enabled. Disabled by default for security (exposes runtime internals). ``` TAG_PPROF_ENABLED=true ./tag ``` Endpoints (when enabled): * `/debug/pprof/` — Index * `/debug/pprof/profile?seconds=30` — CPU profile * `/debug/pprof/heap` — Heap profile * `/debug/pprof/goroutine` — Goroutine stacks Usage with `go tool pprof`: ``` go tool pprof http://localhost:8080/debug/pprof/profile?seconds=30 go tool pprof http://localhost:8080/debug/pprof/heap ``` ## Command line flags[​](#command-line-flags "Direct link to Command line flags") | Flag | Description | | ----------------- | ------------------------------------------------------ | | `--version` | Print version information and exit | | `--config` | Path to configuration file | | `--http-port` | HTTP listen port (default: 8080, env: `TAG_HTTP_PORT`) | | `--log-level` | Log level (overrides config file and env) | | `--log-format` | Log format (overrides config file and env) | | `--disable-cache` | Disable caching (pass-through mode) | ``` # Print version ./tag --version # Use configuration file ./tag --config /etc/tag/config.yaml # Override port and log level via flags ./tag --http-port 9090 --log-level debug # Disable caching via flag (overrides config) ./tag --config /etc/tag/config.yaml --disable-cache # Use environment variables only (no config file) AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=yyy ./tag ``` ## Example configurations[​](#example-configurations "Direct link to Example configurations") ### Development (standalone)[​](#development-standalone "Direct link to Development (standalone)") ``` server: http_port: 8080 upstream: endpoint: "https://t3.storage.dev" cache: disk_path: "/tmp/tag-cache" node_id: "dev-node" log: level: "debug" ``` ### Production (single node)[​](#production-single-node "Direct link to Production (single node)") ``` server: http_port: 8080 bind_ip: "0.0.0.0" cache: disk_path: "/var/tmp/tag" max_disk_usage_bytes: 429496729600 # 400 GiB ttl: 24h size_threshold: 1073741824 node_id: "tag-prod" log: level: "info" format: "json" ``` To add TLS to any of these configs, set `tls_cert_file` and `tls_key_file` under `server`. See [TLS/HTTPS](/docs/acceleration-gateway/tls/.md) for full examples. ### Production (cluster mode)[​](#production-cluster-mode "Direct link to Production (cluster mode)") Configure each node with a unique `node_id` and the same `seed_nodes` list: ``` server: http_port: 8080 cache: disk_path: "/var/tmp/tag" max_disk_usage_bytes: 429496729600 # 400 GiB per node ttl: 24h size_threshold: 1073741824 # Cluster configuration — unique per node node_id: "tag-1" cluster_addr: ":7000" grpc_addr: ":9000" advertise_addr: "tag-1.tag-svc.default.svc.cluster.local:9000" seed_nodes: - "tag-svc.default.svc.cluster.local:7000" log: level: "info" format: "json" ```