Tigris is designed with a reliable and scalable architecture at its core, allowing for building a globally distributed system that can be readily scaled to accommodate evolving needs.
One of the key design decisions is to have a composable architecture that allows building a complex distributed system by combining smaller, independent building blocks or services. Each component within Tigris is designed to serve a specific function and can be scaled independently.
A Tigris object store deployment consists of API gateways, a cache layer, a data distribution and replication framework, and data and metadata storage services.
The subsequent section describes each of the major components of Tigris in more detail.
API Gateway layer is the first layer of interaction between users' applications and the Tigris object storage service. The gateway layer conforms to S3 APIs, understands the semantics of the request, and is responsible for authentication, authorization, request processing, and routing.
The API gateway is deployed across multiple regions as stateless compute workers and handles the requests close to the user.
Tigris transparently caches the data close to the user to provide low-latency access. Caching is provided through a distributed global caching layer with cache nodes deployed in all regions where gateways are deployed. This ensures that user requests can be served from the region closest to the user.
The figure above shows a cache deployment in one of the regions (US-WEST). Similar deployments exist in all the regions.
Tigris supports two caching strategies:
- Cache on Read (default) Depending on the access pattern of objects, the objects get cached.
- Cache on Write (configurable) This is eager caching, where the cache is populated when the object is written. Cache-on-Write can be configured on a per-bucket basis. We have found Cache-on-Read to be sufficient for most of the use cases and the most cost-effective, but Cache-on-Write is available for use cases that need it.
We have designed the object storage service such that metadata storage is a separate layer that is deployed separately from object storage. We have also designed the metadata storage to be transactional so that we can provide strong consistency guarantees and powerful semantics such as Compare-And-Set, Transactions over objects, and rich querying functionality, none of which is provided by S3.
Metadata includes metadata about the objects (such as object location, user-supplied metadata, etc), buckets information, users and organization information, access policies, and permissions.