Skip to main content

· 5 min read
Garren

Autumn trees on a dusty road in Magoebaskloof, South Africa

Autumn trees on a dusty road in Magoebaskloof, South Africa. Photo by Garren Smith, iPhone 13 Pro.

Tigris now supports object notifications! Object notifications are how you receive events every time something changes in a bucket. Think of it as your bucket's way of saying "Hey, something happened! Come check it out!", much like the inotify subsystem in Linux. These notifications can be helpful for keeping track of what's going on in your application.

Use Case: Automatic Image Processing

Imagine you're building a photo-sharing app. Every time a user uploads a new picture, you want to automatically generate a thumbnail and maybe even run it through an AI to detect any inappropriate content. With object notifications, this becomes a breeze!

  1. User uploads an image to your Tigris bucket.
  2. Tigris sends a notification to your webhook.
  3. Your server receives the notification and springs into action.
  4. It downloads the new image, creates a thumbnail, and runs it through an AI check.
  5. The processed image and its metadata are saved back to Tigris.

All of this happens automatically, triggered by that initial upload.

Behind the Scenes: Building Object Notifications

Now, let's pull back the curtain and see how we built this feature and a few tricky situations we had to handle. Grab your hard hat, because we're going on a little tour of Tigris's inner workings!

Tigris isn't just any object store – it's a global object store. This means that objects can be changed in multiple regions around the world. This makes them available in multiple regions, always ready when you need them. But means we need a way of keeping track of all the changes for the same object. This is where replication comes in.

Replication: Keeping Everyone in the Loop

To make sure everything stays in sync, we replicate changes to multiple regions. This ensures high availability and improved redundancy of our objects.

The caveat to this is that replication is a background task, and the speed at which an object is replicated from one region to another can be affected by many external factors.

To solve this, when a change is received at a region it looks at the Last Modified timestamp of the metadata to determine if the change is new and needs to be applied or if the region has already seen a newer change. It will discard the change if it is old.

The Object Notification Hub

When object notifications are enabled for a bucket, we assign one region to be the object notification hub for that bucket. This region gets the important job of keeping track of all the changes. We create a special index which is very similar to a secondary index in that region's FoundationDB. We order the changes by FoundationDB Versionstamp, when the change is added to the index, and Last Modified timestamp of object metadata.

The Versionstamp helps the worker keep track of which events it has seen and processed.

Why one region you may ask? If we didn't do this, we end up with multiple regions sending the same events to the webhook, hello friendly DDos attack, or having to build a complex system to try and co-ordinate the regions so they don't send duplicate events.

The Background Task: Our Diligent Messenger

In our object notification region, we have a background task running. Think of it as a tireless worker that's always on the lookout for changes. Every so often, it checks the special index we mentioned earlier, collects all the latest changes, and sends them off to the webhook.

The worker will also keep track of the last processed change and will retry a few times if the request failed. Finally it will remove old changes from the index that have already been processed.

Why We Can't Guarantee Ordered Events

We talked about how object changes replicated from many regions can take different times. The problem arises when the worker is ready to send the latest events for an object. It has no way of knowing if all changes for an object have been replicated to its region. It could in theory contact every region and check, but this would be prohibitively expensive. And still not a complete guarantee.

This forces us to make the trade off of sending events out of order. The worker will read the latest list of changes that have been replicated to the region and send them to the webhook.

Wrapping Up

That's how we built object notifications in Tigris. We took a global system, added some global replication, threw in a change index, topped it off with a hardworking background task.

The result? A system that keeps you in the loop about what's happening in your buckets, no matter where in the world those changes occur. Whether you're building the next big photo-sharing app or just want to keep tabs on your storage, object notifications have got your back!

We hope this peek behind the scenes was fun and informative. Happy coding!

· 5 min read
Xe Iaso

Docker is the universal package format of the internet. When you deploy software to your computers, chances are you build your app into a container image and deploy it through either Docker or something that understands the same formats that Docker uses. However, this is where they get you: Docker image storage in the cloud is not free. Docker registries also have strict image size limits and will charge you egress fees based on the size of your images.

What if you could host your own registry though? What if when doing it you could actually get a better experience than you get with the hosted registries on the big cloud.

A sea of scattered clouds covers the land beneath.

A sea of scattered clouds covers the land beneath. Photo by Xe Iaso, iPhone 15 Pro Max @ 22mm.

· 4 min read
Katie Schilling
Xe Iaso

At Tigris Data, we provide object storage to our users. People put bytes into our servers with a name, and expect that come hell and high water, when they put in the name, they get the exact same bytes back. This is a very high trust position to be in because when people ask themselves things like “Oh, what would happen if my object storage provider is unreliable”, that conversation usually involves phrases like “Maybe we should have gone with The Big Cloud afterall”.

Such conversations are rarely good for the business.

A battle rages on in the field, yet the strong oak tree remains unscathed

A battle rages on in the field, yet the strong oak tree remains unscathed

· 6 min read
Lars Wikman

Introduction

I admit it. My first Tigris blog post about Eager and Lazy caching was kind of basic. It was important to cover the ground-work. The CDN aspect is important and I do like the summon-your-data pre-fetch header a lot. Now we get to the significantly more disruptive stuff. The things that while Tigris is an S3-compatible API it also provides features that enable entirely new use-cases and push the boundaries of what you can do with object storage. Let's see if we can't set your internal constraint-solver aflame with possibilities.

· 6 min read
Jesse Thompson

If you've been toying around in the AI space over the past few months, you've probably heard of Ollama. Ollama is a tool for running various LLMs locally on your own hardware, and currently supports a bunch of open models from Google, Facebook and independent sources.

Besides the basic terminal chat function, Ollama has an API for use from within your favourite programming languages. This means you can build your very own LLM-powered apps!

Let's say we've built the next killer LLM app: ChatWich (which allows you to chat with your sandwich) and people are loving it when you show it off on your laptop, but personally visiting all your customers with your computer in hand is getting tiring, and the travel bills are starting to outweigh the (awesome) frequent flyer miles you're getting.

It's time to move to the cloud.

· 4 min read
Ovais Tariq

Since we launched our public beta three months ago, our usage has skyrocketed, and hundreds of early adopters have picked Tigris as their storage solution. We've implemented tons of requested features and invested heavily in Tigris' performance, security, and reliability. We're grateful for your feedback and confident that we are on track to make the most developer-friendly object storage service.

Tigris globally distributed object
storage [Credits: Xe Iaso - https://xeiaso.net/]

With that, we will start billing for Tigris usage in July because we're confident that Tigris is reliable enough for us to justify doing that. Check out our pricing page for details on the pricing structure. The beta tag will stay, but we'll offer the same support expected from a highly reliable production-ready platform. Check out our SLA page for details about our uptime commitment.