Automatic database fallback: reads that survive a cache outage

Most conversations about caching are about the good days. What is the hit rate, how fast is a read, how much load came off the database. Those numbers matter, but they describe the system when everything is working. The more revealing question is what the application does on the day Redis is not there.

Redis, the in-memory store that serves cached reads, is a separate piece of infrastructure with its own failure modes. It can be restarted, run out of memory, lose a network path, or be taken down for maintenance by someone who did not know what depended on it. If the application has been written to expect it, a missing cache becomes a missing application.

How a cache outage usually plays out

In a hand-written cache layer, the read path typically looks like this: check Redis, and on a miss, query the database and store the result. That handles a missing key. It does not necessarily handle a missing Redis.

When the cache client cannot connect, one of a few things happens. The call throws and the request fails outright. Or the client waits for a timeout on every read, pages that took milliseconds now take seconds, and soon every worker that handles requests is stuck waiting, so the whole service stalls. Or, in the well-prepared case, a developer wrote a fallback, but it was written two years ago, has never run in production, and nobody is sure it still works.

The pattern is familiar because caches are usually added for speed, and speed is what gets tested. Availability is assumed.

What automatic fallback means

With redfly, the application does not talk to Redis at all. It calls the redfly API, and the API decides where a read is served from. On a normal day that is Redis. If Redis is unreachable, the API serves the same read from the database instead.

Nothing in the application changes for this to happen. There is no degraded-mode switch to flip, no configuration setting to change, no code path the customer's engineers wrote and hope still works. All the application notices is that responses are slower, because a read from disk takes longer than a read from memory. Requests that would have succeeded still succeed.

Read availability should track the database, not the cache.

Why this matters more than hit rate

Cache hit rate, the share of reads answered from memory, is the number most teams optimize. It is a real measure of how much load the cache is absorbing. But a system with a very high hit rate and no fallback has traded a database problem for a Redis problem: it is fast until the cache is gone, and then it is down.

Fallback changes the shape of the risk. Instead of two components that must both be up, there is one component that must be up, the database, and a second that makes things faster when it is present. That is a much easier system to reason about and to operate at three in the morning.

It also makes the cache safer to touch. Upgrading Redis, resizing it, or moving it to another host becomes far less of an event, because the application keeps answering reads throughout.

Can the database carry the load?

The honest question that follows is whether the database can cope with all reads landing on it at once. The answer is not always yes.

The database was probably sized with the cache in mind. During fallback it sees the full read load. Whether it copes depends on a few things.

• How long the outage lasts. A restart measured in seconds is very different from a lost host that takes an hour to replace.
• When it happens. A cache outage during a quiet period is a non-event; the same outage during a peak is not.
• How much headroom the database has. If the cache allowed the database to be sized smaller, some of that headroom has been spent.

Our view is that fallback should be treated as a short-lived condition, not a steady state. It exists so that a brief Redis interruption is a slowdown rather than an outage. Deliberately running on fallback for a long period, at peak, on a database that was shrunk to match the cache, is a decision to make with eyes open.

Thinking about it in practice

The useful exercise is to ask two questions before going live. First, what does the database look like under the full read load for a few minutes, and does it stay responsive. Second, how quickly can Redis be restored, and is that time shorter than the database can comfortably hold out. If both answers are reassuring, fallback does its job. If not, that is a sizing conversation, and it is better to have it before the outage than during it.

What it does not promise

Fallback is not a guarantee that nothing ever goes wrong. If the database itself is down, reads are down, because the database is the source of truth and the cache is not a substitute for it. Fallback does not make a slow database fast, and it does not make an under-sized database large.

What it does is remove one entire category of failure, the cache-down outage, from the list of things that can take an application offline.

redfly sits between an application and its SQL Server, MongoDB or PostgreSQL database, serving reads from a Redis cache it keeps in sync and passing writes straight through. If Redis is unreachable, reads go to the database automatically.