Cache invalidation with no invalidation code

There is an old joke that there are only two hard problems in computer science: cache invalidation, naming things, and off-by-one errors. It survives because the first item is true. Deciding when a cached value is no longer trustworthy, and making sure something acts on that decision, is where a large share of production data bugs come from.

We think the joke points at the wrong culprit. Invalidation is hard when it is done by hand, from inside the application, at every place a write might happen. Move the job to something that watches the database instead, and most of the difficulty goes with it.

Where stale data bugs actually live

After an update, the code deletes or refreshes the cached copy of the thing it changed. That line is fine on its own; the problems are everything around it.

• A second write path, such as a bulk import or a support script, does not include the line.
• The update touches a row that appears in several cached lists, and only one of them is refreshed.
• The invalidation call runs before the database has finished saving the change, so a read arriving in between refills the cache with the old value.
• A deploy adds a new field to a cached object and the old cached copies are silently missing it.

None of these are exotic. They are what happens when the same rule has to be remembered in dozens of places by people who joined at different times. The cache is only as correct as the least careful write path.

The time-to-live compromise

The usual defense is a time-to-live, or TTL: a timer that throws a cached value away after a fixed period whether or not it changed. It limits how long a bug can persist. It also guarantees that the cache serves stale data for up to that period on every change, and that hot values are repeatedly thrown away and re-fetched from the database for no reason.

Teams tune TTLs downward when they get burned and upward when the database struggles. There is no right value, because the timer is standing in for information the cache does not have: whether the row actually changed.

Let the database say what changed

That information exists. Every database engine keeps track of its own writes, and each one exposes that in a way an outside service can read. redfly's sync service uses whichever mechanism the engine provides.

1. On SQL Server, it uses change tracking, a built-in feature that notes which rows changed, and reads it on a short interval. No triggers are added and nothing reads the transaction log.
2. On PostgreSQL, it uses logical replication, which streams each saved change out of the database as it happens.
3. On MongoDB, it uses change streams, which deliver each change as it is saved.

In each case the sync service applies the change to Redis, the in-memory store that serves reads. A changed row is updated, a deleted row is removed, a new row is added. There is no invalidation step, because there is nothing to invalidate; the cache is being kept current rather than being told when it is wrong.

What disappears from the codebase

Because the sync service watches the database, it sees every write regardless of where it came from. The bulk import, the support script and the application update all land in the same place and are all picked up the same way. The forgotten-write-path class of bug is not fixed; it is removed.

There are no invalidation calls after updates, no TTL settings to tune, no warming jobs on deploy. The application calls Get, Insert, Update, Delete, GetRows or GetTotalRowCount through the redfly API, and correctness of the cache is handled behind it.

The cache is correct because something keeps it correct, not because every engineer remembered to.

What a moment of staleness means

Between a write landing in the database and Redis reflecting it, there is a gap: short on the streamed engines, and bounded by the read interval on SQL Server. A read in that gap returns the previous value. So the question every team has to answer is whether their reads can tolerate being a moment behind.

For most reads the answer is yes. A user opening a list, a permission check on a request, a product page, a dashboard tile: none of these are harmed by seeing the state from a moment ago, and most of them were already seeing older data under a TTL scheme measured in minutes.

Where it is not acceptable

A read that must reflect a write made a moment earlier by the same user is the common exception. Showing a form the user just saved, confirming a balance immediately after a transfer, or enforcing a uniqueness rule are cases where a moment of staleness is visible or unsafe. Those reads need to be identified up front, and handling them is part of the design conversation before anything is deployed.

The trade, stated plainly

Hand-written invalidation offers the possibility of zero staleness for the writes you remembered, and unbounded staleness for the ones you forgot. A sync service offers a small, predictable staleness for every write, with no code to maintain. For most systems we have seen over two decades, the second is the better deal.

redfly sits between an application and its SQL Server, MongoDB or PostgreSQL database, serving reads from a Redis cache that its sync service keeps current by watching the database for changes. Writes go straight to the database, and the codebase carries no invalidation code.