May 18, 2026
Building a cache that doesn't lie to you
There are only two hard things in computer science, the joke goes, and one of them is cache invalidation. After a few years of getting this wrong in production, I've collected a small set of rules that keep me out of trouble.
1. Decide what "stale" means before you cache anything
Every cached value has a tolerance for staleness. A user's avatar can be minutes out of date; an account balance cannot. Write that number down for each thing you cache. If you can't name it, you're not ready to cache it yet.
2. Add jitter to your TTLs
If a thousand keys all expire at the same second, you get a thundering herd hammering the origin. Spreading expirations over a small random window turns a spike into a gentle hum.
ttl = base_ttl + random_between(0, base_ttl * 0.1)3. Prefer stale-while-revalidate
Serving a slightly stale value while you refresh it in the background is almost always better than making the user wait. The cache stays fast, and the origin only sees one refresh request instead of many.
4. Make failures loud
A cache that silently falls back to the origin on every request isn't a cache — it's a liability with extra latency. Track your hit rate and alert when it drops. The first sign of trouble is usually a quiet collapse in hit ratio long before anything pages you.
None of this is novel, but writing it down has saved me from repeating the same mistakes. Future me, you're welcome.