In-process cache, shared cache or CDN?
| Dimension | In-process caches | Caching | CDNs |
|---|---|---|---|
| Where it lives | Inside each service instance’s own memory | Separate cache servers on your network (Valkey, Redis, Memcached) | Edge servers near users, in front of your origin |
| Who shares it | One process only; ten pods means ten separate caches | Every instance of every service that connects to it | Every user routed to that edge location |
| Cost of a hit | A memory lookup: nanoseconds to microseconds, no network | Sub-millisecond on the server, plus a network round trip | A round trip to the nearest edge instead of to your origin |
| What it stores | Live objects in your language’s own types | Serialised values under keys | Whole HTTP responses: pages, images, scripts, API replies |
| How entries go stale | TTL and size limit per instance | TTL per key (SET … EX), or delete the key when the source changes | Cache-Control max-age or s-maxage from the origin, or a default TTL (24 hours on CloudFront without a cache policy) |
| Invalidating | Hard: each instance must be told, or waits out its TTL | Easy: one place to delete or overwrite the key | A purge or invalidation API; CloudFront recommends versioned file names for frequent updates |
| Capacity | Counts against each pod’s memory limit; too big and the pod is OOMKilled | The cluster’s RAM; add nodes or shards to grow | Large and managed by the provider; rarely requested files are evicted |
| After a deploy or restart | Empty; every new instance warms up from scratch | Still warm; it runs apart from the app | Still warm; deploys of the origin don’t touch it |
| Examples | Caffeine, Guava, lru-cache, functools.lru_cache, .NET IMemoryCache | Valkey, Redis, Memcached; ElastiCache, Memorystore, Azure Managed Redis | CloudFront, Cloud CDN, Azure Front Door, Cloudflare |
Rule of thumb
They stack rather than compete. A typical read path is CDN for public HTTP responses, then an in-process cache for the few hottest objects, then a shared key-value cache that all instances agree on, then the database. Put data in the layer whose staleness you can live with: an in-process copy can be out of date per instance for its whole TTL, a shared cache can be fixed with one delete, and a CDN copy lasts until its max-age or a purge.
Sources
- Caffeine on GitHub
- isaacs/node-lru-cache on GitHub
- Python docs: functools.lru_cache
- Guava wiki: Caches explained
- Microsoft Learn: Cache in-memory in ASP.NET Core (cache stored in the web server’s memory)
- Microsoft Learn: HybridCache library in ASP.NET Core (invalidation doesn’t reach other servers’ memory)
- Redis docs: Client-side caching reference (local memory is orders of magnitude faster than a networked service)
- Kubernetes docs: Assign memory resources to containers (OOMKilled)
- AWS: What is caching? (sub-millisecond reads from in-memory caches)
- Valkey docs: SET (EX seconds)
- Nishtala et al.: Scaling Memcache at Facebook (NSDI 2013), delete on write
- ElastiCache User Guide: What is Amazon ElastiCache?
- Google Cloud: Memorystore for Valkey overview
- Microsoft Learn: What is Azure Managed Redis?
- RFC 9111: HTTP Caching (max-age, s-maxage, shared caches)
- CloudFront docs: How CloudFront delivers content (nearest edge location)
- CloudFront docs: Manage how long content stays in the cache (24-hour default TTL; eviction)
- CloudFront docs: Invalidate files (versioned file names recommended)
- Google Cloud docs: Cloud CDN overview
- Microsoft Learn: What is Azure Front Door?
- Cloudflare docs: Purge cache