User-uploaded images are the first feature that quietly turns into an infrastructure bill. Text is cheap: a document read here, a few kilobytes there. Images are megabytes, they get requested constantly, and every one of those requests either hits a cache (cheap) or hits your origin (not cheap). PlayersEngine is a social platform, so avatars and uploaded media show up on nearly every screen. Here's how I keep that from becoming the biggest line on the invoice.
The three-tier reality of image cost
There are only three places an image request can land, and they have wildly different prices:
- The browser cache. Free. The request never leaves the device.
- The CDN edge. Cheap. You pay for edge egress, but nothing touches your infrastructure.
- Origin storage. Expensive in aggregate. You pay storage-class egress, per-operation fees, and — if a service is proxying the bytes — compute time too.
The entire game is pushing as many requests as possible up that list. A platform designed for a million users cannot treat cache behavior as a tuning detail you get to later; the caching strategy is the cost model — in the same way that scale-to-zero is a budget strategy rather than an infrastructure preference.
Immutable caching without a build pipeline
The gold-standard trick is content fingerprinting: hash the file, put the hash in the filename, cache forever. Frontend bundlers do this for free. But user-uploaded media doesn't go through a bundler, and as a solo founder I wasn't about to build a fingerprinting pipeline — hashing on upload, tracking hashes in the database, garbage-collecting the old ones — just to get cache headers right.
So I use the boring version: serve media with
Cache-Control: public, max-age=31536000, immutable, and
treat the URL as the cache key. When an asset actually changes at
the same logical path, the reference to it changes — a
?v= version parameter bumped on the tag or the stored
record. The old URL stays validly cached until nobody references it;
the new URL is a cold key that fills the cache once.
The catch — and I learned this on my own landing page before it ever bit me in the app — is that immutable caching is a contract. If you ever replace bytes in place without changing the URL, some users will see the stale version for up to a year and there is nothing you can do about it remotely. So the rule is mechanical: any in-place asset swap requires a version bump on every referencing tag, and that rule lives in the deploy checklist, not in my memory.
Store filenames, not URLs
The subtler decision was about what goes in the database. The tempting move is to denormalize the full image URL everywhere it's displayed — onto posts, comments, notifications — so rendering never needs a join. I did some of that early, and unwinding it taught me the better rule: store the filename, build the URL at the service edge.
The reason is that a full URL bakes three volatile things into permanent records: the domain, the path scheme, and the version parameter. The day you switch CDN providers, move buckets, or restructure paths, every denormalized URL in the database is wrong, and you're writing a mass-migration script over millions of documents to fix a string prefix. That's the definition of a self-inflicted outage risk.
A filename, by contrast, is stable forever. Each service that
returns image references passes the filename through one small
buildUrl() helper at the response boundary. Changing
CDNs becomes a one-line config change and a deploy — not a data
migration. The cost is a string concatenation per response, which
is free at any scale I will ever care about. This is one of those
asymmetries worth memorizing:
denormalize data when reads dominate, but never denormalize infrastructure decisions into
data.
Derivatives: resize once, serve forever
The other budget killer is serving originals. A 4 MB upload displayed as a 90-pixel avatar is paying roughly 40× more egress than the pixels justify, on every single view. The fix is standard — generate sized derivatives — but the placement of the work matters on a small budget.
I resize once, at write time, in the async worker path rather than on-demand at read time. Upload events flow through Pub/Sub to a worker anyway — that's the platform's general pattern for side-effects — so image processing is just another subscriber concern. On-demand resizing services are elegant, but they turn every cache miss into compute, and they're one misconfigured cache header away from resizing the same image thousands of times. Write-time processing has a fixed, predictable cost: uploads are rare, views are common, so you want the work attached to the rare thing.
A few sizes cover everything; more variants means more storage and more cold cache keys for marginal quality gains. Storage is the cheap dimension here — pennies per gigabyte-month — while egress and operations are the expensive ones. Trading a little storage for a much higher cache hit ratio is nearly always the right direction.
The math that makes it all work
Roughly speaking, origin egress costs on the order of ten times what you'd like it to, and CDN edge delivery is significantly cheaper per gigabyte — but the real lever is the hit ratio. At a 95% cache hit ratio, only one request in twenty touches origin. Push that to 99% and origin traffic drops another 5×. Every point of hit ratio is a direct discount on the bill.
That's why the earlier decisions compound: immutable headers mean edge nodes never revalidate; stable URLs mean the cache never gets needlessly invalidated; a small set of derivative sizes means the same few keys stay hot instead of a long tail of cold variants. None of these techniques is clever on its own. Together they mean that when an image gets popular, the marginal cost of the next ten thousand views rounds to zero.
The takeaway
If you're building anything with user media on a budget: set
immutable headers and treat URL changes as the only
invalidation mechanism; keep infrastructure out of your database by
storing filenames and building URLs at the edge; resize at write
time because uploads are rare and views are not; and watch your
cache hit ratio the way you'd watch an error rate. The cheapest
byte is the one your origin never serves.