Most developers learn system design in the wrong way, especially with the help of AI.

They usually study concepts such as load balancing, the CAP theorem, Kafka and Saga. Something AI can explain.

But knowing the components is not the same as designing a system.

Every real system is the result of a handful of decisions and trade-offs, and each one costs you something.

For example:

• Pick strong consistency, and you pay in latency.
• Pick a queue, and you pay in debugging across multiple services.
• Pick microservices, and you pay in premium operations.

Over the past years, I have built and scaled .NET systems that made every one of these choices, sometimes badly. This is the short version of what I wish someone had handed me at the start.

In this post, we will explore the 7 decisions every system has to make:

• How fresh does this data have to be?
• Where should this data live?
• Do you buy a bigger machine or more machines?
• What do you cache, and where?
• Should this be a call or an event?
• What happens when more work arrives than you can handle?
• How many things do you deploy?

Let's dive in.

Most teams answer this once, for the whole system, and then live with the answer for years. That is the mistake. Freshness is a per-data-type decision.

A payment balance and a product review count do not need the same guarantee. Treating them the same means you either overpay for the reviews or underprotect the money.

These are the models you are actually choosing between:

Now the part most articles get wrong. The CAP theorem says a distributed system can only guarantee two of Consistency, Availability, and Partition tolerance.

That "pick two of three" framing is the part Eric Brewer himself walked back in 2012. Partition tolerance is not optional because networks fail, and a system that does not tolerate them just falls over.

So the real choice is binary, and only active during a partition. Either you keep accepting writes and risk divergence (AP), or you refuse writes to protect correctness (CP). CA databases do not exist in any meaningful sense.

A more honest everyday framing is PACELC:

• Partition: choose Availability or Consistency.
• Else: choose Latency or Consistency.

When the network is healthy, and it usually is, you trade consistency against latency on every write. Synchronous quorum costs roughly 1-5 ms per write inside a region and 50-200 ms across regions, and a US East to Europe round trip is 80-100 ms before your code does anything.

CAP only shows up when something is on fire. PACELC is the tax you pay every day.

When you do have to pick a side for a specific piece of data, answer these four questions:

1. What is the cost of stale data? An old balance leads to a bad decision. An old like count leads to nothing.
2. What is the cost of downtime? If 30 seconds of unavailability costs thousands in lost sales, lean toward AP.
3. Can you reconcile later? Merging two shopping carts is easy. Merging two conflicting bank transfers is not.
4. How long do partitions last? Short ones favour CP. Long multi-region ones favor AP.

Most production systems use both CP for payments, orders, and auth tokens and AP for the catalog, recommendations, and activity tracking.

Once you know how fresh each piece of data must be, the list of databases that can hold it gets much shorter.

The SQL versus NoSQL argument is usually framed as a question of scale. It is not.

It is a question about the shape of your data and the way you query it.

Note: do not fall for the "SQL only scales horizontally with NoSQL" framing. That trade-off was real in 2018, and it is outdated now. Citus for PostgreSQL, Vitess for MySQL, Aurora Limitless, and distributed SQL engines like CockroachDB, Spanner, and YugabyteDB all give you horizontal write scaling while keeping ACID transactions. The cost is operational complexity and money.

Reach for NoSQL when your schema genuinely changes shape per record, when your data is naturally nested, or when you need millions of writes per second by a known key.

Stay with SQL when you need joins, ad-hoc queries, and transactions across multiple tables (though NoSQL databases like MongoDB and RavenDB do support transactions).

Most .NET systems end up using both: core transactional data in SQL Server or PostgreSQL with EF Core, high-volume or flexible data in Cosmos DB or MongoDB.

The factors that actually hurt six months later:

• How long a restore takes, and whether you have ever tested one?
• How long does it take to find a slow query and fix a broken one in production?
• Whether you can hire people who know this engine?

Note: RDS and Azure SQL can be restored to a point in time within the retention period. DynamoDB provides point-in-time recovery for up to 35 days. Redis gives you snapshots, not PITR at all. Test a restore once a quarter, because a backup you have never restored is not a backup.

The engine is chosen. The next question shows up when the traffic doubles.

Vertical scaling means one bigger (higher) box. Horizontal scaling means more boxes behind a load balancer.

Vertical is the boring, correct default for much longer than people admit.

A single VM today comfortably gives you roughly 100-200 general-purpose vCPUs. The ceiling is rarely CPU or RAM. It is cost and blast radius.

Per-vCPU pricing climbs sharply past the mid-tier, and around 16-32 vCPU SKUs, ten small instances become cheaper than one large one, with fault tolerance thrown in.

But horizontal is not free either. More boxes mean management overhead, higher metric cardinality in your monitoring bill, rolling deploys instead of a service restart, and a per-core license tax on every box you run.

Here is the part that decides the shape of your architecture: the decision tree splits at the storage layer.

Stateless app servers scale out trivially. Add a box, add it to the load balancer target group, done.

A relational database primary does not. There is exactly one writer, and that writer's CPU and IO is the ceiling for write throughput.

Read replicas scale reads. Writes only scale vertically, until you commit to sharding or distributed SQL.

That is why most production systems run a vertically scaled database primary next to a horizontally scaled app tier.

When the primary finally runs out, you shard: split the data across independent databases, each holding a subset. The shard key decides which database each row lives in.

Four criteria pick a good shard key, in priority order:

1. High cardinality. customer_id with millions of values spreads. country_code with 200 does not.
2. Uniform distribution. If 90% of traffic hits 10% of keys, that is a hotspot (bad shard).
3. Query locality. Most queries should be answerable from one shard, or scatter-gather becomes your dominant pattern.
4. Evolvability. Sharding by created_year puts all current traffic on the newest shard.

Note: changing a shard key later means rebalancing every row, which can be a multi-month dual-write, backfill, and cutover project. Treat it with the same care as a primary key.

Scaling adds machines. The cheaper move is to stop asking the database in the first place.

Three properties of the data choose a cache layer:

• Read frequency. How often is this requested?
• Change frequency. How often does the underlying value change?
• Staleness tolerance. How long can a user see an old value before it causes a real problem?

Score your data on those three, and the layer picks itself.

L1, in-process memory, takes microseconds and holds data read thousands of times a second that change rarely: feature flags, currency tables, role definitions. Each server has its own copy, so keep TTLs at 1-2 minutes.

L2, a distributed cache like Redis, takes around 1 ms and holds data read often, shared across servers, and changed occasionally: user profiles, catalogs, expensive query results. Every node sees the same value, so 10-30 minute TTLs work.

L3, the CDN edge, takes around 10 ms and holds data identical across many users: static assets, public API responses, pre-rendered pages. It never touches your servers at all.

Compare that to a database query at 10-100 ms and the reason to stack them is obvious.

Do not cache one-time-use tokens, or anything that changes faster than it is read. And one reliable sign that your layers are wrong: a local TTL longer than your Redis TTL means your servers are serving data that the shared cache has already thrown away.

Caching makes reads cheap. It does nothing for writes, and writes are where your components start waiting on each other.

The rule fits in one line: If the caller needs the answer right now, make a direct call (network?).

If the caller only needs the work to be done, send an event via a message queue.

Loading a product page needs an answer now. Charging a card needs an answer now. Sending the confirmation email, updating the search index, and notifying the warehouse do not.

What people underestimate is the bill.

A queue does not remove coupling. It moves it from a URL to a schema. Add a required field to your event and old consumers break. Rename one, and everyone breaks.

Plan event versioning on day one, with a schema registry or an explicit schemaVersion field.

A broker is also a new single point of failure, with its own clustering, dead-letter queues to monitor, and on-call expertise to hire for.

And almost every broker delivers at-least-once, so your consumer will eventually see the same message twice. It has to be idempotent, or you will charge a customer twice.

The cheapest defense is a unique constraint (in the database):

csharp
1public sealed class Order
2{
3    public Guid Id { get; init; }
4    public required string IdempotencyKey { get; init; }
5    public decimal Total { get; init; }
6}
7
8builder.Entity<Order>()
9    .HasIndex(o => o.IdempotencyKey)
10    .IsUnique();

The second insert of the same key throws, and that is intended. You catch the duplicate-key violation, treat the message as already handled, and acknowledge it. Without a middleware and distributed lock.

Choosing a broker comes down to three questions.

• Need to replay history? Kafka.
• Need complex routing rules? RabbitMQ.
• Want zero operations inside one cloud? SQS or Azure Service Bus.

A queue absorbs a burst. It does not create capacity. So what happens when the burst is bigger than the buffer?

"We need to handle 10x traffic" - it's how many systems got broken in real life (even big ones).

Before any architecture work, write the numbers down:

• Baseline requests per second at the tier that matters: "400 sustained, 800 at peak on the catalog."
• Peak target. Ten times that peak is 8000 requests per second.
• Read and write split per endpoint. They scale very differently.
• Latency targets as numbers. p99 under 200 ms for browsing, under 1000 ms for checkout.
• Inventory and error budgets. With 5000 units you can serve at most 5000 purchases, so design the "sold out" path too. And "5% error rate is acceptable" is a defensible posture where "0% errors" is not.

Note: These numbers are only examples.

Now decide who absorbs the pain when you exceed those numbers.

There are exactly three answers.

All three are valid. The architectural question is whose pain you prefer: the producer fails, the producer slows, or the producer waits.

Choosing without deciding gives you the worst of all three.

In .NET you get all of them from one primitive:

csharp
1builder.Services.AddRateLimiter(options =>
2{
3    options.AddConcurrencyLimiter("checkout", limiter =>
4    {
5        limiter.PermitLimit = 10;
6        limiter.QueueLimit = 20;
7        limiter.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
8    });
9});

PermitLimit is the throttle, QueueLimit is the bounded buffer, and anything past that is shed as a 503 automatically. Three config values, three strategies.

Always send Retry-After with a 429 or 503, and cap retries at two or three attempts with exponential backoff and jitter. A misconfigured retry policy turns one rejection into a cascade that overwhelms the limiter that issued it.

Two more things catch teams out.

Autoscaling arrives late. Metric collection takes 10-60 seconds, the scaling decision another 30-60, and provisioning plus warmup 60-300 more. That is 2-7 minutes from "load arrives" to "new instances serving traffic," which is exactly the window in which the sale happens.

Pre-scale to peak 10-15 minutes before the event and let autoscaling add headroom on top.

Scaling the app tier crushes the database. Fifty instances opening 100 connections each is 5000 connections against a PostgreSQL default of 100. Keep app_instances * max_pool_per_instance <= db_max_connections * 0.8, or put a pooler like PgBouncer or RDS Proxy in front.

Every decision so far was about one system. The last one is about how many systems you agree to run.

Microservices solve an organizational problem. Not necessarily a performance one.

That single sentence prevents most of the bad decisions made in this area.

"We might need to scale someday" is a bad reason. A stateless ASP .NET Core modular monolith behind a load balancer runs at 10-20 instances without any problem.

Vertical-first is advice for stateful components, not for your app tier.

"Our monolith is slow" is usually not a reason either. The problem is normally bad queries, missing indexes, or synchronous calls that should be async.

The real failure mode is building a distributed monolith: all the operational cost of microservices, none of the autonomy.

Five questions tell you whether you have one:

• Do many services have to be deployed together?
• Do services share a database?
• Do most user requests chain through three or more services synchronously?
• Does a schema change in one service require a coordinated change in another?
• Do you have one integration test suite that spins up everything?

Microservices earn their keep when:

• Multiple teams need to deploy on different schedules
• When parts of the system have wildly different scaling needs
• When one component's failure must not touch another.

Everything else is better served by a modular monolith: clean boundaries, one deployment, in-process calls, and one stack trace when something breaks. You can always extract a module into a service later once you have a concrete reason.

Let's recap the key takeaways:

• Freshness is a per-data-type decision. Payments need strong consistency, review counts do not. Most days you trade consistency against latency, not availability.
• The database is chosen by data shape, not by scale. Horizontal SQL is a solved problem now. What bites you later is restore time, debuggability, and who you can hire.
• Vertical scaling is the boring default, until the storage layer. The stateless tier scales out trivially. A relational write primary does not, and the shard key is the one decision you cannot walk back easily.
• Cache by read frequency, change frequency, and staleness tolerance. The layers compound, so each only sees what the one above it missed.
• A queue does not remove coupling, it moves it to a schema. At-least-once delivery means your consumer is idempotent or your customer gets charged twice.
• Write the capacity number down before designing for it. Then decide whether the producer fails, slows, or waits, because autoscaling will not arrive for another few minutes.
• Microservices solve an organizational problem first. start your app with a modular monolith.

There is no single correct answer to any of these. The right choice depends on your context, your constraints, and what you can afford to trade when things go wrong.

I am building a deep-dive System Design course that goes far past this list.

It will be better than anything you have seen so far:

• Reading
• Watching videos
• Designing real systems in the browser
• Taking tests

Join the waitlist here to get early access and a launch discount.

Hope you find this newsletter useful. See you next time.