Introduction

These are my notes from studying system design over at ByteByteGo.

Scale From Zero To Millions Of Users

Database

There are SQL databases and NoSQL ones. Relational databases are often the best ones because they have worked well and been around for a long time, however, NoSQL may be befitting when:

• Your application requires super-low latency.

• Your data are unstructured, or you do not have any relational data.

• You only need to serialize and deserialize data (JSON, XML, YAML, etc.).

• You need to store a massive amount of data.

Vertical & Horizontal scaling

Vertical scaling means scaling up, replacing an existing machine with a bigger and more expensive one. Horizontal scaling means scaling out and adding more machines. Horizontal scaling is more desirable due to the limitations of vertical scaling.

Load balancer

A load balancer evenly distributes incoming traffic among web servers that are defined in a load-balanced set.

Database replication

Database replication is when you have a master database and slave ones. Write operations happen on the master ones which then inform the slave ones. The read operations are done directly on the slave ones.

This results in:

• Better performance

• Reliability

• High availability

Cache

A cache is a temporary storage between the database and web server to serve frequent requests much more quickly. If data exists in the cache, we get it from the cache, otherwise, we get the data from the database and store the response in the cache.

A few considerations for using a cache:

• Decide when to use a cache.

• Expiration policy. When should data get expired and be removed.

• Consistency. Keeping the data in sync.

• Mitigating failures. Multiple cache servers across different data centers are recommended to avoid a single point of failure.

• Eviction policy. Once the cache is full, which data should get removed as new ones need to get added.

CDN

A CDN is a network of geographically distributed servers that serve static content such as images, videos, CSS, JS files, etc. They are close to the users, hence it's good to use them when serving static files.

Stateless web tier

As we're scaling horizontally, we need to consider where to move the state that sits in each web server. For instance user session data. A good practice here is to store such a state in a database. This state can then be shared across all the web servers.

Message queue

A message queue is a durable component, stored in memory, that supports asynchronous communication. You would have producers sending messages to the queue and subscribers who are consuming the messages. This reminds me of event-driven architecture.

The decoupling here makes this type of architecture nice. The consumer can still consume the messages if the producer doesn't work, and the same way goes the other way around. There is no direct touchpoint between producers and consumers.

Database Scaling

We can scale our database horizontally into shards. Each shard is responsible for a subset of data. The most important thing to consider here is the choice of the sharing key, also known as the partition key.

Sharding is a good technique, however, it introduces new challenges:

• Resharding data

• Celebrity problem

• Join and de-normalization

Back-of-the-envelope Estimation

Back-of-the-envelope calculations are estimates you create using a combination of thought experiments and common performance numbers to get a good feel for which designs will meet your requirements.

Tips when doing the estimation:

• Rounding and approximation.

• Write down your assumptions.

• Label your units. Don't write 5, write the unit too. It's easy to forget or confuse things.

• Commonly asked: QPS, peak QPS, cache, etc.

A Framework For System Design Interviews

A 4-step process for effective system design interview

Step 1 - Understand the problem and establish design scope

Don't answer right away. There is no right answer in system design interviews, it's a major red flag if you answer right away.

Ask questions. Think deeply and clarify the requirements. Take notes. The interviewer will either answer your questions or ask you to make an assumption.

Questions you could ask to understand the requirements:

• What specific features are we going to build?

• How many users does the product have?

• How fast does the company anticipate to scale up? What are the anticipated scales in 3 months, 6 months, and a year?

• What is the company’s technology stack? What existing services you might leverage to simplify the design?

Step 2 - Propose high-level design and get buy-in

Develop a high-level design and reach an agreement with the interviewer on the design. Collaborate with your interviewer, and treat them as a teammate.

• Come up with an initial blueprint.

• Draw box diagrams with key components.

• Do back-of-the-envelope calculations to evaluate if your blueprint fits the scale constraints.

Step 3 - Design deep dive

Objectives that should have been achieved so far:

• Agreed on the overall goals and feature scope

• Sketched out a high-level blueprint for the overall design

• Obtained feedback from your interviewer on the high-level design

• Had some initial ideas about areas to focus on in deep dive based on her feedback

Work with the interviewer to identify and prioritize components in the architecture to dive into. Depending on what type of system you are designing, different areas are more interesting diving into.

Step 4 - Wrap up

Directions that could follow towards the end:

• Identifying the system bottlenecks and discuss potential improvements. There is always something that could be improved, never be content or think your design is perfect.

• Give a recap of the design.

• Discuss handling error cases.

• Operation issues such as observability: Metrics, error logs, etc.

• How to handle the next scale curve.

List

List of Dos and Don'ts:

Dos:

• Always ask for clarification. Do not assume your assumption is correct.

• Understand the requirements of the problem.

• Let the interviewer know what you are thinking. Communicate with your interview.

• Suggest multiple approaches if possible.

• Once you agree with your interviewer on the blueprint, go into detail on each component. Design the most critical components first.

• Bounce ideas off the interviewer. A good interviewer works with you as a teammate.

Don'ts:

• Don't be unprepared for typical interview questions.

• Don’t jump into a solution without clarifying the requirements and assumptions.

• Don’t go into too much detail on a single component in the beginning. Give the high-level design first then drill down.

• If you get stuck, don't hesitate to ask for hints.

• Again, communicate. Don't think in silence.

• Don’t think your interview is done once you give the design. You are not done until your interviewer says you are done. Ask for feedback early and often.