Overview of System Design

System design is the process of planning the architecture, components, data and communication of a software system so that it meets its goals. It helps engineers build systems that are scalable, fast, reliable and easy to maintain, before a single line of production code is written.

This tutorial explains system design in simple terms: why it matters, the building blocks every system uses, the core ideas such as scalability and availability, a step-by-step design process, and a complete example of an online bookstore with Java code.

1. What Is System Design?

Writing code solves a single problem inside one program. System design looks at the whole picture: how many servers you need, where data is stored, how parts talk to each other, what happens when something fails, and how the system grows when more people use it.

A good system design answers questions like these:

• What should the system do, and for how many users?
• Which parts (services, databases, caches, queues) are needed, and how do they connect?
• How is data stored, read and kept safe?
• How does the system keep working when a server or network fails?
• How will it handle ten or a hundred times more traffic in the future?

2. Why Is System Design Important?

• Scalability: The system can handle growth in users, requests and data without a full rewrite.
• Performance: Pages and APIs respond quickly, even when many people use them at once.
• Reliability and Availability: The system stays up and keeps data safe, even when some parts fail.
• Maintainability: Clear boundaries make it easier to fix bugs and add features without breaking existing ones.
• Security: Sensitive data such as passwords and payment details is protected from the start.
• Cost Control: You use the right amount of infrastructure instead of over- or under-buying servers.
• Career Growth: System design is a core skill for senior engineers and a standard part of technical interviews.

3. Functional and Non-Functional Requirements

Every design starts with requirements. They come in two types:

Non-functional requirements usually drive the big design decisions. Learn more in Functional vs Non-Functional Requirements.

4. High-Level vs Low-Level Design

See High-Level vs Low-Level Design for a deeper comparison.

5. Key Components of System Design

Most large systems are built from the same set of building blocks:

6. Core Concepts

Scalability

Scalability is the ability to handle more load by adding resources. There are two ways to scale:

Availability

Availability is the percentage of time a system is up and working. It is often written as a number of "nines":

Higher availability comes from redundancy (more than one copy of every important part) and failover (switching to a healthy copy automatically). Read more in Availability in System Design.

Latency and Throughput

• Latency is how long one request takes, for example 120 ms to load a page. Lower is better.
• Throughput is how many requests the system handles per second. Higher is better.

Caching, CDNs, efficient queries and fewer network hops reduce latency. See Latency in System Design.

Consistency and the CAP Theorem

Consistency means every user sees the same, latest data. In a distributed system, the CAP theorem says that when the network splits (a partition), you must choose between:

• Consistency: refuse some requests so nobody sees outdated data, or
• Availability: keep answering every request, even if some answers are slightly out of date.

A bank balance usually favours consistency; a social media like counter usually favours availability. See Consistency in System Design.

7. A Step-by-Step Design Process

Whether you are designing a real product or answering an interview question, these steps give you a clear path:

1. Clarify requirements: List the main features and the non-functional goals (users, speed, availability).
2. Estimate the scale: Roughly calculate requests per second, storage and bandwidth.
3. Draw the high-level design: Clients, load balancer, services, databases, caches and queues.
4. Design the data model: Choose databases and define the main tables or documents.
5. Define the APIs: Decide the endpoints, inputs and outputs.
6. Deep dive into key parts: Search, payments or whatever is hardest or most important.
7. Find bottlenecks and failures: Ask "what breaks first?" and add caching, replicas or queues.
8. Review trade-offs: Explain why you chose each option and what you gave up.

For more detail, see System Design Process and Methodologies.

8. Example: Designing an Online Bookstore

Imagine building an online bookstore. System design helps you decide:

• Which database will store books, users and orders?
• How will users search for books quickly?
• How will the site handle thousands of users at the same time?
• How will payment information stay secure?
• What happens if a server crashes during a big sale?

Step 1: Requirements

• Functional: browse and search books, view details, add to cart, place orders, pay online.
• Non-functional: 1 million daily active users, search in under 200 ms, 99.9% availability, secure payments.

Step 2: Back-of-the-Envelope Estimation

Daily active users        = 1,000,000
Requests per user per day = 10
Requests per day          = 10,000,000
Average requests/second   = 10,000,000 / 86,400  ≈ 116 req/s
Peak (about 3x average)   ≈ 350 req/s

Books in catalog          = 1,000,000
Book details (2 KB each)   ≈ 2 GB    -> fits easily in one database
Cover images (200 KB each) ≈ 200 GB  -> object storage + CDN

These rough numbers tell us that one database can hold the catalog, but images belong in object storage behind a CDN, and a few app servers behind a load balancer can comfortably handle the traffic.

Step 3: High-Level Architecture

 Users (web / mobile)
    |
    |---------------------------> [ CDN ] <--- book cover images (object storage)
    v
 [ Load Balancer ]
    |
    +--> App Server 1 --+
    +--> App Server 2 --+--> [ Redis Cache ]      (popular books)
    +--> App Server 3 --+
                        |
                        +--> [ PostgreSQL ]       (books, users, orders)
                        +--> [ Search Engine ]    (fast full-text search)
                        +--> [ Message Queue ] --> Email / Shipping workers

Step 4: Key Decisions

• Database: PostgreSQL for books, users and orders, because orders need reliable transactions.
• Fast search: a search engine such as Elasticsearch, kept in sync with the database.
• Handling many users: stateless app servers behind a load balancer, plus a Redis cache for popular books.
• Images: object storage with a CDN, so images load quickly worldwide.
• Slow tasks: confirmation emails and shipping requests go through a message queue, so checkout stays fast.
• Security: HTTPS everywhere, hashed passwords, and a trusted payment provider so card details never touch our servers.
• Reliability: at least two app servers and a database replica, so one failure does not stop the store.

By planning all these aspects up front, the bookstore can run smoothly, handle growth and give users a good experience.

9. System Design Ideas in Java

System design is mostly about architecture, but its ideas show up directly in code. Here are four small Java examples from the bookstore.

API Design: A Clean REST Endpoint

A versioned URL (/api/v1) and paging with a maximum page size keep the API stable and protect the server from huge requests.

package com.javabytechie.bookstore;

import java.util.List;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/v1/books")
public class BookController {

    private static final int MAX_PAGE_SIZE = 100;

    private final BookService bookService;

    public BookController(BookService bookService) {
        this.bookService = bookService;
    }

    @GetMapping("/{id}")
    public BookResponse getBook(@PathVariable Long id) {
        return bookService.findById(id);
    }

    @GetMapping
    public List<BookResponse> search(@RequestParam String query,
                                     @RequestParam(defaultValue = "0") int page,
                                     @RequestParam(defaultValue = "20") int size) {
        // Never let a client ask for an unlimited number of results.
        return bookService.search(query, page, Math.min(size, MAX_PAGE_SIZE));
    }
}

Caching: Fast Reads for Popular Books

With the cache-aside approach, the first request loads a book from the database and stores it in the cache; later requests are served from memory. When data changes, the cached copy is removed.

package com.javabytechie.bookstore;

import java.math.BigDecimal;
import java.util.List;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.data.domain.PageRequest;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class BookService {

    private final BookRepository bookRepository;

    public BookService(BookRepository bookRepository) {
        this.bookRepository = bookRepository;
    }

    @Cacheable(cacheNames = "books", key = "#id")
    public BookResponse findById(Long id) {
        // Runs only on a cache miss; the result is then stored in the "books" cache.
        return bookRepository.findById(id)
                .map(BookResponse::from)
                .orElseThrow(() -> new BookNotFoundException(id));
    }

    @Transactional
    @CacheEvict(cacheNames = "books", key = "#id")
    public void updatePrice(Long id, BigDecimal newPrice) {
        // Removing the cached copy makes the next read load the new price.
        Book book = bookRepository.findById(id)
                .orElseThrow(() -> new BookNotFoundException(id));
        book.setPrice(newPrice);
    }

    public List<BookResponse> search(String query, int page, int size) {
        return bookRepository.findByTitleContainingIgnoreCase(query, PageRequest.of(page, size))
                .stream()
                .map(BookResponse::from)
                .toList();
    }
}

package com.javabytechie.bookstore;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;

@SpringBootApplication
@EnableCaching // switches on @Cacheable and @CacheEvict
public class BookstoreApplication {

    public static void main(String[] args) {
        SpringApplication.run(BookstoreApplication.class, args);
    }
}

Adding spring-boot-starter-data-redis makes Spring Boot store this cache in Redis, so all app servers share it.

Load Balancing: Round Robin

A load balancer sends each new request to the next server in turn. This simplified version shows the idea in plain Java:

import java.util.List;
import java.util.concurrent.atomic.AtomicInteger;

public class RoundRobinLoadBalancer {

    private final List<String> servers;
    private final AtomicInteger counter = new AtomicInteger(0);

    public RoundRobinLoadBalancer(List<String> servers) {
        if (servers.isEmpty()) {
            throw new IllegalArgumentException("At least one server is required");
        }
        this.servers = List.copyOf(servers);
    }

    public String nextServer() {
        // floorMod keeps the index valid even after the counter overflows.
        int index = Math.floorMod(counter.getAndIncrement(), servers.size());
        return servers.get(index);
    }

    public static void main(String[] args) {
        RoundRobinLoadBalancer balancer = new RoundRobinLoadBalancer(
                List.of("app-server-1", "app-server-2", "app-server-3"));

        for (int request = 1; request <= 6; request++) {
            System.out.println("Request " + request + " -> " + balancer.nextServer());
        }
    }
}

Request 1 -> app-server-1
Request 2 -> app-server-2
Request 3 -> app-server-3
Request 4 -> app-server-1
Request 5 -> app-server-2
Request 6 -> app-server-3

Protecting the System: Token Bucket Rate Limiter

Rate limiting stops one client from overloading the system. In the token bucket approach, a bucket refills with tokens at a fixed speed, and each request must take one token. When the bucket is empty, the request is rejected.

public class TokenBucketRateLimiter {

    private final long capacity;
    private final double refillTokensPerNano;
    private double availableTokens;
    private long lastRefillTime;

    public TokenBucketRateLimiter(long capacity, long refillTokensPerSecond) {
        this.capacity = capacity;
        this.refillTokensPerNano = refillTokensPerSecond / 1_000_000_000.0;
        this.availableTokens = capacity;
        this.lastRefillTime = System.nanoTime();
    }

    public synchronized boolean tryAcquire() {
        refill();
        if (availableTokens >= 1) {
            availableTokens -= 1;
            return true;   // request allowed
        }
        return false;      // request rejected (HTTP 429 Too Many Requests)
    }

    private void refill() {
        long now = System.nanoTime();
        double newTokens = (now - lastRefillTime) * refillTokensPerNano;
        availableTokens = Math.min(capacity, availableTokens + newTokens);
        lastRefillTime = now;
    }
}

For example, new TokenBucketRateLimiter(10, 5) allows a burst of up to 10 requests and then about 5 requests per second. Learn more in the Rate Limiting Pattern tutorial.

10. Common Trade-offs

There is no perfect design, only good trade-offs for a given situation:

Explore architecture choices in Monolithic Architecture and Microservices Architecture.

11. Tips for Beginners

1. Always start with requirements; never jump straight to technologies.
2. Keep the first design simple, then improve it where the numbers say it is needed.
3. Do rough math for traffic and storage; it guides every big decision.
4. Remove single points of failure by having at least two of everything important.
5. Keep application servers stateless so they are easy to scale out.
6. Cache data that is read often and changes rarely.
7. Move slow work, such as emails and reports, to background queues.
8. Design for failure: use timeouts, retries and health checks.
9. Plan monitoring and alerts from day one.
10. Explain every choice with its trade-off; "it depends" is fine if you say on what.

12. Conclusion

System design is a critical skill for developers and engineers. It turns a list of features into a blueprint that is scalable, reliable, fast, secure and affordable. Start with clear requirements, estimate the scale, combine well-known building blocks such as load balancers, databases, caches and queues, and always explain your trade-offs. Learning system design helps you think bigger and plan smarter, and the rest of this System Design Tutorial will take you deeper into each topic.

Frequently Asked Questions (FAQ)

1,000,000 users x 10 requests/day = 10,000,000 requests/day
10,000,000 / 86,400 seconds        ≈ 116 requests/second