Why Redis Is Fast

1. An Engineering Question

A few months ago, we were building a feature that required retrieving configuration toggles and basic feature flags on almost every API call. This data was accessed frequently but updated rarely. To keep the retrievals fast, we set up a Redis instance to store these settings.

It worked exactly as expected. But after a few days, the engineering curiosity kicked in. I started questioning the setup:

• Why did we need a separate tool like Redis?
• Could we not have simply created a small, highly indexed table in our existing PostgreSQL database and queried that?
• Why is making a database call to a small PostgreSQL table slower than querying Redis?
• What makes Redis fundamentally faster than standard databases at a code and operating system level?

This question led me down a rabbit hole of database internals and operating system designs. It turned out that Redis's speed isn't a magic trick of 'running in RAM'—it is the result of continuous bottleneck removal at the system level. This article summarizes those architectural differences.

Before putting this article together, I spent a lot of hours digging into the source code and understanding the system-level details of Redis, debating extensively with ChatGPT on the trade-offs, thread dynamics, and CPU constraints. I have prepared this article as a crux and a summary of my key learnings from that deep dive.

2. Why Not Just a Database Table?

If you query a small, 100-row table in an indexed PostgreSQL database, the data is cached in PostgreSQL's shared buffers (RAM). Yet, that query still takes 1.5ms to 3ms, whereas Redis returns the same key in 0.2ms. Why?

The difference lies in the design goals. Relational databases are built for complex SQL queries, multi-table joins, ACID transactions, and disk-bound reliability. Even when data is fully loaded in memory, a query must still be parsed, optimized, and execute locks. Redis, by contrast, removes these layers. It trades relational query flexibility in exchange for direct memory lookup paths.

3. Bypassing the Disk

The first major optimization is the physical storage medium. Disk drives are slow, SSDs are faster, but main memory (RAM) is in a completely different league. To put this in perspective: if accessing the CPU L1 cache is like grabbing a coffee on your desk, accessing RAM is like walking down the street to a café. Fetching data from an SSD? That is like booking a flight to another city. And a spinning HDD? That is a journey around the world.

Let's look at the storage hierarchy latencies:

By keeping all active data directly in RAM, Redis avoids the slow disk read path. Writes and reads are pointer offsets in memory—not physical disk seeks or cache page swaps.

4. Algorithmic Efficiency

How data is represented in memory dictates how many CPU cycles are spent looking it up. Relational databases use search trees (like B+ Trees or LSM-Trees) to navigate data pages. Redis uses a global HashTable where every key maps directly to a value, providing O(1) lookups.

Additionally, Redis is not just a basic key-value string store. It exposes rich, in-memory structures like Hashes, Lists, Sets, and Sorted Sets. Each structure is implemented with extreme care. Take Redis Strings, for example: they do not use standard null-terminated C strings. Instead, they use a custom representation called Simple Dynamic Strings (SDS). The SDS header stores the string length, which means checking string length is O(1) instead of O(N), and appends are memory-safe.

5. Zero SQL Compiler Overhead

In a standard SQL database, a query string has to pass through a massive pipeline: parsing the SQL text, compiling an Abstract Syntax Tree (AST), checking schemas and permissions, running the query optimizer, selecting indices, and executing the plan.

When you run GET user:42 in Redis, there's no parsing engine or query planner. The protocol maps the `GET` command directly to a C function pointer, hashes the key, and retrieves the data from memory. By keeping it simple, Redis saves crucial CPU cycles.

6. The Concurrency Paradox

One of the most counter-intuitive decisions in Redis is its single-threaded execution model. Beginners often ask, "How can a single thread be faster than my multi-core CPU?" The answer lies in the cost of concurrency. Multi-threaded systems spend a massive amount of CPU cycles on thread scheduling, context switching, cache line invalidations, and lock contention. When two threads try to update the same memory block, one has to wait. Redis avoids all of this by processing client commands sequentially on a single thread. No locks, no context switches, and no race conditions. It's simple, predictable, and incredibly fast.

7. Multiplexing Network I/O

But how does a single thread handle 50,000 active connections at once? If we spawned a thread for every client connection (the classic thread-per-connection model), the OS scheduler would fall over. Redis solves this with non-blocking I/O multiplexing. Under the hood, it leverages OS-specific mechanisms like epoll on Linux or kqueue on macOS. The single main thread monitors thousands of network sockets. When data is ready on a socket, an event triggers, and Redis processes it immediately. This event-driven loop keeps memory usage low and ensures that connection scaling doesn't hurt performance.

8. Decoupling Disk Writes

What happens if the power goes out? If Redis kept everything in memory without writing to disk, it would be a volatile cache, not a database. But disk writes are slow. To preserve its sub-millisecond latencies, Redis doesn't block client writes while writing to disk. Instead, it decouples persistence. You can configure it to take periodic binary snapshots (RDB) or log write commands in the background (AOF). These persistence tasks run on fork-created background processes, leaving the main execution thread free to serve clients.

9. Architectural Matrix

By layering these optimizations together, Redis systematically removes latency at every level of the query pipeline. Here is the architectural summary of traditional databases versus the Redis approach:

10. Latency Benchmarks

To put this in perspective, here is a latency comparison for a simple key query under identical hardware constraints:

11. When Redis Isn't Fast

Redis is not a silver bullet. Its speed degrades under specific workloads:

12. Key Takeaways

The core lesson of Redis is that high performance is rarely achieved by a single optimization. Instead, it is the result of continuous bottleneck elimination: storing data in RAM exposed network bottlenecks, solved by epoll; which exposed lock contention bottlenecks, solved by single-threading; which exposed disk-write bottlenecks, solved by background persistence.

Apply this compounding optimization mindset to your own distributed systems, APIs, and front-end architectures.