Phase 1: Architecture, Process Model & Sandbox Mechanics¶
Audience: Principal Systems Architect / High-Performance C Engineer
Source References:src/server.c,src/networking.c,src/ae.c,src/config.c
1. The Single-Threaded Philosophy & Kernel Context¶
Why did Redis opt for a single-threaded execution model when multi-core CPUs became ubiquitous?
1.1 The In-Memory Performance Equation¶
In an in-memory database, typical operation durations are on the order of tens to hundreds of nanoseconds: - L1 cache hit: \(\approx 1 \text{ ns}\) - L2 cache hit: \(\approx 3-4 \text{ ns}\) - L3 cache hit: \(\approx 10-15 \text{ ns}\) - DRAM access: \(\approx 60-100 \text{ ns}\)
In comparison, the synchronization primitives required for multi-threaded state management impose substantial latency penalties:
- pthread_mutex_lock (uncontended): \(\approx 20-30 \text{ ns}\)
- pthread_mutex_lock (contended / futex syscall): \(\approx 1,000-5,000 \text{ ns}\)
- Inter-core cache line bouncing (MESI cache protocol invalidation when Core 1 writes to a pointer cached in Core 2): \(\approx 40-80 \text{ ns}\) per write.
Multi-Threaded Locking Bottleneck:
Core 0: [Mutex Lock] ──► [Write to Hash Slot] ──► [Mutex Unlock]
▲ │
│ (Contention Queue: Futex Sleep) ▼
Core 1: [Wait / Sleep in Kernel] ◄──────────────────────┘
(Cost: 2 context switches + CPU cache invalidation = ~10 microseconds!)
Single-Threaded Multiplexed Engine:
Core 0: [Event 1: GET] ──► [Event 2: SET] ──► [Event 3: INCR] ──► [Event 4: LPUSH]
(Zero Locks, Zero Kernel Context Switches, 100% CPU Cache Line Reuse!)
1.2 Eliminating Race Conditions¶
Because command execution is non-preemptive and sequential, every Redis command is implicitly atomic. Operations like INCR, HSET, and LPUSH do not require read-modify-write locks or compare-and-swap (CAS) CPU instructions.
2. Anatomy of the Redis Process & Threads¶
While the command execution engine is strictly single-threaded, a modern Redis instance (redis-server) spawns multiple internal operating system threads (src/bio.c - Background I/O):
┌────────────────────────────────────────────────────────────────────────┐
│ redis-server (OS Process) │
├────────────────────────────────────────────────────────────────────────┤
│ [Main Thread] │
│ - aeEventLoop (epoll / kqueue multiplexing) │
│ - Command Table Parsing & Dispatch (processCommand) │
│ - In-Memory Dict Mutation (dictFind, dictAdd) │
│ - Replication Buffer Management & Heartbeats │
├────────────────────────────────────────────────────────────────────────┤
│ [Background BIO Thread 1: BIO_CLOSE_FILE] │
│ - Asynchronous close() of stale file descriptors (avoids disk wait) │
├────────────────────────────────────────────────────────────────────────┤
│ [Background BIO Thread 2: BIO_AOF_FSYNC] │
│ - Background fsync() flushing AOF dirty OS pages to physical disk │
├────────────────────────────────────────────────────────────────────────┤
│ [Background BIO Thread 3: BIO_LAZY_FREE] │
│ - Asynchronous reclamation of memory for UNLINK, FLUSHDB ASYNC │
├────────────────────────────────────────────────────────────────────────┤
│ [Threaded I/O Worker Pool (Redis 6.0+)] (io-threads 4) │
│ - Worker 1: Socket read() + RESP Protocol Parsing │
│ - Worker 2: Socket read() + RESP Protocol Parsing │
│ - Worker 3: Socket write() + Output Buffer Flushing │
└────────────────────────────────────────────────────────────────────────┘
3. Client Connection Lifecycle & Buffer Internals¶
Every client connection is represented in C by the client struct (src/server.h):
typedef struct client {
uint64_t id; /* Incremental unique client ID */
connection *conn; /* Socket connection abstraction (POSIX fd or TLS) */
int resp; /* RESP protocol version (2 or 3) */
redisDb *db; /* Pointer to currently selected redisDb (0-15) */
sds querybuf; /* Dynamic SDS buffer storing incoming raw bytes */
size_t qb_pos; /* Current read position in querybuf */
struct redisCommand *cmd, *lastcmd; /* Command table lookup result */
list *reply; /* Linked list of output buffer reply blocks */
char buf[PROTO_REPLY_CHUNK_BYTES]; /* Static 16KB quick output buffer */
int bufpos; /* Current byte offset in static reply buffer */
size_t reply_bytes; /* Total bytes in output reply list */
/* ... security flags, client tracking, authenticated user ... */
} client;
3.1 The Input Buffer (querybuf) Mechanics¶
- When bytes arrive on the socket, the
aeevent loop triggersreadQueryFromClient(). - Bytes are read into
c->querybufviaread(fd, buf, size). - If a rogue client sends a stream of bytes without terminating with
\r\nor pipelined requests exceedingclient-query-buffer-limit(default 1GB), Redis terminates the client with a connection reset to prevent RAM exhaustion.
3.2 The Output Buffer Crisis & client-output-buffer-limit¶
When Redis generates responses (e.g., executing LRANGE mylist 0 -1 returning 500,000 items), the response is formatted and pushed to:
1. c->buf: A static fast buffer of 16KB (PROTO_REPLY_CHUNK_BYTES).
2. If c->buf is full, subsequent bytes are appended as SDS nodes in c->reply (a double-linked list).
If the client is slow to read (e.g., high latency network link) while the server produces data rapidly, c->reply expands indefinitely in RAM until the OS terminates the entire Redis instance via OOM Killer!
Defensive Protection: client-output-buffer-limit¶
# client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
4. Master Configuration Anatomy (redis.conf)¶
A production-grade redis.conf requires deliberate kernel-aligned settings:
################################## NETWORK #####################################
bind 0.0.0.0
protected-mode yes
port 6379
tcp-backlog 2048 # Must match Linux sysctl net.core.somaxconn
timeout 300 # Close idle connections after 300 seconds
tcp-keepalive 300 # Send TCP ACKs every 300s to detect dead client sockets
################################# MEMORY ######################################
maxmemory 16gb # Hard ceiling for dataset
maxmemory-policy volatile-lru # Eviction strategy once ceiling is hit
maxmemory-samples 10 # Number of keys sampled for LRU approximation
################################ PERSISTENCE ###################################
appendonly yes
appendfilename "appendonly.aof"
appenddirname "appendonlydir"
appendfsync everysec # Flush dirty pages every 1 second (1-second data loss SLA)
no-appendfsync-on-rewrite yes # Prevent disk fsync stalls during background rewrite
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
############################## THREADED I/O ####################################
io-threads 4 # Match physical CPU cores (typically total_cores / 2)
io-threads-do-reads yes # Enable multi-threaded socket reads and protocol parsing
5. Linux Kernel Tuning Matrix for Redis¶
Before running Redis in production, the host operating system kernel must be tuned to eliminate severe latent stalls:
| Kernel Parameter | Recommended Value | Pathological Failure if Not Set |
|---|---|---|
vm.overcommit_memory |
1 |
fork() fails during BGSAVE or AOF rewrite because Linux thinks RAM is insufficient. |
transparent_hugepage |
never |
Extreme latency spikes (up to 40ms) during BGSAVE due to 2MB page fault copy-on-write overhead. |
net.core.somaxconn |
4096 |
High connection rates get silently dropped at the TCP handshake queue, causing client timeouts. |
fs.file-max |
1000000 |
Redis hits EMFILE ("Too many open files") and rejects incoming client connections. |