I/O Multiplexing, Explained Visually: select, poll, and epoll

Picture a chat server with 10,000 open connections. At any given moment, maybe 100 of them have data waiting to be read. The other 9,900 are silent.

If your program tries to read a silent connection, it stops. It sits there, waiting for a client who may not type anything for another hour, while 99 other clients are tapping their fingers. That is the problem I/O multiplexing solves: how does one thread wait for many connections at once, without wasting CPU and without one thread per connection?

By the end of this post you will be able to answer that from first principles. We will start with the designs people actually used — a process or a thread per connection — and the wall each one hits. Then we will look at how the three classic Linux APIs — select, poll, and epoll — solve it, and finish with a complete working server and the mistakes that show up in real code.

First, the words that confuse everyone

Before the diagrams, let’s make four terms concrete. None of them are as scary as they sound.

The kernel is the part of the operating system that owns the hardware. Your program cannot touch a network card directly; it asks the kernel through system calls (syscalls) such as read, write, and accept.

A file descriptor (fd) is a small integer that names “an open thing” inside your process: a file, a pipe, a terminal, or a network socket. Your process has a table of them. When you open a socket, the kernel picks the lowest free number and hands it back.

A socket is one endpoint of a network conversation. After a client connects, your server gets a new socket for that client. The number that comes back, say 7, is what you pass to read() and write().

Blocking means “the syscall does not return until it can finish.” If you call read() on a socket with no data, the kernel puts your thread to sleep and runs something else. It is not burning CPU — it is genuinely asleep — but your thread cannot do anything else meanwhile.

That last sentence is the entire tension. Sleeping is good for the CPU and bad for your throughput, because one thread can only sleep on one thing at a time.

Blocking I/O: the default, and why it hurts

Here is what happens during a blocking read(fd, buf, 4096) on a socket with an empty receive buffer:

1. Your thread traps into the kernel.
2. The kernel checks the socket’s receive buffer. Empty.
3. The kernel marks your thread as sleeping and removes it from the CPU run queue.
4. The scheduler runs another thread — maybe a completely different process.
5. Later, a packet arrives, the kernel copies it into the socket buffer, and marks your thread runnable.
6. Your thread wakes up, copies the bytes into buf, and read returns.

Step 3 is the important one. Blocking is actually efficient: the CPU is free while you wait. The problem is that the thing doing the waiting is your only thread. If that thread is asleep on fd 3, it cannot serve fd 4.

That single fact — one blocked thread waits for exactly one socket — shaped decades of server design. Before we look at the fix, it is worth meeting the designs people reached for. Each one is a reasonable answer to a real constraint, and each one breaks in a specific way.

How servers handle many connections

Every server that accepts a connection has to answer one question: who waits for the next byte? There are only a few honest answers, and they have not changed much in forty years.

Process per connection

The oldest answer is to let the kernel do the bookkeeping. The parent accepts, forks a child, and the child spends its whole life in a blocking read loop.

for (;;) {
    int cfd = accept(lfd, NULL, NULL);
    if (cfd < 0) continue;

    if (fork() == 0) {        /* child: one connection, one process */
        close(lfd);           /* it must not accept new connections */
        serve(cfd);           /* plain blocking read/write loop      */
        _exit(0);
    }
    close(cfd);               /* parent: back to accepting           */
}

This is the inetd model, and it is still with us: PostgreSQL forks a backend process per connection, and PHP-FPM hands each request to a worker process. The appeal is that the code is linear. serve reads, thinks, writes, and reads again; the OS does the scheduling, and a crash in one connection cannot corrupt another. The costs are the ones you would guess: every connection carries a whole address space, sharing a cache or a session table needs shared memory or IPC, and creating a process is heavier than creating a thread.

Thread per connection

Swap the process for a thread and you keep the linear code while paying less per connection:

for (;;) {
    int cfd = accept(lfd, NULL, NULL);
    if (cfd < 0) continue;
    pthread_t t;
    if (pthread_create(&t, NULL, serve, &cfd) == 0)
        pthread_detach(t);    /* serve() blocks on read(cfd) */
}

Real code passes the fd through a small heap struct and checks the return value; the shape is what matters. Now every connection lives in one address space, so a shared hash table with a mutex is enough. This is the design behind classic Apache worker, many Java servlet containers, and the first version of nearly every server anyone writes. It works. It is also the design that made the C10K problem famous.

A bounded thread pool

Unbounded threads are a liability, so the next refinement is a fixed pool: a small number of workers pull work from a queue.

/* N worker threads, one shared queue */
for (;;) {
    int cfd = queue_pop(&q);  /* blocks when the queue is empty */
    serve(cfd);               /* and blocks again on read(cfd)   */
}

This caps concurrency and memory, which is exactly what you want for CPU-bound work. For long-lived connections it has a catch: a worker is pinned to its client for the entire session, so eight workers means eight busy clients — not eight thousand idle ones. A client that connects and says nothing for an hour owns 1/8 of your capacity for an hour.

Busy polling

The other extreme is to set every socket non-blocking and spin:

for (;;) {
    for (int i = 0; i < n; i++) {
        ssize_t r = read(fds[i], buf, sizeof buf);
        if (r > 0) handle(fds[i], buf, (size_t)r);
    }
}

No thread blocks, so one thread can serve every connection. But the loop never sleeps: with 10,000 sockets and three ready, it still asks 10,000 times, then starts over. Polling converts waiting into CPU burn. It is fine for a device driver where events are constant; it is a space heater for a chat server.

model	who waits	per-connection cost	typical use
process per connection	one process	address space, fork, IPC for sharing	PostgreSQL, PHP-FPM
thread per connection	one thread	stack, scheduler entry, locks	classic app servers
bounded thread pool	one worker	queueing, head-of-line blocking	CPU-bound jobs
busy polling	the CPU	100% of a core, always	drivers, not servers

A fifth family changes the contract rather than the resource: completion-based I/O (io_uring on Linux, IOCP on Windows) lets you hand the kernel a buffer and get told when the read finished. That is a bigger conceptual step than readiness, and we will come back to it near the end.

What is wrong with all of them

Read the table again and notice what every row has in common: waiting is attached to an execution unit. A process, a thread, or a spinning core is consumed by the act of waiting for one specific connection.

That is a poor match for how network traffic actually looks. A busy chat server might hold 10,000 connections and see data on 100 of them in a given second. The other 9,900 are idle, and idle should be nearly free. In the models above it is not:

1. Idle costs the same as active. 10,000 threads means 10,000 stacks, 10,000 kernel task structs, and 10,000 entries for the scheduler to keep track of — to do the work of 100. A connection that says nothing for an hour still reserves everything.
2. Memory adds up quietly. glibc reserves 8 MB of stack per thread by default on x86-64. Most of it is untouched virtual memory, but the guard pages, page tables, and kernel-side structures are real, and a few thousand threads will show up in ps and in your container’s memory limit long before you expect it.
3. Every wakeup costs a context switch. Each blocking read parks a thread and later resumes it; that round trip costs a trap, a scheduler decision, and cache and TLB damage. Add thousands of mostly idle connections doing this and the overhead becomes a measurable share of the machine.
4. Shared state needs locks. The moment two threads touch the same room list or session table, you need mutexes. Now you have lock contention, priority inversion, and bugs that only reproduce under load. A single-threaded event loop has none of that by construction.
5. A pool turns concurrency into a queue. A fixed pool protects the machine but caps how many clients can be connected and quiet at once. The ninth client waits even though it has nothing to say.
6. The kernel has limits. ulimit -u, pids.max in a cgroup, and plain memory all bound how many threads you can create. Hitting those limits is an outage, not a slowdown.
7. One slow handler can poison the rest. A worker stuck in a slow database call stalls its client; in a pool with a shared queue, a few slow requests can starve everyone behind them.

None of this means these designs are wrong. Thread-per-connection is still a good default for a few hundred connections, and for CPU-heavy work a thread pool is exactly right. The point is narrower: they spend a whole execution unit to express “wait for one socket,” and that does not scale when most sockets are waiting.

I/O multiplexing: one wait, many sockets

The word multiplexing comes from communications: combining several signals onto one channel so a single carrier can serve them all. In an operating system it means the same thing for I/O — one thread waits on a whole set of file descriptors, and the kernel reports which of them are ready.

Concretely, you hand the kernel a set of descriptors and a timeout. The call blocks until at least one descriptor in the set is ready (or the timeout expires), then returns the ready subset. The thread is never parked on a particular connection; it is parked on the set.

wait({fd 3, fd 4, fd 5, ...}, timeout) -> {fd 4}    # fd 4 has bytes
wait({fd 3, fd 4, fd 5, ...}, timeout) -> {}       # timeout, nothing ready

Three properties follow, and they are the whole reason the model wins:

• The cost of waiting is decoupled from the number of connections. One thread, one stack, one scheduler entity — whether you watch 10 descriptors or 10,000. Each connection costs a small kernel structure, not a thread.
• Idle is cheap. A silent socket sits in a data structure. It does not occupy a worker, a stack, or a core.
• Readiness is reported, not performed. The kernel tells you who can make progress; you still call read and write. That keeps the API small and lets you decide how much work to do per connection.

It is worth being precise about what you get. Multiplexing gives you concurrency without parallelism: one thread makes progress on many connections, but it handles them one at a time. It does not make CPU-bound work faster and it does not add cores. What it removes is the waste of having a thread asleep per connection.

Next to a thread-per-connection server and a spinning loop, multiplexing is the middle path: one thread, one sleep, many sockets. You hand the kernel the whole set, sleep once, and wake up with a list of sockets that are actually ready. Linux exposes three classic versions of this idea — select, poll, and epoll — and the differences between them matter more than you would think.

Why the kernel can answer at all

A definition is only useful if the kernel can actually answer the question. It can, because readiness is already part of its state. When a packet arrives, the network stack copies it into that socket’s receive buffer and marks the socket readable. When a socket’s send buffer has room, it marks the socket writable. Your program does not have to interrogate the hardware; the answer is waiting in the kernel.

That is what the multiplexing syscalls are: a query against state that already exists. You are not asking about each socket individually — you are asking one question about the whole set.

"Here is a set of descriptors and a timeout.
 Put me to sleep until at least one of them is ready.
 Then tell me which ones."

While nothing is ready, the thread sleeps inside the syscall at 0% CPU. When a socket changes state, the kernel wakes the waiting thread through that socket’s wait queue, so nothing is being polled in a loop. That is the difference between multiplexing and busy waiting: the wakeup is pushed to you, not discovered by you.

A useful mental picture is a night porter in a hotel. The porter cannot stand outside one guest’s door all night. Instead, every room has a light in the lobby. The porter naps in the lobby; when any light turns on, the porter wakes, checks which lights are lit, and serves those rooms. select is a porter who walks down a long corridor of light switches every time. epoll is a porter with a switchboard that lights up only the active rooms.

File descriptors are just integers

This trips up beginners because the API looks like it is passing around pointers and bitmasks. Underneath, a socket is an int. A typical process looks like this:

fd	what it is
0	standard input
1	standard output
2	standard error
3	the listening socket
4	a connected client socket
5	another connected client socket
6	… and so on
7	the epoll instance itself

Two consequences follow. First, the fd number means nothing by itself; it is an index into a table the kernel owns for your process. Second, the number of open fds is a resource with a limit. Check yours with ulimit -n. Servers usually raise it well above the default 1024, because every connection costs one fd.

select(): the original multiplexer

select has been around since the 1980s. It takes three sets of fds — one for reading, one for writing, one for exceptions — plus an optional timeout.

The sets are bitmasks, not arrays. A fd_set is a fixed-size block of bits: bit n is set if you care about fd n.

fd_set readfds;
FD_ZERO(&readfds);              /* clear every bit            */
FD_SET(listen_fd, &readfds);    /* bit 3 = 1: watch for input */
FD_SET(client_fd, &readfds);    /* bit 4 = 1                  */

int maxfd = client_fd;          /* select needs the largest fd */
int n = select(maxfd + 1, &readfds, NULL, NULL, NULL);

if (n > 0 && FD_ISSET(client_fd, &readfds)) {
    /* read() will not block (right now) */
}

Two details in that snippet are easy to miss. maxfd + 1 is not a typo: select needs to know how many bits to scan, so you pass the highest-numbered fd plus one. And select modifies the sets in place: on return, each bit tells you whether that fd is ready, which means you must rebuild the sets before the next call.

The timeout is a struct timeval, so it can express microseconds. On Linux it is updated to the time remaining, which is a classic source of bugs when you reuse it in a loop.

The 1024 problem

FD_SETSIZE is 1024 by default. If fd 1100 is a valid socket, select simply cannot watch it — the bitmask has no room. You can recompile with a larger FD_SETSIZE, but that does not change the fact that the kernel copies a fixed-size bitmap in and out, and the cost grows with the highest fd, not the number of fds you actually care about.

There is a second, subtler cost. Every call copies the whole bitmap from user space into the kernel and back. Then the kernel scans every bit to find the ones that are set. If you watch 10,000 connections and three are ready, you paid for 10,000.

inside one select() call

fd 0 fd 1 fd 2 fd 3 fd 4 fd 5 fd 6 fd 7

The kernel asks each bit in turn: ready? — eight checks even though only fd 4 has data. Then it repeats the whole scan on the next call.

select is still useful when portability matters most — it works on Linux, macOS, the BSDs, and Windows (as select in Winsock). For a handful of fds, it is perfectly fine. For a server, it is not the tool you want.

poll(): select without the bitmask wall

poll keeps the same “wait on a set, get a subset back” model but replaces the bitmap with an array of structs:

struct pollfd fds[2] = {
    { .fd = listen_fd, .events = POLLIN },
    { .fd = client_fd, .events = POLLIN },
};

int n = poll(fds, 2, -1);          /* -1: wait forever */

if (fds[1].revents & POLLIN) {
    /* client_fd is readable */
}

The array solves two annoyances at once. There is no 1024 limit, and there is no need to encode fd numbers as bit positions: each entry carries its own fd, the events you want (events), and the events that actually happened (revents). Because input and output are separate fields, poll does not destroy your request the way select does — though you still rebuild or reset the array between calls.

The fundamental cost did not change. The kernel still must look at every entry, and the whole array is still copied in and out on every call. If 5,000 fds are idle, poll scans 5,000 entries to discover that. So poll scales to more connections than select, but its complexity is still O(n) per call.

A few practical notes:

• The timeout is in milliseconds, not microseconds.
• POLLIN means readable, POLLOUT means writable. POLLERR and POLLHUP are reported whether or not you asked for them, so always check them.
• A return value of 0 means the timeout expired; -1 means an error such as EINTR.
• If a single fd appears multiple times in the array, all matching entries get the same revents. Don’t do that.

Both select and poll answer the question “who is ready?” correctly. The difference between them and epoll is how much work the kernel does to answer it.

epoll: register once, wait many times

The key observation behind epoll is that most of the work in select and poll is repeated. You register the same 10,000 sockets, the kernel scans the same 10,000 entries, and nothing has changed since the last call except which three sockets are ready.

epoll splits the API into two phases:

1. Registration. Tell the kernel once which fds you care about. The kernel keeps that set.
2. Waiting. Ask for the ready subset. The kernel maintains that list as events happen.

There are three calls:

syscall	what it does
epoll_create1(0)	Creates an epoll instance and returns an fd for it.
epoll_ctl(ep, op, fd, &event)	Adds, modifies, or removes an fd in the interest list. op is EPOLL_CTL_ADD, EPOLL_CTL_MOD, or EPOLL_CTL_DEL.
epoll_wait(ep, events, maxevents, timeout)	Sleeps until at least one fd is ready (or the timeout expires), then fills the events array with ready entries.

Inside the kernel, the epoll instance has two data structures that matter:

• An interest list of all registered fds. It is kept in a red-black tree so that adding, removing, and looking up an fd is O(log n). Each entry holds a callback and the data value you attached (data.fd or a pointer).
• A ready list of fds that currently have events. When a packet arrives, the socket’s wait queue runs the callback registered by epoll, which appends that fd to the ready list.

epoll_wait does not walk the interest list to find ready fds. It pops entries from the ready list. The cost is proportional to the number of ready events, not the number of registered fds. That is the whole magic trick — not a faster scan, but no scan at all.

What “O(1)” really means here

You will see epoll described as O(1). That is a useful shorthand, but be precise:

• epoll_ctl(ADD/DEL) is O(log n) because of the tree.
• epoll_wait costs O(k), where k is the number of events returned, plus the cost of putting the thread to sleep and waking it.
• The size of the interest list does not directly add to epoll_wait’s cost.

So with 10,000 idle connections and 3 ready ones, select scans 10,000 bits while epoll returns 3 entries. As connections grow, the gap turns into a different order of magnitude.

A small detail: what is in data?

Every registration carries a struct epoll_event:

struct epoll_event ev;
ev.events = EPOLLIN;
ev.data.fd = client_fd;      /* or ev.data.ptr = my_connection; */

The kernel does not interpret data. It hands the same value back in epoll_wait. Most servers store a pointer to a per-connection struct so they can find buffers, state, and timers without a lookup table. If you use data.fd, keep in mind that an fd number can be reused after close() — stale events are possible if you are careless, which is one reason pointer-based state is common.

Level-triggered vs. edge-triggered

epoll has two notification styles, and this is where many beginners get stuck.

Level-triggered (LT) is the default. As long as the condition is true, epoll_wait keeps reporting it. If 2 KB are sitting in the socket buffer and you read only 1 KB, the next epoll_wait says “fd 4 is readable” again. It is forgiving: forget to drain the socket, and you get another chance.

Edge-triggered (ET) reports the transition. You are told once when the socket becomes readable. If you do not drain it completely, you may never be told again, even though data is still sitting there. You must loop on read() until it returns EAGAIN.

Level-triggered default

data arrives wake wake wake drained

Edge-triggered EPOLLET

data arrives wake once silence — you must drain it yourself

Here is the shape of a correct edge-triggered read:

for (;;) {
    ssize_t n = read(fd, buf, sizeof buf);
    if (n > 0) {
        handle(buf, n);
        continue;                 /* keep going: there may be more */
    }
    if (n == 0) {
        close_connection(fd);     /* peer closed */
        break;
    }
    if (errno == EAGAIN || errno == EWOULDBLOCK) {
        break;                    /* buffer is empty: done for now */
    }
    /* real error */
    close_connection(fd);
    break;
}

Two related flags are worth knowing:

• EPOLLONESHOT disables the fd after one event. You must re-arm it with EPOLL_CTL_MOD. It is useful when multiple threads share one epoll instance and you want exactly one thread to handle a given connection at a time.
• EPOLLEXCLUSIVE is for the listening socket when several processes call epoll_wait on the same one. It wakes only one waiter instead of all of them — the “thundering herd” problem.

My advice for beginners: start level-triggered. It is harder to break, and the extra wakeups are rarely your bottleneck. Move to edge-triggered when you have measurements and a drain loop you trust.

The event loop, in one pseudocode block

Every multiplexed server — nginx, Redis, Node.js, a Go netpoller, your own — is a loop around the same skeleton:

create a listener
register the listener for "readable"

forever:
    ready = wait_for_ready_fds()        # blocks here when nothing is happening

    for each fd in ready:
        if fd is the listener:
            accept every pending connection
            register each new connection for "readable"
        else:
            read as much as is safe
            handle the bytes
            maybe write a response

That is it. The loop is single-threaded in the simplest form, and that is a feature: no locks around connection state, no context switches between handlers, and a natural place to implement timeouts and fairness. The hard part is not the loop; it is making sure nothing inside the loop blocks.

If one handler calls a blocking database driver, reads a slow disk, or waits on a mutex, the entire server freezes. One slow operation stalls all 10,000 connections. This is why “never block the event loop” is the first rule of every event-driven runtime.

Try it: be the kernel

The widget below simulates one iteration of an event loop. Click a socket to pretend data arrived, then run epoll_wait(). Notice that the call returns only the sockets that are actually ready — and that with nothing ready, the thread sleeps instead of spinning.

A real kernel does not “scan the four buttons” either. Each socket has a callback registered in the epoll instance; the callback adds the socket to the ready list when data arrives. The simulator shows the result, which is what matters for building the mental model.

A complete epoll server you can run

Theory is cheap. Here is a working TCP echo server in just over 100 lines of C. It accepts connections, reads whatever arrives, and writes it back. It is deliberately small, but it demonstrates the real shape of an epoll server.

#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <netinet/in.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <sys/epoll.h>
#include <sys/socket.h>
#include <unistd.h>

#define PORT 8080
#define MAX_EVENTS 64

static int make_nonblocking(int fd) {
    int flags = fcntl(fd, F_GETFL, 0);
    return flags < 0 ? -1 : fcntl(fd, F_SETFL, flags | O_NONBLOCK);
}

static int create_listener(void) {
    int fd = socket(AF_INET, SOCK_STREAM, 0);
    if (fd < 0) return -1;

    int one = 1;
    setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof one);

    struct sockaddr_in addr = {0};
    addr.sin_family = AF_INET;
    addr.sin_addr.s_addr = htonl(INADDR_ANY);
    addr.sin_port = htons(PORT);

    if (bind(fd, (struct sockaddr *)&addr, sizeof addr) < 0) return -1;
    if (listen(fd, SOMAXCONN) < 0) return -1;
    return fd;
}

int main(void) {
    signal(SIGPIPE, SIG_IGN);

    int lfd = create_listener();
    if (lfd < 0 || make_nonblocking(lfd) < 0) {
        perror("listener");
        return 1;
    }

    int ep = epoll_create1(0);
    if (ep < 0) {
        perror("epoll_create1");
        return 1;
    }

    struct epoll_event ev = { .events = EPOLLIN, .data.fd = lfd };
    if (epoll_ctl(ep, EPOLL_CTL_ADD, lfd, &ev) < 0) {
        perror("epoll_ctl");
        return 1;
    }

    struct epoll_event events[MAX_EVENTS];

    for (;;) {
        int n = epoll_wait(ep, events, MAX_EVENTS, -1);
        if (n < 0) {
            if (errno == EINTR) continue;   /* a signal interrupted us */
            perror("epoll_wait");
            return 1;
        }

        for (int i = 0; i < n; i++) {
            int fd = events[i].data.fd;

            if (fd == lfd) {
                /* One event can mean a whole queue of new connections. */
                for (;;) {
                    int cfd = accept4(lfd, NULL, NULL, SOCK_NONBLOCK);
                    if (cfd < 0) {
                        if (errno == EAGAIN || errno == EWOULDBLOCK) break;
                        perror("accept4");
                        break;
                    }

                    struct epoll_event cev = { .events = EPOLLIN, .data.fd = cfd };
                    if (epoll_ctl(ep, EPOLL_CTL_ADD, cfd, &cev) < 0) {
                        close(cfd);
                    }
                }
                continue;
            }

            char buf[4096];
            ssize_t nread = read(fd, buf, sizeof buf);

            if (nread == 0) {                    /* peer closed the connection */
                epoll_ctl(ep, EPOLL_CTL_DEL, fd, NULL);
                close(fd);
                continue;
            }

            if (nread < 0) {
                if (errno == EAGAIN || errno == EWOULDBLOCK) continue;
                epoll_ctl(ep, EPOLL_CTL_DEL, fd, NULL);
                close(fd);
                continue;
            }

            /* Echo. A real server would buffer partial writes; see below. */
            ssize_t nwritten = write(fd, buf, (size_t)nread);
            if (nwritten < 0 && errno != EAGAIN && errno != EWOULDBLOCK) {
                epoll_ctl(ep, EPOLL_CTL_DEL, fd, NULL);
                close(fd);
            }
        }
    }
}

Build and test it:

cc -O2 -Wall -Wextra server.c -o server
./server &
printf 'hello over epoll\n' | nc -q1 127.0.0.1 8080

You should see the line echoed back. If you want to watch the syscalls, run it under strace:

strace -f -e trace=epoll_create1,epoll_ctl,epoll_wait,accept4,read,write ./server

You will see epoll_wait sitting there, then returning exactly one event when the connection arrives.

What each part is doing

• make_nonblocking on the listener. After epoll_wait says the listener is readable, there might be one connection or fifty. The accept loop keeps accepting until EAGAIN, because each accept can return immediately. A non-blocking listener makes that loop safe.
• accept4(..., SOCK_NONBLOCK). New client sockets are non-blocking from birth. This saves a separate fcntl and, more importantly, prevents a later read from blocking the loop.
• One epoll_ctl(ADD) per client. The connection is now in the interest list. When bytes arrive, the kernel adds it to the ready list.
• read returning 0. That is not an error; it is an orderly shutdown. Remove the fd from epoll and close it. Closing alone usually removes it, but being explicit avoids surprises with duplicated fds.
• EAGAIN / EWOULDBLOCK. On a non-blocking socket, this means “nothing available right now”. It is normal, not an error.
• EPOLL_CTL_DEL before close. Not strictly required, but it keeps the intent clear and matters if the fd is shared.
• SIGPIPE ignored. Writing to a socket whose peer has closed raises SIGPIPE by default, which terminates the process. A server should ignore it and handle the EPIPE error instead.

This server reads once per event. With level-triggered epoll that is correct, because if more data remains, the next epoll_wait reports the fd again. With edge-triggered epoll, the same code is a bug: you would leave bytes in the buffer and never be told again. That is exactly why edge-triggered mode needs the drain loop shown earlier.

The same idea in Python

If C feels like too much ceremony, Python’s selectors module exposes the same model with a portable wrapper. On Linux, DefaultSelector uses epoll; on macOS it uses kqueue.

import selectors
import socket

sel = selectors.DefaultSelector()

srv = socket.socket()
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("0.0.0.0", 8080))
srv.listen()
srv.setblocking(False)
sel.register(srv, selectors.EVENT_READ, data=None)   # None marks the listener

while True:
    for key, mask in sel.select():          # blocks until >= 1 fd is ready
        if key.data is None:                # the listening socket
            conn, _ = srv.accept()
            conn.setblocking(False)
            sel.register(conn, selectors.EVENT_READ, data=b"")
        else:                               # a client socket
            conn = key.fileobj
            chunk = conn.recv(4096)
            if chunk:
                conn.send(chunk)            # one write; see the pitfalls
            else:
                sel.unregister(conn)
                conn.close()

The structure is identical: register, wait, dispatch. The difference is that Python hides the bitmasks, the arrays, and the platform-specific calls behind a uniform interface.

The pitfalls that actually bite

Multiplexing is conceptually simple, but the bugs are not. These are the ones I see most often.

1. Assuming one read equals one message

TCP is a byte stream, not a message stream. One read() might return half a JSON object, three messages glued together, or one byte. The kernel does not preserve your write() boundaries. If your protocol needs messages, add framing: a length prefix, newline-delimited JSON, or a fixed-size header.

This is not an epoll bug, but it shows up the moment you write your first event-driven server, so it belongs here.

2. Partial writes

write() can accept fewer bytes than you gave it, especially on a non-blocking socket whose send buffer is full. The sample server ignores this on purpose to stay small; a real server must keep an output buffer and register EPOLLOUT until the buffer drains. The usual pattern is:

• try to write immediately;
• if EAGAIN, append the remainder to a per-connection buffer and enable EPOLLOUT;
• on EPOLLOUT, flush the buffer, then disable EPOLLOUT.

Forgetting this produces truncated responses that are maddening to debug because they only appear under load.

3. Blocking inside the loop

One blocking call stalls every connection. The usual suspects:

• a blocking DNS lookup (getaddrinfo) — use an async resolver or a thread pool;
• a blocking database driver — use its async mode or a worker pool;
• reading a regular file — epoll does not help here (see below);
• sleep(), waiting on a mutex, or a synchronous HTTP call to another service.

The event loop is a single resource. Treat every handler as if it has a 1 ms budget.

4. Edge-triggered without a drain loop

If you set EPOLLET and read once, leftover bytes stay in the buffer and no new edge occurs. The connection hangs while looking perfectly healthy. Either drain until EAGAIN, or stay level-triggered until you are confident.

5. Forgetting EPOLLERR and EPOLLHUP

These are always reported, even if you did not request them. If you ignore them, a dead connection can spin in your loop. Always handle them, and pair them with EPOLLRDHUP if you want to detect half-closes promptly.

6. Ignoring SIGPIPE

On Linux, writing to a closed socket kills your process by default. Ignore SIGPIPE at startup (signal(SIGPIPE, SIG_IGN)) or use send(..., MSG_NOSIGNAL). Then handle EPIPE as a normal disconnect.

7. Running out of file descriptors

Each connection costs an fd. When accept4 returns EMFILE, your process is out of descriptors and cannot accept anything. A common trick is to keep a spare fd open: on EMFILE, close the spare, accept the pending connection, close it immediately, and reopen the spare. That buys you a clean rejection instead of a stuck accept queue. Also raise ulimit -n and monitor it.

8. Starving other connections

If a handler loops over a fast sender until EAGAIN, one connection can occupy the loop for a long time. Bound the work per event — for example, read at most 64 KB per callback, then return to epoll_wait. Fairness matters more than raw per-connection speed.

9. Expecting epoll to help with regular files

Regular files are always “ready”: read() on a local file rarely blocks in the way a socket does. epoll on a regular file is not useful, and on some filesystems it returns EPERM. For disk I/O, use a thread pool, io_uring, or asynchronous file APIs. epoll is a network/pipe/terminal tool.

10. Assuming readiness is a guarantee

Readiness means “a read would not block right now”, not “your read will succeed”. The peer can close, another thread can consume the data, or an error can surface. Always check the result of the actual read or write. Also be ready for EINTR; signals happen.

11. The thundering herd

Several processes or threads waiting on the same listening socket can all wake for one connection, and all but one fail with EAGAIN. It is wasted work. EPOLLEXCLUSIVE (Linux) or SO_REUSEPORT with one listener per worker are the usual fixes. This is an optimization, not something to worry about on day one.

Which one should you use?

	select	poll	epoll
Storage	fixed bitmask	array of structs	kernel interest list + ready list
Max fds	FD_SETSIZE (1024 by default)	no fixed limit	no fixed limit
Cost per wait	O(n) scan + copy	O(n) scan + copy	O(k) ready events
Rebuild set each call	yes	yes (reset revents)	no
Edge-triggered	no	no	yes (EPOLLET)
Portability	POSIX + Windows	POSIX	Linux only

A few close relatives are worth knowing by name:

• kqueue (FreeBSD, macOS, NetBSD) — the BSD equivalent of epoll, with a richer filter model. If you write portable event-driven code, you end up wrapping epoll and kqueue.
• IOCP (Windows) — completion-based, not readiness-based. You post buffers and get completion notifications.
• io_uring (Linux 5.1+) — completion-based and works for both network and disk I/O. It is the modern answer when you need to go beyond readiness multiplexing, but it is a bigger conceptual step.

The practical rule of thumb:

• A handful of fds, or you need Windows portability: select or poll.
• Many idle connections on Linux: epoll (level-triggered first).
• You need disk I/O or very high throughput with fewer syscalls: look at io_uring.
• You are writing application code: use a runtime that already did this for you — libuv, libevent, Tokio, Go’s netpoller, nginx, Redis.

The mental model to keep

If you remember one picture, make it this one:

1. A socket is a buffer plus a readiness flag. The kernel owns both.
2. Blocking read sleeps on one buffer. Great for one connection, useless for many in one thread.
3. select and poll ask the kernel to scan a set and return the ready subset. Correct, but O(n) every time.
4. epoll registers once and keeps a ready list. Waiting costs proportional to what is ready, not to what exists.
5. Multiplexing is readiness, not completion. You still do the read and write, with all their partial-result sharp edges.
6. The event loop must never block. One slow handler stops everything.

That is the whole idea. The rest is engineering: buffers, framing, timeouts, backpressure, fairness, and the humility to measure instead of guessing.

Further reading

• man 2 select, man 2 poll, man 7 epoll — the primary sources, and better than most blog posts.
• The C10K problem — Dan Kegel’s classic write-up of why thousands of connections broke the thread-per-connection model.
• Beej’s Guide to Network Programming — the friendliest introduction to sockets.
• libuv design overview — how a production event loop is actually structured.
• io_uring — Jens Axboe’s paper, for when readiness is not enough.