Hi,
I was wondering how best to use IOLib to write an efficient HTTP server that
can handle perhaps 10,000+ simultaneous connections. It seems like iolib
has all the right ingredients: a sytem-level sockets interface and a io
multiplexer that uses epoll/kqueue for efficiently querying sockets. There
is quite a bit of code already written so I was hoping for some advice about
how this would be best implemented.
Here is a possible architecture for a server that can handle tons of
connections at once:
Some lisp thread originally sets up a passive-socket (server socket) to
listen for connections on some port. There are a few worker threads (on the
order of the number of processors in the machine). When a connection is
received, one of these worker threads will dequeue an active socket with
ACCEPT.
However, after the initial connection all the HTTP headers and content must
be read from the socket. Presumably not all the data will be ready as soon
as a connection is received, and read operations will block if allowed to
block. While it waits for the full HTTP request to come across the wire,
the worker thread could be accepting new connections or processing older
ones where the fully request is available. To quickly send HTTP responses
off, writes to sockets should also never block--so if we try to send more
bytes than a socket can handle, we should handle that asynchronously so the
worker can get on to the next thing to do.
So, a worker thread will either
1) be processing a request (arbitrary lisp code to respond appropriately to
an HTTP request). When a response is is ready, it should be written to some
non-blocking gray stream.
2) be waiting for the next of the following events:
a) socket writable. Some gray stream that was written to in (1) but
blocked now has enough room in its buffers to allow more data to be sent
immediately. (ie a "would block" message was received in a "send to" call).
b) socket readable. Some socket that we are listening to (gray stream)
has more data available. When this data is sufficient to respond to the
request, this connection is now elgible for the processing described in (1).
c) socket accepted. Some socket is available in the queue of the
passive socket. We can now begin listening to the socket's read events for
processing as in (2.b) or process the socket as in (1).
My question about IOLib is how this sort of processing model could be
implemented on top of iolib. Do passive sockets generate epoll/kqueue
events when a new connection is available to accept? If so it seems like
the multiplexer could be used to listen for events 2.a,2.b, and 2.c all
simultaneously.
I see there are some gray stream implementations in the code right now,
though I have not figured out how to use them. How do I, for example,
create a stream with an underlying socket? Could these sockets work with
the multiplexer implementation to accomplish the processing model described?
I think that sums it up. Thanks for the great library!
-Red
