On Sun, Jun 17, 2007 at 9:48 AM, Douglas Crosher dtc@scieneer.com wrote:
Hello Erik,
Changing the wait-for-input-internal function to set a flag in the socket objects to indicate they are ready would avoid the need to cons up a list of ready sockets which can become very inefficient when managing a large number of connections.
That's true. On the other hand, the efficiency of needing to walk all provided socket objects to find out which ones changed seems to be the main objection against the select() system call (which is why poll(), epoll() and others have been invented, right?).
The lack of performance probably shows that the 'wait-for-input-internal interface is too high-level - it implicitly defines a socket set as a list and this list needs to be consed when calling and on return and scanning the list slow. In contrast a CL implementation with an interface to poll() and select() can implement non-consing socket sets that have higher performance. The poll() function has an advantage when waiting on only a small subset of the open sockets and select() may have an advantage when waiting on a large set of sockets because the bit sets can be scanned in word sized chunks.
Well, indeed did the interface meet some problems. That's why I've now introduced the concept of a wait-list, something like a poll-set or Java channel-selector. The wait-list (which is a structure btw) gets set up once and reused many times. Also, the sockets now have a STATE slot where the fact of read-readyness is stored.
I think I succesfully updated all backends, except for the Scieneer backend. Could you have a look at how to efficiently use this new wait-list structure? The %WAIT slot is free for use by the backend, the WAITERS slot contains a list of all waiting sockets.
Perhaps a portable interface to the select() and poll() functions would be better.
Unfortunately, that seems impossible with the bounds I've imposed on the project (for now): I'd hate to have to depend on CFFI because that's quite a strong requirement (when looking at Windows). [Not all LispWorks developers have a C compiler installed.]
Anyway, if you could have a look, I would be very greatful. The code is at svn://common-lisp.net/project/usocket/svn/usocket/branches/new-wfi.
Thanks in advance!
Bye,
Erik.