[iolib-devel] Bug when using READ-LINE on non-blocking fds
Hi, Iolib behaves in the following way when using READ-LINE with a non-blocking fd: ;; *client-stream is the stream associated with a client connected to our server IRCD> *client-stream #<active IPv6 stream socket connected to ::ffff:127.0.0.1/41460 {1004D5D451}> IRCD> (fd-non-blocking *) T ;; client sends the string "foo" down the socket, no newline is sent IRCD> (ignore-errors (read-line *client-stream)) NIL #<System-Error EWOULDBLOCK(11) "Resource temporarily unavailable"> ;; client sends the string "bar", followed by a newline IRCD> (ignore-errors (read-line *client-stream)) "bar" NIL So, essentially, the string "foo" is lost. Shouldn't the correct behaviour be to wait till a newline arrives on the socket? Chaitanya
On Sun, 2009-02-08 at 00:48 +0530, Chaitanya Gupta wrote:
Hi,
Iolib behaves in the following way when using READ-LINE with a non-blocking fd:
;; *client-stream is the stream associated with a client connected to our server
IRCD> *client-stream #<active IPv6 stream socket connected to ::ffff:127.0.0.1/41460 {1004D5D451}>
IRCD> (fd-non-blocking *) T
;; client sends the string "foo" down the socket, no newline is sent
IRCD> (ignore-errors (read-line *client-stream)) NIL #<System-Error EWOULDBLOCK(11) "Resource temporarily unavailable">
;; client sends the string "bar", followed by a newline
IRCD> (ignore-errors (read-line *client-stream)) "bar" NIL
So, essentially, the string "foo" is lost. Shouldn't the correct behaviour be to wait till a newline arrives on the socket?
That's correct. Thanks for the notice, I've just pushed a fix ;) -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur.
Stelian Ionescu wrote:
That's correct. Thanks for the notice, I've just pushed a fix ;)
Cool! Thanks for the quick fix. I wanted to point out that even READ-SEQUENCE had similar problems, but that too seems to have been fixed i.e. READ-SEQUENCE now correctly waits till the required amount of data has been sent by the client. On that note, does iolib provide any function to read a chunk of data without blocking i.e. something like READ-CHAR-NO-HANG for sequences? (or, something akin to what read(2) provides when O_NONBLOCK is set). Chaitanya
On Sun, 2009-02-08 at 01:59 +0530, Chaitanya Gupta wrote:
Stelian Ionescu wrote:
That's correct. Thanks for the notice, I've just pushed a fix ;)
Cool! Thanks for the quick fix. I wanted to point out that even READ-SEQUENCE had similar problems, but that too seems to have been fixed i.e. READ-SEQUENCE now correctly waits till the required amount of data has been sent by the client.
On that note, does iolib provide any function to read a chunk of data without blocking i.e. something like READ-CHAR-NO-HANG for sequences? (or, something akin to what read(2) provides when O_NONBLOCK is set).
No. The current iolib streams are gray-streams therefore are blocking(because CL streams are designed that way). The other stream layer I'm working on(zeta-streams) should provide that, but it's not finished yet. -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur.
On Sun, 08 Feb 2009 01:59:56 +0530 Chaitanya Gupta <mail@chaitanyagupta.com> wrote:
On that note, does iolib provide any function to read a chunk of data without blocking i.e. something like READ-CHAR-NO-HANG for sequences? (or, something akin to what read(2) provides when O_NONBLOCK is set).
It is possible to use RECEIVE-FROM which would signal IOLIB.SYSCALLS:EWOULDBLOCK if the socket is non-blocking and no remaining data is ready to read, or IOLIB.SYSCALLS:EINTR on reception of a signal, but I'm not sure if this could work with buffering, or if it would conflict with stream operations (i.e. in C you wouldn't use stdio fread()/fgets() and read() concurrently). I've used RECEIVE-FROM without issues when creating sockets with 0 bytes receive and send buffers and using custom buffers. An example may be found at http://cvs.pulsar-zone.net/cgi-bin/cvsweb.cgi/mmondor/mmsoftware/cl/lib/rw-q... (Note that READ-LINE-FROM-QUEUE and WRITE-STRING-TO-QUEUE are SBCL-specific as they use SB-EXT to convert between UTF-8 strings and 8-bit bytes representations). -- Matt
On Sun, 2009-02-08 at 02:16 -0500, Matthew Mondor wrote:
On Sun, 08 Feb 2009 01:59:56 +0530 Chaitanya Gupta <mail@chaitanyagupta.com> wrote:
On that note, does iolib provide any function to read a chunk of data without blocking i.e. something like READ-CHAR-NO-HANG for sequences? (or, something akin to what read(2) provides when O_NONBLOCK is set).
It is possible to use RECEIVE-FROM which would signal IOLIB.SYSCALLS:EWOULDBLOCK if the socket is non-blocking and no remaining data is ready to read, or IOLIB.SYSCALLS:EINTR on reception of a signal, but I'm not sure if this could work with buffering, or if it would conflict with stream operations (i.e. in C you wouldn't use stdio fread()/fgets() and read() concurrently).
It's possible: you can use drain-input-buffer to copy data from the buffers(it returns two values: the offset of the first element of the sequence that wasn't overwritten, and the number of octets remaining in the buffer or NIL if there aren't any left). After that you can use receive-from. Technically, you can use it at any moment, but since there may be data left in the stream buffer, you lose ordering of the stream data)
I've used RECEIVE-FROM without issues when creating sockets with 0 bytes receive and send buffers and using custom buffers.
About receive-from, I should probably make it clearer in the documentation that you must either specify (buffer,start,end) or size: if you specify size and not buffer, a new array is consed and the data is copied there.
An example may be found at http://cvs.pulsar-zone.net/cgi-bin/cvsweb.cgi/mmondor/mmsoftware/cl/lib/rw-q... (Note that READ-LINE-FROM-QUEUE and WRITE-STRING-TO-QUEUE are SBCL-specific as they use SB-EXT to convert between UTF-8 strings and 8-bit bytes representations).
Since you use iolib, you might as well use babel's o-t-s and s-t-o functions. -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur.
On Sun, 08 Feb 2009 14:09:08 +0100 Stelian Ionescu <stelian.ionescu-zeus@poste.it> wrote:
It's possible: you can use drain-input-buffer to copy data from the buffers(it returns two values: the offset of the first element of the sequence that wasn't overwritten, and the number of octets remaining in the buffer or NIL if there aren't any left). After that you can use receive-from. Technically, you can use it at any moment, but since there may be data left in the stream buffer, you lose ordering of the stream data)
Indeed
Since you use iolib, you might as well use babel's o-t-s and s-t-o functions.
Thanks for the suggestion, I tried it and verified if performance was affected but it doesn't seem to be (at least not noticeably). -- Matt
participants (3)
-
Chaitanya Gupta -
Matthew Mondor -
Stelian Ionescu