[usocket-devel] How to tell size of data waiting in a TCP socket?
Hi, I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like (iter (for ready next (wait-for-input socket :timeout 0)) (while (read-message socket))) I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads. The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter. There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)? Thanks. -- Elliott Slaughter "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
I have another minimal testcase. It actually doesn't exhibit a freeze like I see in my application, but it's definitely not working, so I think it's worth looking at. The code: (defun receive-each (connections) (let ((ready (usocket:wait-for-input connections :timeout 0 :ready-only t))) (loop for connection in ready collect (read-line (usocket:socket-stream connection))))) (defun receive-all (connections) (loop for messages = (receive-each connections) then (receive-each connections) while messages append messages)) (defun send (connection message) (format (usocket:socket-stream connection) "~a~%" message) (force-output (usocket:socket-stream connection))) (defun server () (let* ((listen (usocket:socket-listen usocket:*wildcard-host* 12345)) (connection (usocket:socket-accept listen))) (loop for messages = (receive-all connection) then (receive-all connection) do (format t "Got messages:~%~s~%" messages) do (sleep 1/50)))) (defun client () (let ((connection (usocket:socket-connect "localhost" 12345))) (loop for i from 0 do (send connection (format nil "This is message ~a." i)) do (sleep 1/100)))) Here is what the output looks like on everything I've tested except SBCL on Windows. (Tested on SBCL on Mac, and Clozure CL on Windows.) * (server) Got messages: ("This is message 0." "This is message 1." "This is message 2.") Got messages: ("This is message 3." "This is message 4.") ... Here is the output on SBCL on Windows. * (server) Got messages: NIL Got messages: NIL ... I think in this case wait-for-input never returns anything as ready so I never read from the socket. On Sat, Apr 30, 2011 at 1:13 AM, Elliott Slaughter < elliottslaughter@gmail.com> wrote:
Hi,
I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like
(iter (for ready next (wait-for-input socket :timeout 0)) (while (read-message socket)))
I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads.
The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter.
There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)?
Thanks.
-- Elliott Slaughter
"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
-- Elliott Slaughter "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
Hi, Elliott Sorry again for late response, but I think I've found the reason about this issue: WAIT-FOR-INPUT correctly detected that there're inputs on the server side socket, but due to a subtle bug when updating the STATE slot of the socket object, the socket wasn't marked as READY, so when W-F-I was called by (READY-ONLY T), a empty list returned (and your code have no chance to get touch with the stream behind the socket, and nothing could be output ...) Due to some problems on USOCKET SVN, I cannot commit my work right now, but I have a patch as .diff file in attach, which could solve all your needs. I hope you can take a little download it and apply to your local USOCKET directory and see if your application work well this time. The bug you reported also affect ECL on Windows, but I don't sync my patch to ECL until I get your positive confirmation. Regards, Chun Tian (binghe) 在 2011-5-1，11:44， Elliott Slaughter 写道：
I have another minimal testcase. It actually doesn't exhibit a freeze like I see in my application, but it's definitely not working, so I think it's worth looking at.
The code:
(defun receive-each (connections) (let ((ready (usocket:wait-for-input connections :timeout 0 :ready-only t))) (loop for connection in ready collect (read-line (usocket:socket-stream connection)))))
(defun receive-all (connections) (loop for messages = (receive-each connections) then (receive-each connections) while messages append messages))
(defun send (connection message) (format (usocket:socket-stream connection) "~a~%" message) (force-output (usocket:socket-stream connection)))
(defun server () (let* ((listen (usocket:socket-listen usocket:*wildcard-host* 12345)) (connection (usocket:socket-accept listen))) (loop for messages = (receive-all connection) then (receive-all connection) do (format t "Got messages:~%~s~%" messages) do (sleep 1/50))))
(defun client () (let ((connection (usocket:socket-connect "localhost" 12345))) (loop for i from 0 do (send connection (format nil "This is message ~a." i)) do (sleep 1/100))))
Here is what the output looks like on everything I've tested except SBCL on Windows. (Tested on SBCL on Mac, and Clozure CL on Windows.)
* (server) Got messages: ("This is message 0." "This is message 1." "This is message 2.") Got messages: ("This is message 3." "This is message 4.") ...
Here is the output on SBCL on Windows.
* (server) Got messages: NIL Got messages: NIL ...
I think in this case wait-for-input never returns anything as ready so I never read from the socket.
On Sat, Apr 30, 2011 at 1:13 AM, Elliott Slaughter <elliottslaughter@gmail.com> wrote: Hi,
I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like
(iter (for ready next (wait-for-input socket :timeout 0)) (while (read-message socket)))
I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads.
The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter.
There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)?
Thanks.
-- Elliott Slaughter
"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
-- Elliott Slaughter
"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay _______________________________________________ usocket-devel mailing list usocket-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
2011/5/10 Chun Tian (binghe) <binghe.lisp@gmail.com>
Hi, Elliott
Sorry again for late response, but I think I've found the reason about this issue: WAIT-FOR-INPUT correctly detected that there're inputs on the server side socket, but due to a subtle bug when updating the STATE slot of the socket object, the socket wasn't marked as READY, so when W-F-I was called by (READY-ONLY T), a empty list returned (and your code have no chance to get touch with the stream behind the socket, and nothing could be output ...)
Due to some problems on USOCKET SVN, I cannot commit my work right now, but I have a patch as .diff file in attach, which could solve all your needs. I hope you can take a little download it and apply to your local USOCKET directory and see if your application work well this time.
The bug you reported also affect ECL on Windows, but I don't sync my patch to ECL until I get your positive confirmation.
Yes, it works! Thanks! Regards,
Chun Tian (binghe)
在 2011-5-1，11:44， Elliott Slaughter 写道：
I have another minimal testcase. It actually doesn't exhibit a freeze like I see in my application, but it's definitely not working, so I think it's worth looking at.
The code:
(defun receive-each (connections) (let ((ready (usocket:wait-for-input connections :timeout 0 :ready-only t))) (loop for connection in ready collect (read-line (usocket:socket-stream connection)))))
(defun receive-all (connections) (loop for messages = (receive-each connections) then (receive-each connections) while messages append messages))
(defun send (connection message) (format (usocket:socket-stream connection) "~a~%" message) (force-output (usocket:socket-stream connection)))
(defun server () (let* ((listen (usocket:socket-listen usocket:*wildcard-host* 12345)) (connection (usocket:socket-accept listen))) (loop for messages = (receive-all connection) then (receive-all connection) do (format t "Got messages:~%~s~%" messages) do (sleep 1/50))))
(defun client () (let ((connection (usocket:socket-connect "localhost" 12345))) (loop for i from 0 do (send connection (format nil "This is message ~a." i)) do (sleep 1/100))))
Here is what the output looks like on everything I've tested except SBCL on Windows. (Tested on SBCL on Mac, and Clozure CL on Windows.)
* (server) Got messages: ("This is message 0." "This is message 1." "This is message 2.") Got messages: ("This is message 3." "This is message 4.") ...
Here is the output on SBCL on Windows.
* (server) Got messages: NIL Got messages: NIL ...
I think in this case wait-for-input never returns anything as ready so I never read from the socket.
On Sat, Apr 30, 2011 at 1:13 AM, Elliott Slaughter < elliottslaughter@gmail.com> wrote:
Hi,
I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like
(iter (for ready next (wait-for-input socket :timeout 0)) (while (read-message socket)))
I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads.
The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter.
There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)?
Thanks.
-- Elliott Slaughter
"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
-- Elliott Slaughter
"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay _______________________________________________ usocket-devel mailing list usocket-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
-- Elliott Slaughter "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay
Hi again, Elliott To find "how much (bytes) data is available on a socket", you can use a internal function called USOCKET::BYTES-AVAILABLE-FOR-READ, the only argument is the STREAM-SOCKET object, and it returns the number of bytes. But if you want to know how many "messages" is available, I'm afraid that unless you know exactly the length of your messages, above function still cannot let you cut down the number of calls to WAIT-FOR-INPUT. And this function is currently only available on Windows for LispWorks, SBCL and ECL. Hope this helps. Regards, Chun Tian (binghe) 在 2011-4-30，16:13， Elliott Slaughter 写道：
Hi,
I'm wondering if there is a good way to figure out how much data is available on a socket. Right now I have a situation where my program accepts messages over a TCP socket. To check to see if a single message is available, I can just call wait-for-input with :timeout 0. But in order to check for multiple messages on a single socket, I need multiple calls to wait-for-input. So something like
(iter (for ready next (wait-for-input socket :timeout 0)) (while (read-message socket)))
I'm avoiding multithreading to preserve compatibility with Lisps that don't support (or have unstable support for) threads.
The above works on most implementations, but breaks on SBCL/Windows, where subsequent calls to wait-for-input appear to block, despite the :timeout 0 parameter.
There may very well be a bug in the SBCL/Windows implementation in usocket to chase down here. Alternatively, is there a better way to do the above (assuming that multithread is unavailable in the implementation I want to run this on)?
Thanks.
-- Elliott Slaughter
"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay _______________________________________________ usocket-devel mailing list usocket-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel
participants (2)
-
Chun Tian (binghe) -
Elliott Slaughter