Hello,
I'm *very* new to Common Lisp and have only just started understanding how to use packages and asdf. I have a client/server application in mind, of which I've already written a fair amount (using the asdf module system), and I would like to use iolib to multiplex the i/o.
I'm using sbcl 1.0.18.debian on an ubuntu 9.04 x86 box.
I initially used asdf to set up iolib and it picked the iolib-0.6.0 package and downloaded/compiled a bunch of denpendencies. There were some problems compiling it concerning its dependencies, which I'm sorry to say I did not record or what I did to get around them--but it had to do with cffi-grovel not being a package asdf new about, and I believe I had to install cffi by hand before iolib would finish compiling becaue the grovel module was now part of the cffi.
So, I started looking around for an example for how to use iolib, and found one online in the current sources for iolib called echo-server.lisp. It took a while for me to realize that net.sockets in the iolib-0.6.0 version was moved to iolib.sockets in the current sources, and after noticing that some other API changes happened and whatnot, I removed the iolib-0.6.0 site and system from my ~/.sbcl/{site,system} directory, removed the asd file symlinks, and git checked out the head of iolib.
I put that checkout into the above mentioned place, hand set up the asd symlinks, and tried (require 'iolib) again.
This is what I got:
* (require 'iolib)
... compile lines ...
debugger invoked on a SB-PCL::INITARG-ERROR in thread #<THREAD "initial thread" RUNNING {A834819}>: Invalid initialization argument: :SONAME in call for class #<STANDARD-CLASS CFFI-GROVEL:WRAPPER-FILE>. See also: The ANSI Standard, Section 7.1.2
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name): 0: [ABORT] Exit debugger, returning to top level.
(SB-PCL::CHECK-RI-INITARGS #<CFFI-GROVEL:WRAPPER-FILE {AC30651}> (:NAME "ffi-wrappers" :PATHNAME "ffi-wrappers-unix" :PARENT #<ASDF:SYSTEM "iolib.syscalls" {AC27D91}> :SONAME "libiolib-syscalls")) 0]
I don't know how to get around this error.
I apologize for my gross misunderstanding of how to do all of this. I'm in the process of figuring it out.
Thank you.
-pete
On Thu, 2009-12-24 at 11:55 -0600, Peter Keller wrote:
Hello,
I'm *very* new to Common Lisp and have only just started understanding how to use packages and asdf. I have a client/server application in mind, of which I've already written a fair amount (using the asdf module system), and I would like to use iolib to multiplex the i/o.
I'm using sbcl 1.0.18.debian on an ubuntu 9.04 x86 box.
sbcl 1.0.18 may be too old (17 months old ATM) so I'd advise you to try with a more recent version.
I initially used asdf to set up iolib and it picked the iolib-0.6.0 package and downloaded/compiled a bunch of denpendencies. There were some problems compiling it concerning its dependencies, which I'm sorry to say I did not record or what I did to get around them--but it had to do with cffi-grovel not being a package asdf new about, and I believe I had to install cffi by hand before iolib would finish compiling becaue the grovel module was now part of the cffi.
So, I started looking around for an example for how to use iolib, and found one online in the current sources for iolib called echo-server.lisp. It took a while for me to realize that net.sockets in the iolib-0.6.0 version was moved to iolib.sockets in the current sources, and after noticing that some other API changes happened and whatnot, I removed the iolib-0.6.0 site and system from my ~/.sbcl/{site,system} directory, removed the asd file symlinks, and git checked out the head of iolib.
No need to apologize. A new release is long overdue and the current HEAD needs CFFI HEAD(http://common-lisp.net/project/cffi/darcs/cffi/). I'll try to arrange a new release of CFFI and IOlib before the end of the vacation
Usually, whenever I switch from requiring a released version to the live version of some dependency, I update the download page (http://common-lisp.net/project/iolib/download.shtml)
If you need assistance with IOlib, you can also find me on irc.freenode.net, channel #iolib :)
On Thu, Dec 24, 2009 at 07:46:13PM +0100, Stelian Ionescu wrote:
sbcl 1.0.18 may be too old (17 months old ATM) so I'd advise you to try with a more recent version.
Another poster mentioned to me to use clbuild to get a newer copy of sbcl and the various supported packages. I used it, installed ALL of the supported packages, and it worked *almost* out of the box.
Oddly, enough, it failed to compile iolib :), but this time I figured out why.
Basically, you have some 8-bit ascii in the documentation string for the method in iolib/src/sockets/base-sockets.lisp:
(defgeneric socket-option (socket option-name) (:documentation "Returns the value(s) of OS option OPTION-NAME on SOCKET. For a complete list of supported options see <C2><AB>iolib.sockets/socket-options.lisp<C2><BB>."))
sbcl complained about an unreadable character in the stream and stopped. Once I removed the << >> marks from the documentation string, it worked and iolib compiled and loaded. I was easily able to get the library I am working on as well to set up and run.
If you need assistance with IOlib, you can also find me on irc.freenode.net, channel #iolib :)
Ok, my main interest is figuring out how in Common Lisp to write a nonblocking multplexed i/o server and the client it speaks to.
I have a project that I need to scale to tens of thousands of clients talking with a single server.
Thank you.
-pete
On Thu, 2009-12-24 at 14:41 -0600, Peter Keller wrote:
On Thu, Dec 24, 2009 at 07:46:13PM +0100, Stelian Ionescu wrote:
sbcl 1.0.18 may be too old (17 months old ATM) so I'd advise you to try with a more recent version.
Another poster mentioned to me to use clbuild to get a newer copy of sbcl and the various supported packages. I used it, installed ALL of the supported packages, and it worked *almost* out of the box.
Oddly, enough, it failed to compile iolib :), but this time I figured out why.
Basically, you have some 8-bit ascii in the documentation string for the method in iolib/src/sockets/base-sockets.lisp:
(defgeneric socket-option (socket option-name) (:documentation "Returns the value(s) of OS option OPTION-NAME on SOCKET. For a complete list of supported options see <C2><AB>iolib.sockets/socket-options.lisp<C2><BB>."))
That's UTF-8 for #\« and #\». You can make SBCL user UTF-8 by default: either use a UTF-8 locale or add (setf sb-impl::*default-external-format* :utf-8) to your ~/.sbclrc
On Thu, Dec 24, 2009 at 10:04:02PM +0100, Stelian Ionescu wrote:
That's UTF-8 for #\« and #\». You can make SBCL user UTF-8 by default: either use a UTF-8 locale or add (setf sb-impl::*default-external-format* :utf-8) to your ~/.sbclrc
This fixed seemed to work. Thank you.
After a long bout of experimenting with the very reasonable tool clbuild and some gnashing of teeth at Limp, the SLIME equivalent for vim, I now have my workflow back up with the newest sbcl.
So, after all that, now I have to check out to see if the echo-server.lisp program works. :)
Thank you.
-pete
On Thu, Dec 24, 2009 at 07:19:59PM -0600, Peter Keller wrote:
So, after all that, now I have to check out to see if the echo-server.lisp program works. :)
Ok, after some experimentation and writing my own code based off of the echo-server.lisp code, I've found a bug in the echo-server.lisp code:
In function make-listener-handler, it will timeout after a certain amount of time, this clause is supposed to fire:
(when (eql :timeout event) (warn "Got a server timeout!") (return))
However, it won't because the (eql :timeout event) should be (eql :timeout exception) instead.
The current behavior of the bug is after 15 seconds, the server becomes completely unresponsive. By default this isn't seen because the timeout for the server itself is 10 seconds.
Also, unrelated, the (defvar *sockets* (make-hash-table)) at the top of the file should be (defvar *sockets* nil) instead.
Now, a question, is there a means by which I can have iomux:event-dispatch return control to me for a while at specified intervals, and then go back to multiplexing i/o? This would allow me to not have to put the server machinery into the read/write functions on the streams themselves or be beholden to my processing only on communication boundaries. If I can get a single threaded solution, then I wouldn't have to worry about the usual things with threads concerning data races and whatnot.
I'm looking forward to a new release of iolib, especially to get an updated API description on the online manual.
Thank you.
-pete
On Sat, 2009-12-26 at 14:05 -0600, Peter Keller wrote:
On Thu, Dec 24, 2009 at 07:19:59PM -0600, Peter Keller wrote:
So, after all that, now I have to check out to see if the echo-server.lisp program works. :)
Ok, after some experimentation and writing my own code based off of the echo-server.lisp code, I've found a bug in the echo-server.lisp code:
In function make-listener-handler, it will timeout after a certain amount of time, this clause is supposed to fire:
(when (eql :timeout event) (warn "Got a server timeout!") (return))However, it won't because the (eql :timeout event) should be (eql :timeout exception) instead.
The current behavior of the bug is after 15 seconds, the server becomes completely unresponsive. By default this isn't seen because the timeout for the server itself is 10 seconds.
Also, unrelated, the (defvar *sockets* (make-hash-table)) at the top of the file should be (defvar *sockets* nil) instead.
Thanks. I've updated the example
Now, a question, is there a means by which I can have iomux:event-dispatch return control to me for a while at specified intervals, and then go back to multiplexing i/o? This would allow me to not have to put the server machinery into the read/write functions on the streams themselves or be beholden to my processing only on communication boundaries. If I can get a single threaded solution, then I wouldn't have to worry about the usual things with threads concerning data races and whatnot.
(event-dispatch *base* :one-shot t)
Hello,
On Sat, Dec 26, 2009 at 10:36:48PM +0100, Stelian Ionescu wrote:
Now, a question, is there a means by which I can have iomux:event-dispatch return control to me for a while at specified intervals, and then go back to multiplexing i/o? This would allow me to not have to put the server machinery into the read/write functions on the streams themselves or be beholden to my processing only on communication boundaries. If I can get a single threaded solution, then I wouldn't have to worry about the usual things with threads concerning data races and whatnot.
(event-dispatch *base* :one-shot t)
Two questions, A. will returning from the event-dispatch function using the above close all connections in the mean time, and B. why is it bad architecture (at least it feels like it) to put my server's background work as the timeout code path of make-listener-handler and set the timeout to something like .01?
Also, I could be potentially sending megabytes to gigabytes of data back and forth between the clients and server, so I assume there a nonblocking interface to reading and writing the stream in an unsigned byte sequence oriented fashion once established?
Thank you.
-pete
-
Peter Keller -
Stelian Ionescu