usocket-devel

usocket-devel@common-lisp.net

248 discussions

[usocket-devel] Why rely on implementation specific network code?
by Hedos 05 Jan '07

05 Jan '07

Why not make a CFFI library based around the C sockets library? Wouldn't that be easier? That would avoid running into some implementation limitations too.

2 2

[usocket-devel] [ANNOUNCE] usocket 0.2.2
by Erik Huelsmann 03 Jan '07

03 Jan '07

Just a minute ago, I released usocket version 0.2.2, which: - Adds ECL support - Fixes an SBCL bug With now 9 Common Lisp implementations supported, I hope usocket proves to be making good on its promise of being a portability layer. If you use or plan to use usocket in your library or application, please send feedback to usocket-devel(a)common-lisp.net! Erik Huelsmann

1 0

[usocket-devel] GCL socket code on slime-devel
by Erik Huelsmann 02 Jan '07

02 Jan '07

Hi! I found a mail from you on slime-devel where you have started a slime backend which includes some socket code for GCL. I'd like to extend usocket (http://common-lisp.net/project/usocket/) to support GCL and you have already done 90% of that work, so, I'd like to have your permission and take the code (if it is yours to give) into usocket. What's the license you put on it? usocket is currently under MIT license. Does your contribution as linked to in the link below fall into the same category (public domain) as the rest of SLIME? http://common-lisp.net/pipermail/slime-devel/2005-April/003392.html Thanks in advance! bye, Erik.

1 0

[usocket-devel] trivial-usocket
by Erik Huelsmann 18 Dec '06

18 Dec '06

Ok. In order to officially supersede trivial-sockets, I created 'trivial-usocket': a migration path for trivial-sockets users. There's not much documentation yet, but I think the sources speak for themselves. If anyone of you would like to give it a try, or even send in patches for test-cases, I'd be most gratefull! Also: the library is mainly meant as a migration path: it depends on trivial-gray-streams to gain platform independent access to gray streams. Both Franz and Scieneer complain about the relative speed of Gray streams, so, presumably, it's not a good idea to move them into production of a high speed system, but for initial migration, it should do fine. Thanks for any review/comments! bye, Erik.

1 0

Re: [usocket-devel] Portably supporting :external-format
by Erik Huelsmann 23 Oct '06

23 Oct '06

> > So, to stay backward compatible, this is what I'd like to do: > > > > - If no external-format is given: just return the raw stream returned > > by the implementation (as we do now) > > - If an external-format is specified, create a raw stream with 'octet' > > element-type and wrap the returned stream with a flexi-stream, > > exposing the flexi-stream to the caller. > > > > How about that? > Flexi-streams depends on trivial-gray-streams, which appears to > support the following implementations: > > - Allegro > - CMUCL > - LispWorks > - OpenMCL > - CLISP > - SBCL > > It doesn't appear to support ABCL or Scieneer CL. For that reason, we > can't simply make it a mandatory dependency, unless we want to give up > on those implementations. How do you propose we make it optional? Well, there's another option: I talked to David Lichteblau and he will accept patches to support more implementations. I already had ABCL installed on my system, so I've been working to add ABCL support today. ABCL users probably won't want to use the facility, because (as the authors say themselves) "CLOS is dead slow" and Gray streams use CLOS... As far as Scieneer is concerned, well given their prices, I won't be able to afford to have an installation, but maybe Douglas Crosher (who I CC'ed in with this message) can provide a patch to trivial-gray-streams too? > One idea that comes to mind is to only use flexi-streams if it has > already been loaded when package.lisp is loaded. If it has, we set a > *use-flexi-streams* variable (or something), and then test that > whenever we need to deal with :external-format. In addition, we could > expose this variable to the user, allowing them to test or set it. Well, I'd like to check and see if it's possible to extend trivial-gray-streams (and possibly flexi-streams) to support our use-case. Looking at ABCL and flexi-streams though, I do see a problem: ABCL doesn't have a generic function close, but defines STREAM-CLOSE instead... At least flexi-streams 0.5.10 seems to depend on close, open-stream-p and maybe others too, to be generic functions, not just defuns... Well, ABCL isn't in wide use anyway and not going very fast development, but, well, I could ask around with the developers what to expect there. It would indeed be too bad not to be able to support :external-format there, but that's an option too (just ignore the external-format in case of ABCL). We need to make sure to document that though. bye, Erik. PS: Douglas, do you have interface documentation for Scieneer Gray streams somewhere for me? Or could you have a look at trivial-gray-streams (and flexi-streams) and tell us if it's possible to add code for Scieneer to them?

3 3

[usocket-devel] Portably supporting :external-format
by Erik Huelsmann 22 Oct '06

22 Oct '06

On IRC, I had some discussions whether it would be enough to just support octet streams, but it looks like not all implementations support octet streams. Definitely not all of them support :external-format for sockets. So, as I mentioned in earlier posts, it might be a good idea to start depending on flexi-streams: this gives us portable external-formats as well as *bivalent* streams. The latter is important for many TCP/IP protocols: HTTP has a character header, but binary content. Other protocols have that too (8BIT smtp). So, to stay backward compatible, this is what I'd like to do: - If no external-format is given: just return the raw stream returned by the implementation (as we do now) - If an external-format is specified, create a raw stream with 'octet' element-type and wrap the returned stream with a flexi-stream, exposing the flexi-stream to the caller. How about that? bye, Erik.

1 0

Re: [usocket-devel] Support for Server Connections
by nick thomas 22 Oct '06

22 Oct '06

On 10/20/06, Erik Huelsmann <e.huelsmann(a)gmx.net> wrote: > Hi Nick, > > Thanks for the - now correct - patch. I'd love to react to it, but you sent > both patches only to me. Was that accidental? If so, I'd like to reply to this > message on-list giving my comments for consideration and discussion. > Sorry! That was, indeed, accidental. For the benefit of those on the list: On 10/16/06, Nick Thomas <jesuswaffle(a)gmail.com> wrote: > A couple weeks ago, I hacked together the beginnings of a TCP server > socket implementation. It has documentation, but no tests. I wrote the > backend for SBCL, and it seems to be working fine. > > It doesn't support a lot of the requirements [that Erik] discussed. It > doesn't support setting socket options, and it only works with IPv4. > However, it might be useful as a basis for further work. Patch attached.

2 4

[usocket-devel] socket volunteerism
by Captain Fourier 18 Oct '06

18 Oct '06

Hey, I am interested in helping make usocket get developed. Is there anyway I can help?

4 3

[usocket-devel] Re: [usocket-cvs] r119 - in usocket/trunk: . backend doc notes
by Erik Huelsmann 18 Oct '06

18 Oct '06

Sorry. That was a bit more than Douglas submitted, I'll revert the non-related part when I get back from work. bye, Erik. On 10/18/06, ehuelsmann(a)common-lisp.net <ehuelsmann(a)common-lisp.net> wrote: > Author: ehuelsmann > Date: Wed Oct 18 02:56:33 2006 > New Revision: 119 > > Added: > usocket/trunk/backend/scl.lisp > usocket/trunk/notes/accept-apis.txt > Modified: > usocket/trunk/condition.lisp > usocket/trunk/doc/design.txt > usocket/trunk/usocket.asd > usocket/trunk/usocket.lisp > Log: > Add Scieneer support. > > Donated by Douglas Crosher <dtc at scieneer dot com>. > > > Added: usocket/trunk/backend/scl.lisp > ============================================================================== > --- (empty file) > +++ usocket/trunk/backend/scl.lisp Wed Oct 18 02:56:33 2006 > @@ -0,0 +1,101 @@ > +;;;; $Id: scl.lisp$ > +;;;; $URL: svn://common-lisp.net/project/usocket/svn/usocket/trunk/backend/scl.lisp $ > + > +;;;; See LICENSE for licensing information. > + > +(in-package :usocket) > + > +(defparameter +scl-error-map+ > + (append +unix-errno-condition-map+ > + +unix-errno-error-map+)) > + > +(defun scl-map-socket-error (err &key condition socket) > + (let ((usock-err (cdr (assoc err +scl-error-map+ :test #'member)))) > + (cond (usock-err > + (if (subtypep usock-err 'error) > + (error usock-err :socket socket) > + (signal usock-err :socket socket))) > + (t > + (error 'unknown-error > + :socket socket > + :real-error condition))))) > + > +(defun handle-condition (condition &optional (socket nil)) > + "Dispatch correct usocket condition." > + (etypecase condition > + (ext::socket-error > + (format t "erron: ~D~%" (ext::socket-errno condition)) > + (scl-map-socket-error (ext::socket-errno condition) > + :socket socket > + :condition condition)) > + (error > + (error 'unknown-error > + :real-condition condition > + :socket socket)))) > + > +(defun socket-connect (host port) > + (let* ((socket > + (with-mapped-conditions (nil) > + (ext:connect-to-inet-socket (host-to-hbo host) port :kind :stream))) > + (stream (sys:make-fd-stream socket :input t :output t > + :element-type 'character > + :buffering :full))) > + ;;###FIXME the above line probably needs an :external-format > + (make-socket :socket socket :stream stream))) > + > +(defmethod socket-close ((usocket usocket)) > + "Close socket." > + (with-mapped-conditions (usocket) > + (ext:close-socket (socket usocket)))) > + > +(defmethod get-local-name ((usocket usocket)) > + (multiple-value-bind (address port) > + (with-mapped-conditions (usocket) > + (ext:get-socket-host-and-port (socket usocket))) > + (values (hbo-to-vector-quad address) port))) > + > +(defmethod get-peer-name ((usocket usocket)) > + (multiple-value-bind (address port) > + (with-mapped-conditions (usocket) > + (ext:get-peer-host-and-port (socket usocket))) > + (values (hbo-to-vector-quad address) port))) > + > +(defmethod get-local-address ((usocket usocket)) > + (nth-value 0 (get-local-name usocket))) > + > +(defmethod get-peer-address ((usocket usocket)) > + (nth-value 0 (get-peer-name usocket))) > + > +(defmethod get-local-port ((usocket usocket)) > + (nth-value 1 (get-local-name usocket))) > + > +(defmethod get-peer-port ((usocket usocket)) > + (nth-value 1 (get-peer-name usocket))) > + > + > +(defun get-host-by-address (address) > + (multiple-value-bind (host errno) > + (ext:lookup-host-entry (host-byte-order address)) > + (cond (host > + (ext:host-entry-name host)) > + (t > + (let ((condition (cdr (assoc errno +unix-ns-error-map+)))) > + (cond (condition > + (error condition :host-or-ip address)) > + (t > + (error 'ns-unknown-error :host-or-ip address > + :real-error errno)))))))) > + > +(defun get-hosts-by-name (name) > + (multiple-value-bind (host errno) > + (ext:lookup-host-entry name) > + (cond (host > + (mapcar #'hbo-to-vector-quad > + (ext:host-entry-addr-list host))) > + (t > + (let ((condition (cdr (assoc errno +unix-ns-error-map+)))) > + (cond (condition > + (error condition :host-or-ip name)) > + (t > + (error 'ns-unknown-error :host-or-ip name > + :real-error errno)))))))) > > Modified: usocket/trunk/condition.lisp > ============================================================================== > --- usocket/trunk/condition.lisp (original) > +++ usocket/trunk/condition.lisp Wed Oct 18 02:56:33 2006 > @@ -102,8 +102,8 @@ > ;; isn't really an error: there's just no data to return. > ;; with lisp, we just return NIL (indicating no data) instead of > ;; raising an exception... > - (ns-host-not-found > - ns-no-recovery) > + (ns-host-not-found-error > + ns-no-recovery-error) > (ns-error)) > > (define-condition ns-unknown-error (ns-error) > > Modified: usocket/trunk/doc/design.txt > ============================================================================== > --- usocket/trunk/doc/design.txt (original) > +++ usocket/trunk/doc/design.txt Wed Oct 18 02:56:33 2006 > @@ -13,7 +13,7 @@ > * Motivation > * Design goal > * Functional requirements > - > + * Comments on the functional requirements > > > > @@ -83,6 +83,33 @@ > - OpenMCL > > > +The lifetime of a socket can be described with these steps: > + > + 1. Socket creation (socket() function) > + 2. Socket initializaiton (setsockopt(), bind() and listen()/connect() funcs) > + 3. Socket use (accept() / recv[from], send[to]) > + 4. Socket termination (shutdown()) > + 5. Socket destruction (close()) > + > +While for most applications steps 1-3 can be condensed into 1 (which most > +implementations do), if the library wants to be extensible into other > +domains than IP, a means should be provided to do socket initialization > +without knowing what parameters to accept beforehand: other protocols > +require parameters for setsockopt we will not know about in advance. > + > +There are several possibilities to address this issue: > + > + a. Force the 3 steps apart [hard to get done with the current status > + for some implementations, as they are currently integrated in the > + public interface]. > + b. Find a mechanism to pass options which we want setsockopt to > + be called with. Problem: what to do with implementations which > + don't support setting of all options *before* the bind() call? > + Does it matter that some options may be set after the bind() > + call? What if they're not set before connect() [buffer size changes > + have to be set before connect()]? > + c. ... ? > + > Comments on the design above > ============================ > > > Added: usocket/trunk/notes/accept-apis.txt > ============================================================================== > --- (empty file) > +++ usocket/trunk/notes/accept-apis.txt Wed Oct 18 02:56:33 2006 > @@ -0,0 +1,173 @@ > + > + -*- text -*- > + > +Part of 'Step 3': Server/passive tcp socket interfaces supplied by > +the different implementations in order to provide the same externally. > + > + > +ABCL > +==== > + > + - ext:make-server-socket port > + - ext:socket-accept socket > + - ext:socket-close socket > + > +Allegro > +======= > + > + - socket:make-socket :type :stream :connect :passive > + :local-port <port> :reuse-address t :backlog <nr> > + :local-host <host-ip> > + - socket:accept-connection sock &key wait > + - close > + > + > +clisp > +===== > + > + - socket:socket-server &optional port &key interface backlog > + - socket:socket-server-close sock > + - socket:socket-server-host sock > + - socket:socket-server-port sock > + - socket:socket-accept sock &key element-type external-format buffered timeout > + - socket:socket-options sock &rest options > + > +... and ofcourse, there's the raw-sockets > + > +CMUCL > +===== > + > + - ext:create-inet-listener port &optional kind > + &key reuseaddress backlog interface > + - ext:accept-tcp-connection socket > + > + > +LispWorks > +========= > + > + - comm::get-fd-from-socket (socket-os-fd lispworks-socket) > + - comm::create-tcp-socket-for-service port > + (may use comm::*use_so_reuseaddr* for that socket option) > + misses the ability to specify an interface to bind to. > + - comm::socket-close > + > + > +OpenMCL > +======= > + > + - openmcl-socket:accept-connection > + - openmcl-socket:make-socket :local-host <if> :local-port port > + :reuse-address t :type :stream :connect :passive > + :backlog <nr> > + - close > + > + > +SBCL > +==== > + > + - make-instance 'inet-socket > + - sb-bsd-sockets:sockopt-* > + - sb-bsd-sockets:socket-bind > + - sb-bsd-sockets:socket-listen > + - sb-bsd-sockets:socket-accept > + > + > +;; > +;; > +;; The above APIs are good enough to implement a simple > +;; accept interface, but doesn't give access to specifying > +;; socket options before the socket is bound to the interface > +;; ==> This may only actually be required for SO_REUSEADDRESS?!??? > +;; > +;; The other option would be to use lots of FFI - where needed - > +;; and use the (mostly internal) glue routines from the implementations > + > + > +ABCL > +==== > + > + With ABCL - lacking a good sockets API - it's still possible to implement > + whatever we need to get good access... > + > + > +Allegro > +======= > + > + Hmm. The accept function in this implementation does not allow limiting > + connections to a given host/ip, but it does allow to create sockets > + with mostly the right options. > + > + Is that enough reason to do this entirely in ffi?! > + > + Also, doing this in FFI would require to reverse engineer the creation > + of socket streams. Maybe Franz tech support could help there though. > + > + Need to investigate the IPC_* symbols an the sockets package: > + there are lots of functions which look like they should be useable. > + > + > +clisp > +===== > + > + This implementation allows access to the full sockets as described > + in http://clisp.cons.org/impnotes/rawsock.html. > + > + > +CMUCL > +===== > + > + Provides (in unix package): > + > + - unix-accept > + - unix-bind > + - unix-listen > + - unix-socket > + > + Provides (in extentions): > + > + - inet-sockaddr > + - get-socket-option > + - set-socket-option > + - create-inet-listener port &key host reuse-address backlog > + > + > +LispWorks > +========= > + > + The implementation provides a lot of undocumented functions the library > + could tap into: > + > + - comm::socket > + - comm::bind > + - comm::accept > + - comm::getsockopt > + - comm::setsockopt > + - comm::initialize-sockaddr_in (helper) > + - comm::streams-from-fd (helper) > + > + > +OpenMCL > +======= > + > + - make-socket provides all options which we'll need to set, > + it doesn't however provide access to [gs]etsockopt... > + > + > +SBCL > +==== > + > + provides (in sb-bsd-sockets): > + socket-bind > + socket-accept > + sokcet-listen > + > + provides (in sb-bsd-sockets-internal [sockint]): > + getsockopt > + setsockopt > + > + SO-* constants > + TCP-* constant(s) > + AF-* constants (and, since AF-* == IF-*, we don't need others) > + > + > + > > Modified: usocket/trunk/usocket.asd > ============================================================================== > --- usocket/trunk/usocket.asd (original) > +++ usocket/trunk/usocket.asd Wed Oct 18 02:56:33 2006 > @@ -28,6 +28,8 @@ > :depends-on ("condition")) > #+cmu (:file "cmucl" :pathname "backend/cmucl" > :depends-on ("condition")) > + #+scl (:file "scl" :pathname "backend/scl" > + :depends-on ("condition")) > #+sbcl (:file "sbcl" :pathname "backend/sbcl" > :depends-on ("condition")) > #+lispworks (:file "lispworks" :pathname "backend/lispworks" > > Modified: usocket/trunk/usocket.lisp > ============================================================================== > --- usocket/trunk/usocket.lisp (original) > +++ usocket/trunk/usocket.lisp Wed Oct 18 02:56:33 2006 > @@ -164,8 +164,8 @@ > (string (let ((ip (ignore-errors > (dotted-quad-to-vector-quad host)))) > (if (and ip (= 4 (length ip))) > - ip > - (host-to-hbo (get-host-by-name host))))) > + (host-byte-order ip) > + (host-to-hbo (get-host-by-name host))))) > ((vector t 4) (host-byte-order host)) > (integer host)))) > > @@ -186,9 +186,19 @@ > ;; > > (setf (documentation 'socket-connect 'function) > - "Connect to `host' on `port'. `host' is assumed to be a string of > + "Connect to `host' on `port'. `host' is assumed to be a string or > an IP address represented in vector notation, such as #(192 168 1 1). > `port' is assumed to be an integer. > > Returns a usocket object.") > > +;; Documentation for the function > +;; > +;; (defun SOCKET-LISTEN (host port &key local-ip local-port > +;; reuseaddress backlog) ..) > + > + > +;; Documentation for the function > +;; > +;; (defun SOCKET-ACCEPT (socket &key element-type external-format > +;; buffered timeout) ..) > _______________________________________________ > usocket-cvs mailing list > usocket-cvs(a)common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-cvs >

1 0

[usocket-devel] Scieneer CL support.
by Douglas Crosher 17 Oct '06

17 Oct '06

Attached are patches for the Scieneer Common Lisp implementation, plus the files backend/scl.lisp and test/scl.conf.in. All the tests pass. Regards Douglas Crosher Index: usocket.asd =================================================================== --- usocket.asd (revision 114) +++ usocket.asd (working copy) @@ -28,6 +28,8 @@ :depends-on ("condition")) #+cmu (:file "cmucl" :pathname "backend/cmucl" :depends-on ("condition")) + #+scl (:file "scl" :pathname "backend/scl" + :depends-on ("condition")) #+sbcl (:file "sbcl" :pathname "backend/sbcl" :depends-on ("condition")) #+lispworks (:file "lispworks" :pathname "backend/lispworks" Index: usocket.lisp =================================================================== --- usocket.lisp (revision 114) +++ usocket.lisp (working copy) @@ -147,8 +147,8 @@ (string (let ((ip (ignore-errors (dotted-quad-to-vector-quad host)))) (if (and ip (= 4 (length ip))) - ip - (host-to-hbo (get-host-by-name host))))) + (host-byte-order ip) + (host-to-hbo (get-host-by-name host))))) ((vector t 4) (host-byte-order host)) (integer host)))) Index: condition.lisp =================================================================== --- condition.lisp (revision 114) +++ condition.lisp (working copy) @@ -95,8 +95,8 @@ ;; isn't really an error: there's just no data to return. ;; with lisp, we just return NIL (indicating no data) instead of ;; raising an exception... - (ns-host-not-found - ns-no-recovery) + (ns-host-not-found-error + ns-no-recovery-error) (ns-error)) (define-condition ns-unknown-error (ns-error) ;;;; $Id: scl.lisp$ ;;;; $URL: svn://common-lisp.net/project/usocket/svn/usocket/trunk/backend/scl.lisp $ ;;;; See LICENSE for licensing information. (in-package :usocket) (defparameter +scl-error-map+ (append +unix-errno-condition-map+ +unix-errno-error-map+)) (defun scl-map-socket-error (err &key condition socket) (let ((usock-err (cdr (assoc err +scl-error-map+ :test #'member)))) (cond (usock-err (if (subtypep usock-err 'error) (error usock-err :socket socket) (signal usock-err :socket socket))) (t (error 'unknown-error :socket socket :real-error condition))))) (defun handle-condition (condition &optional (socket nil)) "Dispatch correct usocket condition." (etypecase condition (ext::socket-error (format t "erron: ~D~%" (ext::socket-errno condition)) (scl-map-socket-error (ext::socket-errno condition) :socket socket :condition condition)) (error (error 'unknown-error :real-condition condition :socket socket)))) (defun socket-connect (host port) (let* ((socket (with-mapped-conditions (nil) (ext:connect-to-inet-socket (host-to-hbo host) port :kind :stream))) (stream (sys:make-fd-stream socket :input t :output t :element-type 'character :buffering :full))) ;;###FIXME the above line probably needs an :external-format (make-socket :socket socket :stream stream))) (defmethod socket-close ((usocket usocket)) "Close socket." (with-mapped-conditions (usocket) (ext:close-socket (socket usocket)))) (defmethod get-local-name ((usocket usocket)) (multiple-value-bind (address port) (with-mapped-conditions (usocket) (ext:get-socket-host-and-port (socket usocket))) (values (hbo-to-vector-quad address) port))) (defmethod get-peer-name ((usocket usocket)) (multiple-value-bind (address port) (with-mapped-conditions (usocket) (ext:get-peer-host-and-port (socket usocket))) (values (hbo-to-vector-quad address) port))) (defmethod get-local-address ((usocket usocket)) (nth-value 0 (get-local-name usocket))) (defmethod get-peer-address ((usocket usocket)) (nth-value 0 (get-peer-name usocket))) (defmethod get-local-port ((usocket usocket)) (nth-value 1 (get-local-name usocket))) (defmethod get-peer-port ((usocket usocket)) (nth-value 1 (get-peer-name usocket))) (defun get-host-by-address (address) (multiple-value-bind (host errno) (ext:lookup-host-entry (host-byte-order address)) (cond (host (ext:host-entry-name host)) (t (let ((condition (cdr (assoc errno +unix-ns-error-map+)))) (cond (condition (error condition :host-or-ip address)) (t (error 'ns-unknown-error :host-or-ip address :real-error errno)))))))) (defun get-hosts-by-name (name) (multiple-value-bind (host errno) (ext:lookup-host-entry name) (cond (host (mapcar #'hbo-to-vector-quad (ext:host-entry-addr-list host))) (t (let ((condition (cdr (assoc errno +unix-ns-error-map+)))) (cond (condition (error condition :host-or-ip name)) (t (error 'ns-unknown-error :host-or-ip name :real-error errno)))))))) # lisp binary test setup file args= # lisp_bin is required! lisp_bin="/opt/scl/bin/scl" lisp_name=SCL # lisp_exit is required! lisp_exit="(unix:unix-exit result)"

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

usocket-devel