Hi Erik,

since I last posted my patch, I have made some more changes to usocket to support connect timeouts. I also removed the need for platform specific syscalls files - Gary Byers suggested that we use #_select instead of the direct syscall interface, which I think is the right thing to do.

Let me know if that suits. -Hans

2008/4/9, Erik Huelsmann ehuels@gmail.com:

On 3/25/08, Hans Hübner hans@huebner.org wrote:

...

Hi,

please consider this patch:

http://bknr.net/trac/changeset/2795?format=diff&new=2795

It fixes loading of system calls for Linux and FreeBSD. I have opened a bug with Clozure to make them move to a unified system call loading interface so that applications do not need to dispatch on an architecture symbol, but until that went into CCL, the fix should be fine.

I missed this patch. I'll commit it tonight and backport it to the release branches it applies to.

Bye,

Erik.

Hi Erik,

thank you for committing the changes I submitted.

Deadlines and timeouts are of particular importance to me, so let me describe what the difference between the two are:

A stream timeout, as it is commonly available (i.e. in Lispworks or CCL), is usually implemented by setting low-level timeouts on the file descriptors underlying the stream. The result of this is that I/O operations on the stream level may take a lot longer than the timeouts specified, as stream operations may map to a series of low-level file descriptor calles that are governed by a timeout.

Stream deadlines, as implemented in CCL 1.2, are specified as absolute time in the future. The meaning of a stream deadline is that no I/O operation on the stream should be possible after the deadline time.

Deadlines are useful if one wants to limit overall request execution times in systems that initiate outgoing network transactions without having to resort to asynchronous mechanisms which are inherently risky and less robust than synchronous ones. I hope that other Lisp implementors can be convinced to support them in the future, but for the moment, I'd be fine if usocket supports deadlines for CCL, as that is what the application I am working on runs on.

I can revisit my patch so that the deadline argument is supported on all platforms, generating an error for those platforms that can't support it.

Just as an unrelated heads up: I am going to port the Hunchentoot http server over to usocket for the same reason. We want to have a fixed upper bound on http request reading and response writing as well.

Thanks again! Hans

On 4/15/08, Hans Hübner hans@huebner.org wrote:

Erik,

I am currently porting Hunchentoot to usocket and there I noticed that no API is provided to set timeouts either. I will work on adding those and also extend the argument list for all implementations. I will be sending a patch later this week.

That's right. I have your patch in my working copy regarding addition of TIMEOUTs, but if you're going to do more work on it, I'll hold. The reason it was not committed yet was my question regarding deadlines (and the fact that I was busy yesterday evening - blocking a commit).

Bye,

Erik.

-Hans

On Fri, Apr 11, 2008 at 11:41 PM, Erik Huelsmann ehuels@gmail.com wrote:

...

...

since I last posted my patch, I have made some more changes to usocket to support connect timeouts. I also removed the need for platform specific syscalls files - Gary Byers suggested that we use #_select instead of the direct syscall interface, which I think is the right thing to do.

Let me know if that suits.

It suits. I committed most of the patch as a series of unrelated commits, because that will let me backport some of the changes independently to the release branches.

The only part I did not commit (yet) was the one which introduced the timeouts for SOCKET-CONNECT: I found you introduced 2 keyword parameters on some implementations, but only one for others. Next to that, I'm unclear on what the difference is between the two: could you (shortly) explain the difference between TIMEOUT and DEADLINE?

Thanks in advance!

Bye,

Erik.

...

...

...

http://bknr.net/trac/changeset/2795?format=diff&new=2795

It fixes loading of system calls for Linux and FreeBSD. I have opened a bug with Clozure to make them move to a unified system call loading interface so that applications do not need to dispatch on an architecture symbol, but until that went into CCL, the fix should be fine.

I missed this patch. I'll commit it tonight and backport it to the release branches it applies to.

Bye,

Erik.

Hi Hans,

I'm sorry to have to be short in my answer right now - I'm rather busy this week, but I saw you're coming to ECLM this weekend too, so maybe we can talk a bit more there.

On 4/15/08, Hans Hübner hans@huebner.org wrote:

Hi Erik,

further investigation showed that the issue is more complex than I had hoped for (certainly I should not hope when it comes to computer related stuff, but I always make that mistake).

What usocket should support in any case is the setting of socket options. This would include setting SO_SNDTIMEO and SO_RCVTIMEO, and that way one would have access to a portable timeout mechanism.

It has been a primary goal to add at least the most basic socket options to usocket from the beginning, so, to me SO_SNDTIMEO and SO_RCVTIMEO are not debatable. Other options might be more in that range, but not these two as I believe they're widely supported by operating systems.

I also think usocket should support access to the deadline mechanism in CCL - I will try to convince Lisp implementors to add the feature to their streams code, but that may or may not be successful. If I can't convince you to support that option, I would propably be able to work around that, but it would be a little painful to have to add system dependencies to application code (Drakma in this case).

I agree that it would be usefull to add deadlines to the usocket. Doing so could be done in several ways: implementation specific support or returning socket streams based on gray streams which do deadline checking within usocket.

In addition, there is the timeout mechanism that SBCL supports on FD-STREAMS. The issue with these is that they are rather SBCL specific and only affect input. Also, timeouts on SBCL's FD-STREAMS cannot be established after the fact without using internally defined SBCL symbols, which is a little painful. SB-BSD-SOCKET does have an interface for setting socket options, but sadly SO_SNDTIMEO and SO_RCVTIMEO can't be set - presumably because of the struct timeval handling involved.

I think having access to socket options is least debatable, so if we could agree that this should be done I would work a little on that. I would like to use the standard identifiers for naming the socket options (SO_SNDTIMEO instead of so-send-timeout or anything else) in order to make it easier to map from standard socket documentation to usocket identifiers.

If you (or anyone else reading) have ideas how implementation specific timeout mechanisms could be added to usocket without having to put implementation specific code into applications, let me know.

As for supporting deadlines, I would - for now - rather use a specific condition type in relation to the warn or signal call: that way callers can figure out what's actually happening and decide to continue or not.

Bye,

Erik.