There was the same decision at Dec 2007. Now, three years later, you have not changed it. Anyway, this patch can be usefull for somebody else. ----------------исходное сообщение----------------- От: "Edi Weitz" edi@weitz.de Кому: "General interest list for Drakma and Chunga" drakma-devel@common-lisp.net CC(Твердая копия): "Александр Литвинов" lan@academsoft.ru Дата: Sat, 26 Mar 2011 11:44:20 +0100 -------------------------------------------------

I think this is something that belongs into usocket and not into Drakma. I'm not monitoring usocket, but if it had the ability to optionally specify timeouts and deadlines for those implementations supporting these features (and making them no-ops on others), I'd happily integrate that into Drakma. I don't want to litter Drakma with conditional expression for various Lisps, though.

Thanks, Edi.

On Sat, Mar 26, 2011 at 5:15 AM, Александр Литвинов lan@academsoft.ru wrote:

...

A long time ago (Dec 2007) there was a patch [1] for drakma to implement timeouts with sbcl. It was rejected with suggestion to post similar patch into usocket library.

Totday, I start to learn lisp programing and I need http-request for my small program. I took drakma for this. But it does not support timeouts on sbcl. I use debian and it seems sbcl - is the best choice for me.

By googling I found a patch for drakma. I repplaied it for 1.2.3 drakma. It seems to me, it still forth appling into main line of drakma. I did not investigate what happen with usocket patch and such long time shows - this patch needs to be applied anyway :-)

1. http://common-lisp.net/pipermail/drakma-devel/2007-December/000601 .html

-- Alexander Litvinov

_______________________________________________ drakma-devel mailing list drakma-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel

Hi Edi, 2011/3/26 Edi Weitz <edi@weitz.de>:

On Sat, Mar 26, 2011 at 6:17 PM, Александр Литвинов <lan@academsoft.ru> wrote:

...

 There was the same decision at Dec 2007. Now, three years later, you have not changed it.

Well, I don't think it's too bad not to change your mind every year... :)

Actually, I had hoped that there had been some development in usocket since.

Well, there has been some development in usocket in the direction of supporting timeouts. On platforms where timeouts are not supported, an UNSUPPORTED-FEATURE condition is signalled. If that's ignored, everything should just continue without the timeout settings. If the above is not good enough, could you indicate what more needs to happen to make it so? Bye, Erik.

Hi Chun, 2011/3/27 Chun Tian (binghe) <binghe.lisp@gmail.com>:

Hi, Erik

Could you tell me what exact timeout feature they want? Setting Read/Write timeout when doing SOCKET-CONNECT? or Apply new timeout values to exist usocket object?

As far as I understand, the request is to for SOCKET-CONNECT to be cancelled after a certain timeout period expires. I don't think that's a read/write timeout, but rather a way to limit the waiting time until a valid connection is established. Bye, Erik.

在 2011-3-27，03:11， Erik Huelsmann 写道：

...

Hi Edi,

2011/3/26 Edi Weitz <edi@weitz.de>:

...

On Sat, Mar 26, 2011 at 6:17 PM, Александр Литвинов <lan@academsoft.ru> wrote:

...

 There was the same decision at Dec 2007. Now, three years later, you have not changed it.

Well, I don't think it's too bad not to change your mind every year... :)

Actually, I had hoped that there had been some development in usocket since.

Well, there has been some development in usocket in the direction of supporting timeouts. On platforms where timeouts are not supported, an UNSUPPORTED-FEATURE condition is signalled. If that's ignored, everything should just continue without the timeout settings.

If the above is not good enough, could you indicate what more needs to happen to make it so?

Bye,

Erik.

Well, I think the TIMEOUT keyword arguments on SOCKET-CONNECT currently has following meanings: 1. When doing TCP work, it means a "connection timeout"; 2. When doing UCP work, it sets the default "read timeout" (because "connection timeout" is meaningless for UDP); The TIMEOUT keyword arguments was added from 0.4.0 (released on Oct 2008, more than two years ago), so I think Drakma should try to use it. However, it may not work on all supported platforms -- I remembered. But this should be considered as bug, I'll try to fix it after a small rewrite of our unit test framework. On the other hand, for "read timeout" and "write timeout", they're changeable properties (or options) for any socket objects. A new API called SOCKET-OPTION (learnt from CLISP) should be added in next USOCKET major version. Dynamically setting options like "broadcast" and "reuse address" should also be added under SOCKET-OPTION. I see hunchentoot has a function called SET-TIMEOUTS, which can be used to set "read timeout" and "write timeout" for any exist usocket, I want to merge it into usocket under the new SOCKET-OPTION API. --binghe 在 2011-3-28，01:50， Edi Weitz 写道：

2011/3/27 Chun Tian (binghe) <binghe.lisp@gmail.com>:

...

Could you tell me what exact timeout feature they want? Setting Read/Write timeout when doing SOCKET-CONNECT? or Apply new timeout values to exist usocket object?

The LW version of Drakma supports connection timeouts, read timeouts, and write timeouts. Existing usocket objects wouldn't be modified.

Cheers, Edi.

Hi, Nikodemus 在 2011-3-28，04:26， Nikodemus Siivola 写道：

2011/3/27 Chun Tian (binghe) <binghe.lisp@gmail.com>:

...

Well, I think the TIMEOUT keyword arguments on SOCKET-CONNECT currently has following meanings:

1. When doing TCP work, it means a "connection timeout"; 2. When doing UCP work, it sets the default "read timeout" (because "connection timeout" is meaningless for UDP);

For SBCL backend at least this is not correct.

I see. But for other backends (at least LW and CCL), seems correct. Among all CL platforms, Clozure CL have the most feature-rich networking API. Its MAKE-SOCKET [1] provided three keyword arguments: INPUT-TIMEOUT, OUTPUT-TIMEOUT and CONNECT-TIMEOUT, and an additional DEADLINE. LispWorks only has TCP support, but its OPEN-TCP-STREAM [2] also provided three keyword arguments: READ-TIMEOUT, WRITE-TIMEOUT, and TIMEOUT. [1] http://ccl.clozure.com/manual/chapter7.2.html [2] http://www.lispworks.com/documentation/lw60/LW/html/lw-553.htm#pgfId-888355

There :TIMEOUT is a read timeout for any single blocking read from the socket. Connections and writes are unaffected -- though SBCL changes may apply the timeouts to writes as well.

Waiting for a connection is not affected -- but adding support for that is pretty easy.

Then I suggest adding these missing part, at lest adjust the meaning of "TIMEOUT" argument of SOCKET-MAKE-STREAM into "connection timeout", and add READ-TIME / WRITE-TIMEOUT support.

Sidenote: I haven't really looked at usocket's SBCL backend beyond SOCKET-CONNECT prior to this, but a quick peek shows that it's pretty cavalier about using SBCL internals. This is not good. Internals _will_ change, and then usocket will break. Specifically, SB-UNIX package is an internal implementation package -- using it is bad. SB-POSIX is the supported POSIX api. Some SB-FOO::BAR stuff in there as well. :/

Well, I think maybe the two projects (USOCKET and SBCL) didn't collaborate quite well in past years. USOCKET authors (at least me) want to support as most versions of SBCL as possible. This means, if we ask SBCL to add all necessary support to make things on usocket side become trivial, then we will loose support for older SBCL versions. Sometimes we have to do some dirty work (all around SBCL backend code) to detect if a "feature" exist in a SBCL version... I admit this is not good, but consider SBCL's current networking API lack of many features, and SBCL's release cycle is too fast to make all in-use SBCL versions quickly become old, I don't think a full rewrite of usocket's SBCL backend is possible at current stage. But if SBCL official can start to improve its networking part from now on, after maybe one year, or after at least 10 new SBCL releases, maybe we can force switch to a new backend with "good code" without using any internals. Glad to know your thoughts on this. Regards, --binghe

Cheers,

-- Nikodemus

Hi, Nikodemus Today I think out another way to solve the SBCL connection timeout issue, I wrap a SB-EXT:WITH-TIMEOUT on SB-BSD-SOCKET:SOCKET-CONNNECT [1], and the result work seems working well: * (time (ignore-errors (usocket:socket-connect "8.8.8.8" 80 :timeout 3))) Evaluation took: 3.010 seconds of real time 0.010234 seconds of total run time (0.009223 user, 0.001011 system) 0.33% CPU 32 lambdas converted 6,608,582,602 processor cycles 916,672 bytes consed NIL #<USOCKET:TIMEOUT-ERROR {1002F34851}> Do you think this is an acceptable solution? --binghe [1] http://trac.common-lisp.net/usocket/changeset/589 在 2011-3-28，04:26， Nikodemus Siivola 写道：

2011/3/27 Chun Tian (binghe) <binghe.lisp@gmail.com>:

...

Well, I think the TIMEOUT keyword arguments on SOCKET-CONNECT currently has following meanings:

1. When doing TCP work, it means a "connection timeout"; 2. When doing UCP work, it sets the default "read timeout" (because "connection timeout" is meaningless for UDP);

For SBCL backend at least this is not correct.

There :TIMEOUT is a read timeout for any single blocking read from the socket. Connections and writes are unaffected -- though SBCL changes may apply the timeouts to writes as well.

Waiting for a connection is not affected -- but adding support for that is pretty easy.

Sidenote: I haven't really looked at usocket's SBCL backend beyond SOCKET-CONNECT prior to this, but a quick peek shows that it's pretty cavalier about using SBCL internals. This is not good. Internals _will_ change, and then usocket will break. Specifically, SB-UNIX package is an internal implementation package -- using it is bad. SB-POSIX is the supported POSIX api. Some SB-FOO::BAR stuff in there as well. :/

Cheers,

-- Nikodemus

Gods, I hate asynch timeouts. Is there a sane way to tell connect() to time out without needing SIGALRM?

Nope. Have you considered using a non-blocking connect() call, combining that with a select() call with a timeout? Bye, Erik.

I know SBCL's WITH-TIMEOUT cannot nest, I learn this from GBBopen's portable-threads.lisp [1], and it also give a nested version SBCL's WITH-TIMEOUT, much shorter than yours: #+sbcl (defmacro with-timeout ((seconds &body timeout-body) &body timed-body) (let ((tag-sym (gensym)) (timer-sym (gensym))) `(block ,tag-sym (let ((,timer-sym (sb-ext:make-timer #'(lambda () (return-from ,tag-sym (progn ,@timeout-body)))))) (sb-ext:schedule-timer ,timer-sym ,seconds) (unwind-protect (progn ,@timed-body) (sb-ext:unschedule-timer ,timer-sym)))))) I didn't use this version simply because I think the WITH-TIMEOUT form in usocket's SOCKET-CONNECT has no chance to be nested. --binghe 在 2011-3-29，04:35， Nikodemus Siivola 写道：

2011/3/28 Chun Tian (binghe) <binghe.lisp@gmail.com>:

...

Today I think out another way to solve the SBCL connection timeout issue, I wrap a SB-EXT:WITH-TIMEOUT on SB-BSD-SOCKET:SOCKET-CONNNECT [1], and the result work seems working well:

That's along the lines I was thinking off, except that SB-EXT:WITH-TIMEOUT is a broken construct. (Soon to be deprecated, in all likelihood.)

Consider this:

(with-timeout 1.0 (handler-case (with-timeout 4.0 (sleep 2)) (sb-ext:timeout ())))

which is to say that you cannot distinguish an outer timeout from an inner one, which is bad.

You need something like this, instead:

(defmacro with-timeout-handler ((seconds timeout-form) &body body) "Runs BODY as an implicit PROGN with timeout of SECONDS. If timeout occurs before BODY has finished, BODY is unwound and TIMEOUT-FORM is executed with its values returned instead.

Note that BODY is unwound asynchronously when a timeout occurs, so unless all code executed during it -- including anything down the call chain -- is asynch unwind safe, bad things will happen. Use with care." (alexandria:with-gensyms (exec unwind timer timeout block) `(block ,block (tagbody (flet ((,unwind () (go ,timeout)) (,exec () ,@body)) (declare (dynamic-extent #',exec #',unwind)) (let ((,timer (sb-ext:make-timer #',unwind))) (declare (dynamic-extent ,timer)) (sb-sys:without-interrupts (unwind-protect (progn (sb-ext:schedule-timer ,timer ,seconds) (return-from ,block (sb-sys:with-local-interrupts (,exec)))) (sb-ext:unschedule-timer ,timer))))) ,timeout (return-from ,block ,timeout-form)))))

with which

(with-timeout-handler (1.0 :outer) (with-timeout-handler (4.0 :inner) (sleep 10.0) :ok))

does the right thing.

Gods, I hate asynch timeouts. Is there a sane way to tell connect() to time out without needing SIGALRM?

Cheers,

-- Nikodemus

Hi Erik, On Sat, Mar 26, 2011 at 8:11 PM, Erik Huelsmann <ehuels@gmail.com> wrote:

Well, there has been some development in usocket in the direction of supporting timeouts. On platforms where timeouts are not supported, an UNSUPPORTED-FEATURE condition is signalled. If that's ignored, everything should just continue without the timeout settings.

If the above is not good enough, could you indicate what more needs to happen to make it so?

I didn't know that usocket supports timeouts now. I was assuming it didn't because the OP resubmitted an old patch.

From your description it sounds as if this should be exactly what Drakma needs, so if someone wants to send a patch to integrate this usocket functionality (assuming this is in a released version) with Drakma, please go ahead. <http://weitz.de/patches.html>

Thanks, Edi.

On 28 March 2011 22:47, Daniel Weinreb <dlw@itasoftware.com> wrote:

I don't know SBCL can't accept :deadline, subtract from it the current time, and then use that as a :timeout, but maybe there's a good reason for

Not really. The timeout becomes the stream timeout, which is the bound given to waiting for new input when a buffer needs to be refilled. Adding stream deadline support for SBCL needs to happen first, really, unless usocket wants to implement streams from scratch or wrap fd streams with a grey stream that check the deadline. Cheers, -- Nikodemus