- drakma-devel@lists.common-lisp.net - mailman3.common-lisp.net

[drakma-devel] Not following redirects and conditions
by Saurabh Nanda 09 Feb '07

09 Feb '07

Hi, I'm trying to write some tests for an HTTP based API. I need to check whether a server responds with a 302 status code and then need to check the referred location as well, without actually visiting that link. How is it possible with drakma? If I use :redirect 0 then the http-request function throws up an error. TIA Nandz. -- http://nandz.blogspot.com http://foodieforlife.blogspot.com

2 4

[drakma-devel] Closing streams with :want-stream t
by Chris Dean 08 Feb '07

08 Feb '07

I have a file handle leak in my code (that is, I run out of file handles after running for a while) and I suspect that I'm not using drakma correctly. I wish to use the :want-stream t parameter and read the resulting stream directly, so I have this code: (defun simple-get (url) "Download the url using GET and return the body as a string." (handler-case (multiple-value-bind (stream code headers dummy-uri dummy-stream must-close?) (drakma:http-request url :want-stream t :keep-alive nil :method :get) (declare (ignore headers dummy-stream dummy-uri)) (unwind-protect (and stream code (= code 200) (with-output-to-string (out) (do ((ch (read-char stream nil :eof) (read-char stream nil :eof))) ((not (characterp ch))) (princ ch out)))) (when (and stream must-close?) (ignore-errors (close stream))))) (error (condition) (format t "Error ~s: ~a~%" url condition) nil))) Is there anything extra I need to do to make sure that all the streams opened by drakma are closed? My production code is much more complex, but the simple stub above will generate the out of file handles problem. Besides the actual error I can use lsof on Linux and Mac OS and see many sockets stuck in CLOSED or CLOSE_WAIT states. This is all under LispWorks 5.0.1 Also, when using :want-stream nil I never encounter the problem. Cheers, Chris Dean

3 12

[drakma-devel] New Drakma release 0.6.0
by Edi Weitz 08 Feb '07

08 Feb '07

ChangeLog: Version 0.6.0 2006-02-08 Make sure stream is closed in case of early errors (thanks to Chris Dean for test data) Robustified cookie parsing Send all outgoing cookies in one fell swoop (for Sun's buggy web server) Deal with empty Location headers Deal with corrupted Content-Type headers Download: http://weitz.de/files/drakma.tar.gz Have fun, Edi.

1 0

[drakma-devel] New Chunga release 0.2.4
by Edi Weitz 08 Feb '07

08 Feb '07

ChangeLog: Version 0.2.4 2007-02-08 Allow more characters in cookie names/values according to original Netscape spec Robustified READ-COOKIE-VALUE Download: http://weitz.de/files/chunga.tar.gz Cheers, Edi.

1 0

[drakma-devel] Re: drakma/chunga problem.
by Edi Weitz 05 Feb '07

05 Feb '07

[Cc to mailing list.] On Mon, 5 Feb 2007 19:55:34 +0800, "Asbjørn Bjørnstad" <asbjxrn(a)bjxrnstad.net> wrote: > BTW, I got a bounce back from the mailing list as I am not a member. > Is that intentional? (Could stop some from submitting bug reports.) It's for subscribers only as are almost all mailing lists I know. Yes, you have to subscribe, but I think it's not asking too much if you want free support for software you didn't pay for. The alternative would be that the list would be swamped with spam which is not really an alternative to me. (And I don't like to handle questions and bug reports off list either, because it very often means that you have to say the same thing more than once.) Cheers, Edi.

1 0

[drakma-devel] Re: drakma/chunga problem.
by Edi Weitz 04 Feb '07

04 Feb '07

Hi! On Sun, 4 Feb 2007 13:45:25 +0800, "Asbjørn Bjørnstad" <asbjxrn(a)bjxrnstad.net> wrote: > I'm not sure whether this is a bug or not. [Please use the mailing list to report bugs. See Cc.] > I'm planning to set up automatic bug reporting into trac. > (http://trac.edgewall.com) > > Posting the message works, but if I try to add a cookie jar, I get > an error. (The cookie jar is empty before the call, if that is not > how it's supposed to be used, you can safely ignore this.) As I said > it's working without cookies, and I don't plan to use cookies, just > thought you might want to know about a possible bug. This is with > the latest version of chunga and drakma. > > Backtrace attached as I don't know if gmail might break lines and > make it unreadable (Password changed, if you want a trac account to > test youself, it can be arranged.) I think this is the relevant part of the backtrace: Call to CHUNGA:READ-NAME-VALUE-PAIR (offset 78) STREAM : #<SYSTEM::STRING-INPUT-STREAM 2175A5B3> CHUNGA::VALUE-REQUIRED-P : NIL CHUNGA::COOKIE-SYNTAX : T Call to CHUNGA:READ-NAME-VALUE-PAIRS (offset 449) STREAM : #<SYSTEM::STRING-INPUT-STREAM 2175A5B3> CHUNGA::VALUE-REQUIRED-P : NIL CHUNGA::COOKIE-SYNTAX : T CHAR : #\; DBG::|accumulator-| : (NIL ("expires" . "Sat, 05-May-2007 05:11:54 GMT") ("Path" . "/realtist")) DBG::|aux-var-| : (("Path" . "/realtist")) Binding frame: CHUNGA:*CURRENT-ERROR-MESSAGE* : NIL Catch frame: #<Unbound Marker> Call to DRAKMA::PARSE-SET-COOKIE (offset 426) STRING : "trac_session=20ae843edfe4ed8c7a3815ec; expires=Sat, 05-May-2007 05:11:54 GMT; Path=/realtist;" DBG::OBJ : #<SYSTEM::STRING-INPUT-STREAM 2175A5B3> DBG::DESC : (#<SYSTEM::STRING-INPUT-STREAM 2175A5B3> T :DONT-CARE) STREAM : #<SYSTEM::STRING-INPUT-STREAM 2175A5B3> CHUNGA:*CURRENT-ERROR-MESSAGE* : "While parsing cookie header \"trac_session=20ae843edfe4ed8c7a3815ec; expires=Sat, 05-May-2007 05:11:54 GMT; Path=/realtist;\":" FIRST : T DRAKMA::NEXT : #\t DRAKMA::NAME/VALUE : ("trac_session" . "20ae843edfe4ed8c7a3815ec") DRAKMA::PARAMETERS : NIL DBG::|accumulator-| : (NIL) DBG::|aux-var-| : (NIL) It seems the problem is the semicolon at the end and that this is a bug that was fixed in Chunga 0.2.3. Are you sure you're using the latest version? http://weitz.de/chunga/CHANGELOG.txt Cheers, Edi.

1 0

[drakma-devel] drakma vs. http://popurls.com
by Chris Dean 30 Jan '07

30 Jan '07

I have what will probably end up being an obvious and foolish question. When I run (http-request "http://popurls.com/") I get an error from flexi-streams that says: Unexpected value #x20 in UTF-8 sequence. [Condition of type FLEXI-STREAM-ENCODING-ERROR] What's going on? Do I need to set external-format-in ? An abbreviated stack trace is below. Cheers, Chris Dean 1: (METHOD STREAM:STREAM-READ-CHAR (FLEXI-STREAMS::FLEXI-UTF-8-INPUT-STREAM)) (#<FLEXI-STREAMS::FLEXI-BINARY-UTF-8-IO-STREAM 200C2FA7>) Locals: STREAM = #<FLEXI-STREAMS::FLEXI-BINARY-UTF-8-IO-STREAM 200C2FA7> CLOS::.ISL. = #(#(NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL ...) #(FLEXI-STREAMS::LAST-CHAR-CODE FLEXI-STREAMS::LAST-OCTET) 126 0) CLOS::.PV. = #(5 6) FLEXI-STREAMS::FIRST-OCTET-SEEN = T OCTET = 194 FLEXI-STREAMS::START = 2 COUNT = 1 DBG::EXTRA-VALS = :DONT-KNOW FLEXI-STREAMS::RESULT = 2 DBG::|repeat-counter-| = 0 OCTET = 32 2: (METHOD TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE (FLEXI-INPUT-STREAM T T T)) (#<FLEXI-STREAMS::FLEXI-BINARY-UTF-8-IO-STREAM 200C2FA7> ... 3: CLOS::GENERIC-FUNCTION-DISCRIMINATOR NIL 4: DRAKMA::READ-BODY (#<FLEXI-STREAMS::FLEXI-BINARY-UTF-8-IO-STREAM 200C2FA7> ((:DATE . "Mon, 29 Jan 2007 23:02:31 GMT") (:SERVER . "Apache") (:EXPIRES . "Mon, 26 Jul 1997 05:00:00 GMT") (:CACHE-CONTROL . "no-store, no-cache, must-revalidate,post-check=0, pre-check=0") (:PRAGMA . "no-cache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=UTF-8")) T #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-8 :EOL-STYLE :LF) 20106C9B>) Locals: STREAM = #<FLEXI-STREAMS::FLEXI-BINARY-UTF-8-IO-STREAM 200C2FA7> DRAKMA::HEADERS = ((:DATE . "Mon, 29 Jan 2007 23:02:31 GMT") (:SERVER . "Apache") (:EXPIRES . "Mon, 26 Jul 1997 05:00:00 GMT") (:CACHE-CONTROL . "no-store, no-cache, must-revalidate,post-check=0, pre-check=0") (:PRAGMA . "no-cache") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "text/html; charset=UTF-8")) DRAKMA::MUST-CLOSE = T DRAKMA::TEXTP = #<FLEXI-STREAMS::EXTERNAL-FORMAT (:UTF-8 :EOL-STYLE :LF) 20106C9B> DRAKMA::CONTENT-LENGTH = NIL DRAKMA::ELEMENT-TYPE = LISPWORKS:SIMPLE-CHAR DRAKMA::CHUNKEDP = T DRAKMA::BUFFER = ... DRAKMA::RESULT = ... DRAKMA::INDEX = 49152 DRAKMA::POS = 8192

2 4

[drakma-devel] New Chunga release 0.2.3
by Edi Weitz 17 Jan '07

17 Jan '07

Changelog: Version 0.2.3 2007-01-17 Guard against stray semicolons when reading name/value pairs (thanks to Bülent Murtezaoglu) Download: http://weitz.de/files/chunga.tar.gz Cheers, Edi.

1 0

[drakma-devel] New Chunga release 0.2.2
by Edi Weitz 09 Jan '07

09 Jan '07

ChangeLog: Version 0.2.2 2007-01-10 Faster vesion of READ-LINE* (provided by Gábor Melis) Download: http://weitz.de/files/chunga.tar.gz Cheers, Edi.

1 0

[drakma-devel] Drakma : Problem saving file
by lam＠tuxfamily.org 18 Dec '06

18 Dec '06

hello, i make some test with drakma and i have a problem with http-request : CL-USER> (with-open-file (os (format nil "/tmp/~A.png" "tttt") :direction :output :if-exists :supersede :element-type 'unsigned-byte) (multiple-value-bind (body-stream status-code headers uri stream must-close) (drakma:http-request "http://stats-dev/cgi-bin/view_image.cgi?filename=/tmp/sms_5Avkjm.png" :want-stream t :force-binary t :content-length t) (declare (ignore headers uri stream must-close)) (when (= status-code 200) (format t "Type ~A~%" (type-of body-stream)) (loop for c = (read-byte body-stream nil) until (null c) do (progn (format t "Write ~%") (write-byte c os))) (close body-stream)))) Type FLEXI-IO-STREAM T after i try with another image : CL-USER> (with-open-file (os (format nil "/tmp/~A.png" "tttt") :direction :output :if-exists :supersede :element-type 'unsigned-byte) (multiple-value-bind (body-stream status-code headers uri stream must-close) (drakma:http-request "http://www.google.com/images/logo_sm.gif" :want-stream t :force-binary t :content-length t) (declare (ignore headers uri stream must-close)) (when (= status-code 200) (format t "Type ~A~%" (type-of body-stream)) (loop for c = (read-byte body-stream nil) until (null c) do (progn (format t "Write ~%") (write-byte c os))) (close body-stream)))) Type FLEXI-IO-STREAM Write Write Write Write Write Write [...] Write Write T and the file /tmp/tttt.png contains the google's logo. With the first request, the file is empty. Here it is the differents between the 2 requests : CL-USER> (drakma:http-request "http://stats-dev/cgi-bin/view_image.cgi?filename=/tmp/sms_5Avkjm.png" :want-stream t) #<FLEXI-STREAMS:FLEXI-IO-STREAM {B0F1BF9}> 200 ((:DATE . "Tue, 05 Dec 2006 18:09:17 GMT") (:SERVER . "Apache/1.3.34 (Debian) PHP/4.4.4-6 mod_jk/1.2.18 mod_ssl/2.8.25 OpenSSL/0.9.8c mod_perl/1.29") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "image/png")) #<URI http://stats-dev/cgi-bin/view_image.cgi?filename=/tmp/sms_5Avkjm.png> #<FLEXI-STREAMS:FLEXI-IO-STREAM {B0F1BF9}> T and : CL-USER> (drakma:http-request "http://www.google.com/images/logo_sm.gif" :want-stream t) #<FLEXI-STREAMS:FLEXI-IO-STREAM {B13C631}> 200 ((:CONTENT-TYPE . "image/gif") (:LAST-MODIFIED . "Wed, 07 Jun 2006 19:38:24 GMT") (:EXPIRES . "Sun, 17 Jan 2038 19:14:07 GMT") (:SERVER . "GWS/2.1") (:CONTENT-LENGTH . "4707") (:DATE . "Tue, 05 Dec 2006 18:10:07 GMT")) #<URI http://www.google.com/images/logo_sm.gif> #<FLEXI-STREAMS:FLEXI-IO-STREAM {B13C631}> T i see that in the request which have a problem content-length is not sended. but i don't really understand why the first image is empty. thanks if someone understand my problem.

2 4