[drakma-devel] GET :PARAMETERS added again during redirects
I use DRAKMA 1.0, Clozure CL 1.4-dev (Ubuntu Linux x86). According to the DRAKMA documentation, parameters is an alist of name/value pairs (the car and the cdr each being a string) which denotes the parameters which are added to the query part of the URI or (in the case of a POST request) comprise the request body. In my example QUOTES1 sends a GET request passing the request parameters in the URI, and QUOTES2 uses the :PARAMETERS instead. The results, unfortunately, are not the same. (defun quotes1 () (multiple-value-bind (body code) (drakma:http-request "http://finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1") (when (= code 200) (split-sequence:split-sequence #\, body)))) ("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M ") <=== It's fine :-) (defun quotes2 () (multiple-value-bind (body code) (drakma:http-request "http://finance.yahoo.com/d/quotes.csv" :method :get :parameters '(("s" . "XIC.TO") ("f" . "sxl1d1t1"))) (when (= code 200) (split-sequence:split-sequence #\, body)))) ("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M \"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M ") <=== It's a mess :-( TRACE DRAKMA:HTTP-REQUEST shows that in case of redirects of GET requests the parameters from :PARAMETERS are added again to the URI. Thus, the same parameters may be added again and again, giving unpredictable results. 0> Calling (DRAKMA:HTTP-REQUEST "http://finance.yahoo.com/d/quotes.csv" :PARAMETERS (("s" . "XIC.TO") ("f" . "sxl1d1t1"))) 1> Calling (DRAKMA:HTTP-REQUEST #<URI http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1> :REDIRECT 4 :STREAM NIL :ADDITIONAL-HEADERS NIL :PARAMETERS (("s" . "XIC.TO") ("f" . "sxl1d1t1"))) <1 DRAKMA:HTTP-REQUEST returned 7 values : <1 #(34 88 73 67 46 84 79 34 44 34 84 111 114 111 110 116 111 34 44 49 55 46 57 56 44 34 49 48 .......... <1 200 <1 ((:DATE . "Wed, 14 Oct 2009 05:00:51 GMT") (:P3P . "policyref=\"http://p3p.yahoo.com/w3c/p3p.xml\", CP=\"CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV\"") (:CACHE-CONTROL . "private") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "application/octet-stream")) <1 #<URI http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1&s=XIC.TO&f=sxl1d1t1> <1 #<FLEXI-STREAMS:FLEXI-IO-STREAM #x15616B76> <1 T <1 "OK" Is it what may be expected from such usage of :PARAMETERS in GET requests? If not, is it worth fixing it? - Eugene
Thanks for the report and sorry for the delay. I've committed a fix to the BKNR repository. Edi. On Sat, Oct 17, 2009 at 7:12 AM, Eugene Ossintsev <eugoss@gmail.com> wrote:
I use DRAKMA 1.0, Clozure CL 1.4-dev (Ubuntu Linux x86). According to the DRAKMA documentation, parameters is an alist of name/value pairs (the car and the cdr each being a string) which denotes the parameters which are added to the query part of the URI or (in the case of a POST request) comprise the request body.
In my example QUOTES1 sends a GET request passing the request parameters in the URI, and QUOTES2 uses the :PARAMETERS instead. The results, unfortunately, are not the same.
(defun quotes1 () (multiple-value-bind (body code) (drakma:http-request "http://finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1") (when (= code 200) (split-sequence:split-sequence #\, body))))
("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M ") <=== It's fine :-)
(defun quotes2 () (multiple-value-bind (body code) (drakma:http-request "http://finance.yahoo.com/d/quotes.csv" :method :get :parameters '(("s" . "XIC.TO") ("f" . "sxl1d1t1"))) (when (= code 200) (split-sequence:split-sequence #\, body))))
("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M \"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M ") <=== It's a mess :-(
TRACE DRAKMA:HTTP-REQUEST shows that in case of redirects of GET requests the parameters from :PARAMETERS are added again to the URI. Thus, the same parameters may be added again and again, giving unpredictable results.
0> Calling (DRAKMA:HTTP-REQUEST "http://finance.yahoo.com/d/quotes.csv" :PARAMETERS (("s" . "XIC.TO") ("f" . "sxl1d1t1"))) 1> Calling (DRAKMA:HTTP-REQUEST #<URI http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1> :REDIRECT 4 :STREAM NIL :ADDITIONAL-HEADERS NIL :PARAMETERS (("s" . "XIC.TO") ("f" . "sxl1d1t1"))) <1 DRAKMA:HTTP-REQUEST returned 7 values : <1 #(34 88 73 67 46 84 79 34 44 34 84 111 114 111 110 116 111 34 44 49 55 46 57 56 44 34 49 48 .......... <1 200 <1 ((:DATE . "Wed, 14 Oct 2009 05:00:51 GMT") (:P3P . "policyref=\"http://p3p.yahoo.com/w3c/p3p.xml\", CP=\"CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV\"") (:CACHE-CONTROL . "private") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "application/octet-stream")) <1 #<URI http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1&s=XIC.TO&f=sxl1d1t1> <1 #<FLEXI-STREAMS:FLEXI-IO-STREAM #x15616B76> <1 T <1 "OK"
Is it what may be expected from such usage of :PARAMETERS in GET requests? If not, is it worth fixing it?
- Eugene
_______________________________________________ drakma-devel mailing list drakma-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
Thanks a lot, Edi! Now it works fine :) - Eugene On Tue, Nov 10, 2009 at 4:00 AM, Edi Weitz <edi@agharta.de> wrote:
Thanks for the report and sorry for the delay. I've committed a fix to the BKNR repository.
Edi.
On Sat, Oct 17, 2009 at 7:12 AM, Eugene Ossintsev <eugoss@gmail.com> wrote:
I use DRAKMA 1.0, Clozure CL 1.4-dev (Ubuntu Linux x86). According to the DRAKMA documentation, parameters is an alist of name/value pairs (the car and the cdr each being a string) which denotes the parameters which are added to the query part of the URI or (in the case of a POST request) comprise the request body.
In my example QUOTES1 sends a GET request passing the request parameters in the URI, and QUOTES2 uses the :PARAMETERS instead. The results, unfortunately, are not the same.
(defun quotes1 () (multiple-value-bind (body code) (drakma:http-request "http://finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1") (when (= code 200) (split-sequence:split-sequence #\, body))))
("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M ") <=== It's fine :-)
(defun quotes2 () (multiple-value-bind (body code) (drakma:http-request "http://finance.yahoo.com/d/quotes.csv" :method :get :parameters '(("s" . "XIC.TO") ("f" . "sxl1d1t1"))) (when (= code 200) (split-sequence:split-sequence #\, body))))
("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M \"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M ") <=== It's a mess :-(
TRACE DRAKMA:HTTP-REQUEST shows that in case of redirects of GET requests the parameters from :PARAMETERS are added again to the URI. Thus, the same parameters may be added again and again, giving unpredictable results.
0> Calling (DRAKMA:HTTP-REQUEST "http://finance.yahoo.com/d/quotes.csv" :PARAMETERS (("s" . "XIC.TO") ("f" . "sxl1d1t1"))) 1> Calling (DRAKMA:HTTP-REQUEST #<URI http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1> :REDIRECT 4 :STREAM NIL :ADDITIONAL-HEADERS NIL :PARAMETERS (("s" . "XIC.TO") ("f" . "sxl1d1t1"))) <1 DRAKMA:HTTP-REQUEST returned 7 values : <1 #(34 88 73 67 46 84 79 34 44 34 84 111 114 111 110 116 111 34 44 49 55 46 57 56 44 34 49 48 .......... <1 200 <1 ((:DATE . "Wed, 14 Oct 2009 05:00:51 GMT") (:P3P . "policyref=\"http://p3p.yahoo.com/w3c/p3p.xml\", CP=\"CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV\"") (:CACHE-CONTROL . "private") (:CONNECTION . "close") (:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE . "application/octet-stream")) <1 #<URI http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1&s=XIC.TO&f=sxl1d1t1
<1 #<FLEXI-STREAMS:FLEXI-IO-STREAM #x15616B76> <1 T <1 "OK"
Is it what may be expected from such usage of :PARAMETERS in GET requests? If not, is it worth fixing it?
- Eugene
_______________________________________________ drakma-devel mailing list drakma-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
_______________________________________________ drakma-devel mailing list drakma-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
participants (2)
-
Edi Weitz -
Eugene Ossintsev