March 2012 - drakma-devel@common-lisp.net

[drakma-devel] Changing content-type seems to have no effect
by Michael Minerva 15 Mar '12

15 Mar '12

I have been trying to get my lisp application to interface and upload a file to my drupal 7 website using Drakma. So far I have been able to to connect to my drupal site and authenticate and store the session cookie (thank you Ryan Davis, I just found your response the other day). Here is the code I use to do that: (with-output-to-string (stream) (setf drakma:*header-stream* stream) (let ((cookie-jar (make-instance 'drakma:cookie-jar))) (drakma:http-request "http://localhost:8888/?q=rest/user/login" :method :post :cookie-jar cookie-jar :parameters '(("username" . "login") ("password" . "password")) ) (drakma:http-request "http://localhost:8888/?q=rest/file" :method :post :cookie-jar cookie-jar :content-length t :parameters '(("file1" . #p"/Users/Mike/hello.png"));("file1" #p"/Users/Mike/Desktop/mario-gif.gif" :content-type "image/gif" :filename "mario-gif.gif")) :content-type "application/x-www-form-urlencoded" ;:parameters ))) This code successfully sends both posts but the second post does not actually upload the file (but it does return 200 OK, no errors). I have successfully uploaded to by drupal site using PHP Curl and in that code, I was only able to get it to work by using a content type of application/x-www-form-urlencoded (if I use a multipart/form-data request on my php page I get similar behavior to what I am now seeing with Drakma). This is why I added the line :content-type "application/x-www-form-urlencoded" to my second request (posted above) but it seems to have no effect. Here is a copy of the posts I am making (as you can see in the second post it still uses multipart/form-data. Is there anyway that I can upload a file using Drakma using a content-type of application/x-www-form-urlencoded (I know this is not the recommended way of making this type of request but I think it may be the only way that my drupal site will accept the file). "POST /?q=rest/user/login HTTP/1.1 Host: localhost:8888 User-Agent: Drakma/1.2.3 (Clozure Common Lisp Version 1.7-r14927M (DarwinX8664); Darwin; 10.8.0; http://weitz.de/drakma/) Accept: */* Connection: close Content-Type: application/x-www-form-urlencoded Content-Length: 34 HTTP/1.1 200 OK Date: Thu, 15 Mar 2012 17:49:51 GMT Server: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 PHP/5.3.6 X-Powered-By: PHP/5.3.6 Expires: Sun, 19 Nov 1978 05:00:00 GMT Last-Modified: Thu, 15 Mar 2012 17:49:51 +0000 Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0 ETag: \"1331833791\" Vary: Accept Set-Cookie: SESSdc0685ed01f285dab628a3700259e6bc=oIJFlVrfoXi-PO7bdwKtauKfTeRSEK1iIucj0tl12UQ; expires=Sat, 07-Apr-2012 21:23:12 GMT; path=/; HttpOnly Content-Length: 115 Connection: close Content-Type: text/yaml POST /?q=rest/file HTTP/1.1 Host: localhost:8888 User-Agent: Drakma/1.2.3 (Clozure Common Lisp Version 1.7-r14927M (DarwinX8664); Darwin; 10.8.0; http://weitz.de/drakma/) Accept: */* Cookie: SESSdc0685ed01f285dab628a3700259e6bc=oIJFlVrfoXi-PO7bdwKtauKfTeRSEK1iIucj0tl12UQ Connection: close Content-Type: multipart/form-data; boundary=----------ndnZ9xjDrDpzStPEQo97xwqPHEKXAhAOd8Ho6C8P3jtKbvNINm Content-Length: 244958 HTTP/1.1 200 OK Date: Thu, 15 Mar 2012 17:49:52 GMT Server: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8r DAV/2 PHP/5.3.6 X-Powered-By: PHP/5.3.6 Expires: Sun, 19 Nov 1978 05:00:00 GMT Last-Modified: Thu, 15 Mar 2012 17:49:52 +0000 Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0 ETag: \"1331833792\" Vary: Accept Content-Length: 4 Connection: close Content-Type: text/yaml " Thanks, --Michael

2 2

[drakma-devel] Unwanted url-encoding of GET parameters.
by William Halliburton 11 Mar '12

11 Mar '12

Hello folks, I'm trying to use drakma to fetch urls that contain utf8 characters but HTTP-REQUEST automatically url encodes any non latin-1 ascii characters. On my cursory reading of the RFCs, this seems conforming behavor, but in this case it is definitely unwanted. For example, the following url http://translate.google.com/translate_tts?tl=ru&q=вы if entered directly into the browser correctly returns the text-to-speech audo file but when attempting to use HTTP-REQUEST, the url is being url encoded into http://translate.google.com/translate_tts?tl=ru&q=%D0%B2%D1%8B of which google does not url-decode and fails to return the correct data. So, for this case, the url-encoding is unwanted. I am willing to submit patch an additional argument into HTTP-REQUEST to disallow the encoding. Thoughts? Thank you, William

3 3

[drakma-devel] drakma and non-ASCII content
by Robert Brown 09 Mar '12

09 Mar '12

A co-worker of mine had some problems today using Drakma to POST a STRING containing non-ASCII characters encoded as UTF-8. He was doing something equivalent to: (http-request "http://zappa.com/favicon.ico" :method :post :content (concatenate 'string "hello" (string #\white_square) "world") :external-format-out :utf-8) which ends up setting the Content-Length header value to 11, which is the LENGTH in characters of the string being sent. The documentation says that "if content is a sequence, Drakma will use LENGTH to determine its length and will use the result for the 'Content-Length' header sent to the server," so Drakma is working as documented. Also, I think I understand why this behavior is the default. You don't want to scan a content string in order to determine what its length will be in octets once it has been encoded. My co-worker could have used a vector of type (unsigned-byte 8) to hold his UTF-8 encoded data. However, I think the current Drakma behavior may be a mistake. People who want high performance are probably manipulating encoded strings as vectors of (unsigned-byte 8). It's the casual users, those sending strings, who are the ones most likely to be bitten by the default behavior, but only when their strings contain non-ASCII characters. bob

2 1