- drakma-devel@lists.common-lisp.net - mailman3.common-lisp.net

[drakma-devel] drakma and non-ASCII content
by Robert Brown 09 Mar '12

09 Mar '12

A co-worker of mine had some problems today using Drakma to POST a STRING containing non-ASCII characters encoded as UTF-8. He was doing something equivalent to: (http-request "http://zappa.com/favicon.ico" :method :post :content (concatenate 'string "hello" (string #\white_square) "world") :external-format-out :utf-8) which ends up setting the Content-Length header value to 11, which is the LENGTH in characters of the string being sent. The documentation says that "if content is a sequence, Drakma will use LENGTH to determine its length and will use the result for the 'Content-Length' header sent to the server," so Drakma is working as documented. Also, I think I understand why this behavior is the default. You don't want to scan a content string in order to determine what its length will be in octets once it has been encoded. My co-worker could have used a vector of type (unsigned-byte 8) to hold his UTF-8 encoded data. However, I think the current Drakma behavior may be a mistake. People who want high performance are probably manipulating encoded strings as vectors of (unsigned-byte 8). It's the casual users, those sending strings, who are the ones most likely to be bitten by the default behavior, but only when their strings contain non-ASCII characters. bob

2 1

[drakma-devel] http redirect following url-encodes location
by Willi Nesensohn 27 Feb '12

27 Feb '12

I'm using drakma to GET data over http. It turns out that when following redirects, drakma uses the location in the redirection header, which it url-encodes. Usually this might not be a problem, but when the location redirected to is already url-encoded this method fails. Is this intended behaviour? I'm not sure what the clean solution to this would be, as decoding the location before using it just doesn't feel right. For the lack of a better solution that's what i'm doing right now though. Any pointers in the right direction would be greatly appreciated. Best regards, Wilfried Nesensohn

1 0

[drakma-devel] A simple Cookie and viewing the actual Post
by Michael Minerva 13 Feb '12

13 Feb '12

I just started using drakma today and so far it seems really great but I had a couple quesitons. First, is it possible to send a simpler for example i want my http-post to have something like this: Cookie: SESSdc0685ed01f285dab628a3700259e6bc=rCfmWLUeugRQNAQTs14yox_DWqZyGE2fYUS8W24j4co I tried making a drakma cookie and adding it to a cookie-jar but none of that seemed to work. Also, is there a way to print the actual post I am sending so I can see what might be wrong with it? Thanks a lot, --Mike

3 3

Re: [drakma-devel] Drakma and client certificates?
by Rob Blackwell 14 Jan '12

14 Jan '12

Okay, I think it was a typo in the Drakma util.lisp file - I have issued a pull request on github to fix it. https://github.com/RobBlackwell/drakma/commit/149e445e39faa940fb211b7dd6271… Thanks, Rob.

1 0

[drakma-devel] Drakma and client certificates?
by Rob Blackwell 14 Jan '12

14 Jan '12

Hi, I'm trying to use client certificates with Drakma to call an existing web based API. I have the latest code from https://github.com/edicl/drakma installed in my quicklisp local-projects directory. I'm using SBCL on OS X and Linux. The client certificate is a .pfx file, and I've converted it to .pem format using openssl pkcs12 -in robblackwellmanage.pfx -out robblackwellmanage.pfx.pem I have verfied that this works as a server certifcate, using hunchentoot, thus: (hunchentoot:start (make-instance 'hunchentoot:ssl-acceptor :ssl-privatekey-file "/Users/reb/certs/robblackwellmanage.pfx.pem" :ssl-certificate-file "/Users/reb/certs/robblackwellmanage.pfx.pem" :ssl-privatekey-password "password" :port 4343)) When I try to use this as a client certificate: (drakma:http-request resource :certificate "/Users/reb/certs/robblackwellmanage.pfx.pem" :key "/Users/reb/certs/robblackwellmanage.pfx.pem" :certificate-password "password" :method :get :content "" :content-type "application/xml" :additional-headers headers) I get the following error SSL initialization error: Can't load certificate passwordSSL error queue is empty. [Condition of type CL+SSL::SSL-ERROR-INITIALIZE] Interestingly, if I deliberately supply the wrong password then I get a different error SSL initialization error: Can't load RSA private key file /Users/reb/certs/robblackwellmanage.pfx.pemSSL error queue is empty. [Condition of type CL+SSL::SSL-ERROR-INITIALIZE] I guess I'm doing something really silly - please excuse my ignorance, but any advice, suggestions or guidance on how to proceed would be very much appreciated. Thanks! Rob

1 0

[drakma-devel] cl-oauth and drakma interoperability issue
by mohegan-skunkworks 08 Jan '12

08 Jan '12

I just filled an issue on github... The text is below... Hi, I'm using cl-oauth to authenticate with twitter (it's in the cl-twitter package, also on github). I received a complaint from a quicklisp user that he couldn't authenticate with twitter. After looking into this a bit I realized that drakma is encoding the uri on get requests. Now, cl-oauth passes in an encoded uri so this uri gets encoded twice. This was never a problem for me because I was using v 1.2.3 of drakma where this didn't happen. Here's the code diff (plus my proposed change :) ========================== v 1.2.3 ================================== (when (and (not parameters-used-p) parameters) (setf (uri-query uri) ;; append parameters to existing query of URI (format nil "~@[~A~]~:*~:[~;&~]~A" (uri-query uri) (alist-to-url-encoded-string parameters external-format-out)))) ===========================current github/v1.2.4============= (when-let (all-get-parameters (append (dissect-query (uri-query uri)) (and (not parameters-used-p) parameters))) (setf (uri-query uri) (alist-to-url-encoded-string all-get-parameters external-format-out))) ===============================my proposed change=================================================== (when (and (not parameters-used-p) parameters) (when-let (all-get-parameters (append (dissect-query (uri-query uri)) (and (not parameters-used-p) parameters))) (setf (uri-query uri) (alist-to-url-encoded-string all-get-parameters external-format-out)))) A few more comments : This is the hand-off in cl-oauth : (defun http-request (uri &key (request-method :get) parameters drakma-args) ;; TODO handle redirects properly (let* ((param-string-encoded (alist->query-string parameters :include-leading-ampersand nil :url-encode t))) (case request-method (:get (apply #'drakma:http-request (uri-with-additional-query-part uri param-string-encoded) :method request-method drakma-args)) (:post (apply #'drakma:http-request uri :method request-method :content param-string-encoded drakma-args)) As you can see url-encode is set to t. That was because (I think !) previous versions required encoding and drakma wasn't providing any. Now, ideally tis flag s/b nil. However the issue the becomes the string splitting in dissect-query. This splits on "=" which is also the terminating symbol for the authentication string... =======================session=================== (drakma::split-string "oauth_signature=oq37d1/qm[....]fIKb778=&include_entities=T&oauth_consumer_key=9[....]cYBg&oauth_token=206[...]Tt5SwRvCJqQWgR3ajEQpk&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1325002586&oauth_nonce=3680613621135035286&oauth_version=1.0" "&") ("oauth_signature=oq37d1/qmFX0YuQUwxsgfIKb778=" "include_entities=T" [....]) CL-USER> (drakma::split-string "oauth_signature=oq37d1/qmFX0YuQUwxsgfIKb778=" "=") ; compiling (DEFUN HTTP-REQUEST ...) STYLE-WARNING: redefining DRAKMA:HTTP-REQUEST in DEFUN ("oauth_signature" "oq37d1/qm[...]778") (I've elided some of the strings...). I think my proposal resolves the issue. However, it would require a bit more work to get to what I think is the 'right' solution... At this stage I'd like to get some feedback on whta you consider the right course of action before proceeding to submit a patch

3 7

[drakma-devel] HTTP Put and form-data
by Ben Holm 13 Nov '11

13 Nov '11

Hi all. I started using drakma the other day to do some client work with reviewboard, and I had a little trouble. Reviewboard requires http puts using form-data for updates ( http://www.reviewboard.org/docs/manual/1.6/webapi/2.0/overview/) Drakma explicitly disallows this. I modified request.lisp to let my client do what I needed. Since this is my first time writing a webservice client, I'm not sure if reviewboard or drakma is wrong, or if I misunderstood something. Below is the diff of what I did. Any thoughts? Thanks, -Ben Index: request.lisp =================================================================== --- request.lisp (revision 4683) +++ request.lisp (working copy) @@ -410,7 +410,7 @@ (parameter-error "Don't know how to handle scheme ~S." (uri-scheme uri))) (when (and close keep-alive) (parameter-error "CLOSE and KEEP-ALIVE must not be both true.")) - (when (and form-data (not (eq method :post))) + (when (and form-data (and (not (eq method :put)) (not (eq method :post)))) (parameter-error "FORM-DATA makes only sense with POST requests.")) (when range (unless (and (listp range) @@ -432,7 +432,7 @@ (when (and file-parameters-p (not (eq method :post))) (parameter-error "Don't know how to handle parameters in ~S, as this is not a POST request." parameters)) - (when (eq method :post) + (when (or (eq method :put) (eq method :post)) ;; create content body for POST unless it was provided (unless content ;; mark PARAMETERS argument as used up, so we don't use it

1 0

[drakma-devel] My open source libraries (aka "ediware")
by Edi Weitz 31 Aug '11

31 Aug '11

[My apologies if you receive multiple copies of this email.] Hi everybody! As some of you will know, I'll start on a new job tomorrow. This new job won't involve much hacking, if at all, and thus it doesn't look like I'll have a lot of time to maintain my open source libraries in the near future. I have no plans to suddenly disappear from the CL world, but don't expect new releases of any of my libs any time soon. (At least none published by me or on my server.) Luckily, Hans Hübner - who already did most of the maintenance and development work for Hunchentoot in the last two years or so - offered to coordinate further development via github. See his full announcement at <http://netzhansa.blogspot.com/2011/08/ediware-moving-to-github.html>. I'll continue to read the mailing lists for my libs and I'm still interested in fixing bugs you might find in the release tarballs available on my web server. However, I will likely not bother to discuss or work on new features or compatibility code for implementations other than LispWorks (which happens to be the one I'm using). Lastly, I hope to see a lot of you in Amsterdam <http://weitz.de/eclm2011/> in October. The number of registrations so far has been pretty disappointing, but you still have three weeks left to change your mind... :) Take care, Edi.

1 0

[drakma-devel] [patch] Resubmit drakma timeout for sbcl.
by Александр Литвинов 29 Mar '11

29 Mar '11

A long time ago (Dec 2007) there was a patch [1] for drakma to implement timeouts with sbcl. It was rejected with suggestion to post similar patch into usocket library. Totday, I start to learn lisp programing and I need http-request for my small program. I took drakma for this. But it does not support timeouts on sbcl. I use debian and it seems sbcl - is the best choice for me. By googling I found a patch for drakma. I repplaied it for 1.2.3 drakma. It seems to me, it still forth appling into main line of drakma. I did not investigate what happen with usocket patch and such long time shows - this patch needs to be applied anyway :-) 1. http://common-lisp.net/pipermail/drakma-devel/2007-December/000601.html -- Alexander Litvinov

6 21

[drakma-devel] read-status-line is seeing the HTTP request line!?
by Daniel Weinreb 17 Mar '11

17 Mar '11

Has anybody seen a behavior in which read-status-line signals a syntax-error, and it turns out that the value of "line" looks like an HTTP request line rather than an HTTP status (response) line? The following is the stack trace. From my old C++ days (may they never return), I conjectured that some string buffer is being reused or something like that.; We are running on CCL (formerly openmcl), and nothing like that is going on at the CCL level.; I don't know much about flexi-streams or gray-streams. Anyway that's just a rather wild guess. Thanks! <READ-STATUS-LINE at pc +669 Arguments: (STREAM &OPTIONAL DRAKMA::LOG-STREAM) STREAM: <FLEXI-STREAMS:FLEXI-IO-STREAM #x30200938EB3D> DRAKMA::LOG-STREAM: NIL Local bindings: CHUNGA:CURRENT-ERROR-MESSAGE "While reading status line:"" DRAKMA::LINE: ""POST /stat/ping HTTP/1.1"" DRAKMA::FIRST-SPACE-POS: 4 DRAKMA::SECOND-SPACE-POS: 15 Thanks! - Dan

3 3