October 2011 - tbnl-devel@common-lisp.net

[hunchentoot-devel] Hunchentoot v1.2.0 released
by Hans Hübner 05 Nov '11

05 Nov '11

After over a year of no releases, Hunchentoot v1.2.0 is out. It incorporates a large number of changes from various people and is the first release that I have made as the maintainer of Hunchentoot. There have been a fair number of changes, many of them contributed by Hunchentoot's users. Thank you all for that! The release tarball is available here: https://github.com/downloads/edicl/hunchentoot/hunchentoot-1.2.0.tar.gz This is my first release, and as such it is likely that I'm not yet up to the release standards that Edi has set forth. Please report any problems you may have with the new version to the Hunchentoot mailing list. I will try to resolve problems quickly and possible make a new minor release for each major problem found. The development repository for Hunchentoot is now hosted by github (https://github.com/edicl/hunchentoot) If you want to contribute, it would be appreciated if you could submit your patches through github, too. Here is a list of changes: Add www/ directory with default file tree that is being served Add error template mechanism and improve error reporting in general. Improve automatic testing, SBCL kludge to support asdf:test-op Allegro CL modern mode fixes Fix bugs in serving partial responses Limit maximum number of threads that Hunchentoot creates (Dan Weinreb, Scott McKay) Export fixes (Gordon Sims, Andrey Moskvitin) Factor out easy-handler logic into separate acceptor subclass Export two session functions (Nico de Jager) Allow no Content-Type header (Chaitanya Gupta) Patch for compilation with ECL (Sohail Somani) Fix DEFINE-EASY-HANDLER for multiple acceptors (Nicolas Neuss) Revived *SHOW-LISP-BACKTRACES-P* Made sure "100 Continue" is returned even if the client sends "Expect: 100-continue" twice (reported by Gordon Sims) Fixed typo in code which interprets transfer encodings Thank you for your patience! Hans

3 4

Re: [hunchentoot-devel] [hunchentoot] wrong character set in parsing request (#1)
by John DeSoi 31 Oct '11

31 Oct '11

Since this issue has been closed on github, I'd like to repost to the list and get some feedback. It still seems like a bug to me. > An incoming request has > > Content-Type: application/json; charset=UTF-8 > > but the server treats it as latin-1 (the default). It seems the code correctly parses the character set from the content type, but ignores it unless the type is "text". See parse-content-type. On Oct 30, 2011, at 4:29 PM, Hans Hübner wrote: > application/json is always encoded as utf-8, at least that is what I understood so far. > So in order to correctly handle a application/json request, I have to explicitly control the request parsing, even though the request has already specified it is UTF-8. (defhandler (rpc-handler :uri "/rpc" :default-request-type :post) () (setf (tbnl:content-type*) "application/json") (setf (tbnl:reply-external-format*) +utf-8+) (invoke-rpc (tbnl:raw-post-data :external-format +utf-8+))) The parse-content-type function parses the "charset" part, but then completely ignores it if the mime type does not have "text" in it. If the content type had been "text/json", it would have worked correctly. John DeSoi, Ph.D.

2 4

[hunchentoot-devel] Hunchentoot support for ECL
by Olivier Dumont 30 Oct '11

30 Oct '11

Hi, the attached patch allows hunchentoot to function properly with the latest release of ECL (11.1.1). best regards, Olivier

2 1

[hunchentoot-devel] Hunchentoot Errors (e.g. Couldn't write to #<SB-SYS:FD-STREAM for "socket ..." ...>)
by Cyrus Harmon 30 Oct '11

30 Oct '11

Hello tbnl-folk, So, I've been seeing transient problems with hunchentoot for as long as I can remember, and I haven't had the chance to try to track down the problems until now. It seems there are a couple of classes of problems. In order of severity, the are: 1. things that crash hunchentoot/SBCL (or, in some cases, the host OS) 2. things that throw "unexpected" errors 3. things that throw "expected" errors (e.g. 404) As for class 1, these are tough to track down as the server is usually crashed by the time I get to it, but some causes I consider plausible are: * running out of some limited resource such as RAM, threads, or network sockets * possible SBCL bug in, say, signal handling or dealing with lots of threads I don't have any data to present on this class, at the moment, so let's move on to the next class. I often see errors such as this in the log: [2011-04-05 00:04:52 [ERROR]] Couldn't write to #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>: Broken pipe 0: (SB-DEBUG::MAP-BACKTRACE #<CLOSURE (LAMBDA #) {B71EC4D}> :START 0 :COUNT 536870911) 1: (BACKTRACE 536870911 #<SB-IMPL::STRING-OUTPUT-STREAM {B71EBE9}>) 2: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM #<SB-IMPL::STRING-OUTPUT-STREAM {B71EBE9}>) 3: (HUNCHENTOOT::GET-BACKTRACE) 4: ((FLET #:LAMBDA271) #<SB-INT:SIMPLE-STREAM-ERROR "~@<~?: ~2I~_~A~:>" {B71EA51}>) 5: (SIGNAL #<SB-INT:SIMPLE-STREAM-ERROR "~@<~?: ~2I~_~A~:>" {B71EA51}>) 6: (ERROR SB-INT:SIMPLE-STREAM-ERROR :STREAM #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}> :FORMAT-CONTROL "~@<~?: ~2I~_~A~:>" :FORMAT-ARGUMENTS ("Couldn't write to ~S" (#<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) "Broken pipe")) 7: (SB-IMPL::SIMPLE-STREAM-PERROR "Couldn't write to ~S" #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}> 32) 8: (SB-IMPL::WRITE-OUTPUT-FROM-QUEUE #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) 9: (SB-IMPL::SUB-SUB-SERVE-EVENT NIL NIL) 10: (SB-IMPL::SUB-SERVE-EVENT NIL NIL NIL) 11: (SB-SYS:SERVE-ALL-EVENTS NIL) 12: (SB-IMPL::FINISH-FD-STREAM-OUTPUT #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) 13: (FINISH-OUTPUT #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) 14: (HUNCHENTOOT:HANDLE-STATIC-FILE #P"/home/sly/projects/www-cyrusharmon-org/static/images/fish-clusters.png" NIL) 15: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-REQUEST (HUNCHENTOOT:ACCEPTOR HUNCHENTOOT:REQUEST)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:ACCEPTOR (host *, port 80)> #<HUNCHENTOOT:REQUEST {B701D09}>) 16: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-REQUEST (T)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:REQUEST {B701D09}>) 17: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION (HUNCHENTOOT:ACCEPTOR T)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:ACCEPTOR (host *, port 80)> #<USOCKET:STREAM-USOCKET {B6FB8F1}>) 18: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION :AROUND (HUNCHENTOOT:ACCEPTOR T)) #<unavailable argument> #S(SB-PCL::FAST-METHOD-CALL :FUNCTION #<FUNCTION #> :PV NIL :NEXT-METHOD-CALL NIL :ARG-INFO (2)) #<HUNCHENTOOT:ACCEPTOR (host *, port 80)> #<USOCKET:STREAM-USOCKET {B6FB8F1}>) 19: ((LAMBDA ())) 20: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK353]358)) 21: ((FLET SB-THREAD::WITH-MUTEX-THUNK)) 22: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]300)) 23: (SB-THREAD::CALL-WITH-MUTEX #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK) {B0E14235}> #S(SB-THREAD:MUTEX :NAME "thread result lock" :%OWNER #<SB-THREAD:THREAD "Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING {B6FCE39}> :STATE 1) #<SB-THREAD:THREAD "Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING {B6FCE39}> T) 24: (SB-THREAD::INITIAL-THREAD-FUNCTION) 25: ("foreign function: call_into_lisp") 26: ("foreign function: funcall0") 27: ("foreign function: new_thread_trampoline") 28: ("foreign function: #xB7FC5506") What does this whole "broken pipe" mean? Is this more of an "expected error" or is something going wrong here? On a slightly different note, I'd like to be able to run a relatively reliable hunchentoot server that can serve a modest traffic load without having to resort to caching proxy servers and the like. I recall some discussion of thread pools, epoll, etc... Has the state of the art for high-reliability hunchentoot instances changed in the last couple years? Any suggestions on improving it? thanks! Cyrus

2 1

[hunchentoot-devel] Garbage in REPL
by Andrey Moskvitin 30 Oct '11

30 Oct '11

Hi, For ACCESS-LOG-DESIGNATION default value is *ERROR-OUTPUT*. It is the cause of littering REPL for SLIME users. Maybe better use NIL as default and for suppress logging use a keyword :suppress? (defmethod acceptor-access-log-destination ((acceptor acceptor)) (let ((destination (slot-value acceptor 'access-log-destination))) (case destination ((nil) *error-output*) (:suppress nil) (otherwise destination)))) In this case, the logging will be sent to *inferior-lisp* and will not interfere with. Andrey

2 1

[hunchentoot-devel] problem with the hunchentoots handler for POST methods
by Farzad Bekran 29 Oct '11

29 Oct '11

hi, i get this error when i send a page that has a form on it that uses POST method, if i change the method to GET it works fine tho. and the POST test on the hunchentoot's own test site gives me this too. here is the stack trace: #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}> is not a binary input stream. [Condition of type SIMPLE-TYPE-ERROR] Restarts: 0: [TERMINATE-THREAD] Terminate this thread (#<THREAD "hunchentoot-worker-3" RUNNING {ED3B5D1}>) Backtrace: 0: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGING-ENVIRONMENT) #<FUNCTION (LAMBDA #) {B8F5345}>) 1: (SWANK::DEBUG-IN-EMACS #<SIMPLE-TYPE-ERROR expected-type: (SATISFIES INPUT-STREAM-P) datum: #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>>) Locals: SB-DEBUG::ARG-0 = #<SIMPLE-TYPE-ERROR expected-type: (SATISFIES INPUT-STREAM-P) ..> 2: (SWANK-BACKEND::CALL-WITH-BREAK-HOOK #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {F0BC495}>) 3: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK) #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {F0BC495}>) 4: (SWANK::CALL-WITH-BINDINGS ((*STANDARD-OUTPUT* . #) (*STANDARD-INPUT* . #) (*TRACE-OUTPUT* . #) (*ERROR-OUTPUT* . #) (*DEBUG-IO* . #) (*QUERY-IO* . #) ...) #<CLOSURE (LAMBDA #) {F0BC4A5}>) 5: (SWANK:INVOKE-SLIME-DEBUGGER #<SIMPLE-TYPE-ERROR expected-type: (SATISFIES INPUT-STREAM-P) datum: #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>>) 6: (SWANK-BACKEND::CALL-WITH-BREAK-HOOK #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {F0BC475}>) 7: ((FLET SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK) #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA #) {F0BC475}>) 8: (SWANK:SWANK-DEBUGGER-HOOK #<SIMPLE-TYPE-ERROR expected-type: (SATISFIES INPUT-STREAM-P) datum: #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>> #<unavailable argument>) 9: (SB-DEBUG::RUN-HOOK *DEBUGGER-HOOK* #<SIMPLE-TYPE-ERROR expected-type: (SATISFIES INPUT-STREAM-P) datum: #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>>) 10: (INVOKE-DEBUGGER #<SIMPLE-TYPE-ERROR expected-type: (SATISFIES INPUT-STREAM-P) datum: #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>>) 11: ((SB-PCL::FAST-METHOD HUNCHENTOOT:MAYBE-INVOKE-DEBUGGER (T)) ..) 12: (SIGNAL #<SIMPLE-TYPE-ERROR expected-type: (SATISFIES INPUT-STREAM-P) datum: #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>>) 13: (ERROR ..) 14: (SB-KERNEL:ILL-BIN #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>) 15: (READ-BYTE #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}> NIL NIL) 16: (CHUNGA:READ-CHAR* #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}> NIL NIL) 17: (CHUNGA:PEEK-CHAR* #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}> NIL NIL) 18: (CHUNGA:READ-TOKEN #<SB-IMPL::STRING-INPUT-STREAM {F0BC191}>) 19: (HUNCHENTOOT::PARSE-CONTENT-TYPE "application/x-www-form-urlencoded" T) 20: ((SB-PCL::FAST-METHOD INITIALIZE-INSTANCE :AFTER (HUNCHENTOOT::REQUEST)) #<unused argument> #<unused argument> #<HUNCHENTOOT::REQUEST {F0BBF91}>) 21: ((LAMBDA (SB-PCL::|.P0.| SB-PCL::|.P1.| SB-PCL::|.P2.| SB-PCL::|.P3.| SB-PCL::|.P4.|)) ..) 22: (HUNCHENTOOT::PROCESS-REQUEST ..) 23: (HUNCHENTOOT::PROCESS-CONNECTION #<HUNCHENTOOT::SERVER {D03A0F1}> #<SB-BSD-SOCKETS:INET-SOCKET 127.0.0.1:4242, peer: 127.0.0.1:35690, fd: 7 {ED3B089}>) 24: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK405]410)) 25: ((FLET SB-THREAD::WITH-MUTEX-THUNK)) 26: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]309)) 27: (SB-THREAD::CALL-WITH-MUTEX ..) 28: (SB-THREAD::INITIAL-THREAD-FUNCTION) 29: ("foreign function: call_into_lisp") 30: ("foreign function: funcall0") 31: ("foreign function: new_thread_trampoline") 32: ("foreign function: #xB7FB6E99")

2 2

[hunchentoot-devel] A few patches from ITA
by Faré 27 Oct '11

27 Oct '11

I have been merging the latest hunchentoot into our (Google/ITA) tree, and I have a few patches. Is this list still the place to exchange patches, or should I be sending github pull requests instead? My attached patches are: * some package tweaks (symbols not exported that should, symbols exported that are no more) * patch to make-cooked-message so it works for 503 and other messages not in the case shortlist. * patch for send-service-unavailable-reply (my "solution" is horrible, someone who understands hunchentoot internals better might know what the "right thing" is). Not included: * a patch for xcvb support, but I believe you're not interested. * some debugging code we use to track instabilities that we've observed in our farms. Of course, I'm willing to massage the code until it is satisfactory to you who maintain the upstream tree. Just tell me what I need to change so it's commit-worthy. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org

2 1