July 2011 - tbnl-devel@common-lisp.net - mailman3.common-lisp.net

[hunchentoot-devel] Hunchentoot support for ECL
by Olivier Dumont 30 Oct '11

30 Oct '11

Hi, the attached patch allows hunchentoot to function properly with the latest release of ECL (11.1.1). best regards, Olivier

2 1

[hunchentoot-devel] Hunchentoot Errors (e.g. Couldn't write to #<SB-SYS:FD-STREAM for "socket ..." ...>)
by Cyrus Harmon 30 Oct '11

30 Oct '11

Hello tbnl-folk, So, I've been seeing transient problems with hunchentoot for as long as I can remember, and I haven't had the chance to try to track down the problems until now. It seems there are a couple of classes of problems. In order of severity, the are: 1. things that crash hunchentoot/SBCL (or, in some cases, the host OS) 2. things that throw "unexpected" errors 3. things that throw "expected" errors (e.g. 404) As for class 1, these are tough to track down as the server is usually crashed by the time I get to it, but some causes I consider plausible are: * running out of some limited resource such as RAM, threads, or network sockets * possible SBCL bug in, say, signal handling or dealing with lots of threads I don't have any data to present on this class, at the moment, so let's move on to the next class. I often see errors such as this in the log: [2011-04-05 00:04:52 [ERROR]] Couldn't write to #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>: Broken pipe 0: (SB-DEBUG::MAP-BACKTRACE #<CLOSURE (LAMBDA #) {B71EC4D}> :START 0 :COUNT 536870911) 1: (BACKTRACE 536870911 #<SB-IMPL::STRING-OUTPUT-STREAM {B71EBE9}>) 2: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM #<SB-IMPL::STRING-OUTPUT-STREAM {B71EBE9}>) 3: (HUNCHENTOOT::GET-BACKTRACE) 4: ((FLET #:LAMBDA271) #<SB-INT:SIMPLE-STREAM-ERROR "~@<~?: ~2I~_~A~:>" {B71EA51}>) 5: (SIGNAL #<SB-INT:SIMPLE-STREAM-ERROR "~@<~?: ~2I~_~A~:>" {B71EA51}>) 6: (ERROR SB-INT:SIMPLE-STREAM-ERROR :STREAM #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}> :FORMAT-CONTROL "~@<~?: ~2I~_~A~:>" :FORMAT-ARGUMENTS ("Couldn't write to ~S" (#<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) "Broken pipe")) 7: (SB-IMPL::SIMPLE-STREAM-PERROR "Couldn't write to ~S" #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}> 32) 8: (SB-IMPL::WRITE-OUTPUT-FROM-QUEUE #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) 9: (SB-IMPL::SUB-SUB-SERVE-EVENT NIL NIL) 10: (SB-IMPL::SUB-SERVE-EVENT NIL NIL NIL) 11: (SB-SYS:SERVE-ALL-EVENTS NIL) 12: (SB-IMPL::FINISH-FD-STREAM-OUTPUT #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) 13: (FINISH-OUTPUT #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>) 14: (HUNCHENTOOT:HANDLE-STATIC-FILE #P"/home/sly/projects/www-cyrusharmon-org/static/images/fish-clusters.png" NIL) 15: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-REQUEST (HUNCHENTOOT:ACCEPTOR HUNCHENTOOT:REQUEST)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:ACCEPTOR (host *, port 80)> #<HUNCHENTOOT:REQUEST {B701D09}>) 16: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-REQUEST (T)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:REQUEST {B701D09}>) 17: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION (HUNCHENTOOT:ACCEPTOR T)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:ACCEPTOR (host *, port 80)> #<USOCKET:STREAM-USOCKET {B6FB8F1}>) 18: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION :AROUND (HUNCHENTOOT:ACCEPTOR T)) #<unavailable argument> #S(SB-PCL::FAST-METHOD-CALL :FUNCTION #<FUNCTION #> :PV NIL :NEXT-METHOD-CALL NIL :ARG-INFO (2)) #<HUNCHENTOOT:ACCEPTOR (host *, port 80)> #<USOCKET:STREAM-USOCKET {B6FB8F1}>) 19: ((LAMBDA ())) 20: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK353]358)) 21: ((FLET SB-THREAD::WITH-MUTEX-THUNK)) 22: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]300)) 23: (SB-THREAD::CALL-WITH-MUTEX #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK) {B0E14235}> #S(SB-THREAD:MUTEX :NAME "thread result lock" :%OWNER #<SB-THREAD:THREAD "Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING {B6FCE39}> :STATE 1) #<SB-THREAD:THREAD "Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING {B6FCE39}> T) 24: (SB-THREAD::INITIAL-THREAD-FUNCTION) 25: ("foreign function: call_into_lisp") 26: ("foreign function: funcall0") 27: ("foreign function: new_thread_trampoline") 28: ("foreign function: #xB7FC5506") What does this whole "broken pipe" mean? Is this more of an "expected error" or is something going wrong here? On a slightly different note, I'd like to be able to run a relatively reliable hunchentoot server that can serve a modest traffic load without having to resort to caching proxy servers and the like. I recall some discussion of thread pools, epoll, etc... Has the state of the art for high-reliability hunchentoot instances changed in the last couple years? Any suggestions on improving it? thanks! Cyrus

2 1

[hunchentoot-devel] Garbage in REPL
by Andrey Moskvitin 30 Oct '11

30 Oct '11

Hi, For ACCESS-LOG-DESIGNATION default value is *ERROR-OUTPUT*. It is the cause of littering REPL for SLIME users. Maybe better use NIL as default and for suppress logging use a keyword :suppress? (defmethod acceptor-access-log-destination ((acceptor acceptor)) (let ((destination (slot-value acceptor 'access-log-destination))) (case destination ((nil) *error-output*) (:suppress nil) (otherwise destination)))) In this case, the logging will be sent to *inferior-lisp* and will not interfere with. Andrey

2 1

[hunchentoot-devel] socket-connector and ssl-socket-connector?
by Cyrus Harmon 29 Sep '11

29 Sep '11

I've been playing around with some changes to ACCEPTOR and friends that introduce the new classes SOCKET-CONNECTOR and SSL-SOCKET-CONNECTOR. The purpose of these changes are to take the specialized socket handling behavior out of the acceptor and into another class, such that subclassing acceptor isn't required to, say, use SSL streams. As a motivating example, consider the following: (hunchentoot:start (make-instance 'hunchentoot:easy-acceptor :port 4243 :socket-connector (make-instance 'hunchentoot::ssl-socket-connector :ssl-certificate-file (asdf:system-relative-pathname :hunchentoot "ssl/certificate.pem") :ssl-privatekey-file (asdf:system-relative-pathname :hunchentoot "ssl/key.pem")))) (hunchentoot:define-easy-handler (say-yo :uri "/yo") (name) (setf (hunchentoot:content-type*) "text/plain") (format nil "Hey~@[ ~A~]!" name)) Without these changes, one would have to subclass both easy-acceptor and ssl-acceptor in order to have an SSL-using easy acceptor. Comments/suggestions greatly appreciated. The changes can be found in the git repo at: https://github.com/slyrus/hunchentoot/tree/socket-connector and in the attached patch. Cyrus

3 9

[hunchentoot-devel] Get client's time
by Matus Kmit 29 Jul '11

29 Jul '11

Hi everyone i am new to web programming and hunchentoot. i have the following question: How do one get client's time with hunchentoot? Does the http request contain this information and can one inspect on that within hunchentoot? Or what is the other way to do it? On my main page, i need to know the exact hour and minute of the client before generating the appropriate site content. Thanks Matus

2 2

[hunchentoot-devel] session-value
by Jens Teich 16 Jul '11

16 Jul '11

Trying to update a project from 0.15.6 to 1.1.1. (setf (session-value 'x) 42) throws error 'called session with unpaired keyword'. What am I missing? Jens

2 4

[hunchentoot-devel] Treten Sie meinem Netzwerk auf LinkedIn bei
by Eli Naeher über LinkedIn 13 Jul '11

13 Jul '11

LinkedIn ------------ Eli Naeher möchte Sie als Kontakt auf LinkedIn hinzufügen: ------------------------------------------ Ich möchte Sie gerne zu meinem beruflichen Netzwerk auf LinkedIn hinzufügen. Einladung von Eli Naeher annehmen http://www.linkedin.com/e/bxhy9y-gq2tbc3i-6/53mvQnCmNTJiD3hCE0Ep5J1I-StDsAz… Einladung von Eli Naeher anzeigen http://www.linkedin.com/e/bxhy9y-gq2tbc3i-6/53mvQnCmNTJiD3hCE0Ep5J1I-StDsAz… ------------------------------------------ Wussten Sie schon, dass Sie sich mit Ihrem Fachwissen auf LinkedIn profilieren können, um Stellen- bzw. Beratungsangebote zu erhalten und Ihr berufliches Ansehen zu verbessern? Beantworten Sie Fragen unter "LinkedIn Fragen & Anworten", um in der weltweiten Gemeinschaft von Fach- und Führungskräften auf sich aufmerksam zu machen. http://www.linkedin.com/e/bxhy9y-gq2tbc3i-6/abq/inv-24/ -- (c) 2011, LinkedIn Corporation

1 0

[hunchentoot-devel] Timing out when reading long post requests?
by Oleg Sivokon 07 Jul '11

07 Jul '11

Hello, I'm not sure what is causing this error (please see the stack trace below), all I can say it's relatively big XML being sent. I must be doing something tremendously inefficient or else this error means something else, but I hope you would be able to advise: [2011-07-06 14:20:27 [ERROR]] I/O timeout reading #<SB-SYS:FD-STREAM for "a socket" {B8E0AA1}>. 0: (SB-DEBUG::MAP-BACKTRACE #<CLOSURE (LAMBDA #) {B9034D5}>)[:EXTERNAL] 1: (BACKTRACE 536870911 #<SB-IMPL::STRING-OUTPUT-STREAM {B903471}>) 2: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM #<SB-IMPL::STRING-OUTPUT-STREAM {B903471}>) 3: (HUNCHENTOOT::GET-BACKTRACE) 4: ((FLET #:LAMBDA284) #<SB-SYS:IO-TIMEOUT {B903139}>) 5: (SIGNAL #<SB-SYS:IO-TIMEOUT {B903139}>)[:EXTERNAL] 6: (ERROR SB-SYS:IO-TIMEOUT)[:EXTERNAL] 7: (SB-IMPL::SIGNAL-TIMEOUT SB-SYS:IO-TIMEOUT)[:EXTERNAL] 8: (SB-IMPL::REFILL-INPUT-BUFFER #<SB-SYS:FD-STREAM for "a socket" {B8E0AA1}>) 9: (SB-IMPL::FD-STREAM-READ-N-BYTES #<SB-SYS:FD-STREAM for "a socket" {B8E0AA1}> #(97 108 121 116 105 99 115 62 10 32 32 102 49 50 51 52 53 54 55 56 57 48 . . . long array of data ) 0 8191 NIL) 10: (SB-IMPL::ANSI-STREAM-READ-SEQUENCE #(97 108 121 116 105 99 115 62 10 32 32 102 49 50 51 52 53 54 55 56 57 48 . . . long array of data ) #<SB-SYS:FD-STREAM for "a socket" {B8E0AA1}> 0 8191) 11: (READ-SEQUENCE #(97 108 121 116 105 99 115 62 10 32 32 102 49 50 51 52 53 54 55 56 57 48 . . . long array of data ) #<SB-SYS:FD-STREAM for "a socket" {B8E0AA1}>)[:EXTERNAL] 12: ((SB-PCL::FAST-METHOD TRIVIAL-GRAY-STREAMS:STREAM-READ-SEQUENCE (FLEXI-STREAMS:FLEXI-INPUT-STREAM T T T)) #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument>)[:EXTERNAL] 13: ((LAMBDA (SB-PCL::.PV. SB-PCL::.NEXT-METHOD-CALL. SB-PCL::.ARG0. SB-PCL::.ARG1. SB-PCL::.ARG2. SB-PCL::.ARG3. SB-INT:&MORE SB-PCL::.DFUN-MORE-CONTEXT. SB-PCL::.DFUN-MORE-COUNT.)) #<unused argument> #<unused argument> #<FLEXI-STREAMS:FLEXI-IO-STREAM {B8EA781}> #(97 108 121 116 105 99 115 62 10 32 32 102 49 50 51 52 53 54 55 56 57 48 . . . long array of data ) 0 8191 -307111303 0) 14: (READ-SEQUENCE #(97 108 121 116 105 99 115 62 10 32 32 102 49 50 51 52 53 54 55 56 57 48 . . . long array of data ) #<FLEXI-STREAMS:FLEXI-IO-STREAM {B8EA781}>)[:EXTERNAL] 15: ((SB-PCL::FAST-METHOD RUNES::XSTREAM-UNDERFLOW (RUNES:XSTREAM)) #<unavailable argument> #<unavailable argument> #<RUNES:XSTREAM [main document :MAIN NIL]>) 16: (CXML::READ-NAME-TOKEN #<unavailable argument>) 17: (CXML::READ-TAG-2 #S(CXML::ZSTREAM :TOKEN-CATEGORY :SEEN-< :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM [main document :MAIN NIL]>)) #<RUNES:XSTREAM [main document :MAIN NIL]> :STAG) 18: (CXML::FIX-SEEN-< #S(CXML::ZSTREAM :TOKEN-CATEGORY :SEEN-< :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM [main document :MAIN NIL]>))) 19: (CXML::P/DOCUMENT #S(CXML::ZSTREAM :TOKEN-CATEGORY :SEEN-< :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM [main document :MAIN NIL]>)) #<RUNE-DOM::DOM-BUILDER {B8EB0B9}>)[:EXTERNAL] 20: ((LAMBDA (CXML::ZSTREAM)) #S(CXML::ZSTREAM :TOKEN-CATEGORY :SEEN-< :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM [main document :MAIN NIL]>))) 21: (CXML::CALL-WITH-ZSTREAM #<CLOSURE (LAMBDA #) {B8EE055}> #S(CXML::ZSTREAM :TOKEN-CATEGORY :SEEN-< :TOKEN-SEMANTIC NIL :INPUT-STACK (#<RUNES:XSTREAM [main document :MAIN NIL]>))) 22: (CXML::PARSE-XSTREAM #<RUNES:XSTREAM [main document :MAIN NIL]> #<RUNE-DOM::DOM-BUILDER {B8EB0B9}>)[:EXTERNAL] 23: (ROUNDS-WEB::ANALYTICS-SERVICE) ;;; This is my service I've posted the sources next 24: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-REQUEST (HUNCHENTOOT:ACCEPTOR HUNCHENTOOT:REQUEST)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:ACCEPTOR (host *, port 8080)> #<HUNCHENTOOT:REQUEST {B8E8A11}>) 25: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-REQUEST (T)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:REQUEST {B8E8A11}>) 26: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION (HUNCHENTOOT:ACCEPTOR T)) #<unavailable argument> #<unavailable argument> #<HUNCHENTOOT:ACCEPTOR (host *, port 8080)> #<USOCKET:STREAM-USOCKET {B8E0B31}>) 27: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION :AROUND (HUNCHENTOOT:ACCEPTOR T)) #<unavailable argument> #S(SB-PCL::FAST-METHOD-CALL :FUNCTION #<FUNCTION #> :PV NIL :NEXT-METHOD-CALL NIL :ARG-INFO (2)) #<HUNCHENTOOT:ACCEPTOR (host *, port 8080)> #<USOCKET:STREAM-USOCKET {B8E0B31}>) 28: ((LAMBDA ())) 29: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK353]358)) 30: ((FLET SB-THREAD::WITH-MUTEX-THUNK)) 31: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]267)) 32: (SB-THREAD::CALL-WITH-MUTEX #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK) {B6C77205}> #S(SB-THREAD:MUTEX :NAME "thread result lock" :%OWNER #<SB-THREAD:THREAD "Hunchentoot worker (client: 127.0.0.1:39378)" RUNNING {B8E2391}> :STATE 1) #<SB-THREAD:THREAD "Hunchentoot worker (client: 127.0.0.1:39378)" RUNNING {B8E2391}> T) 33: ((LAMBDA ())) 34: ("foreign function: #x8066E3B") 35: ("foreign function: #x8052ABD") 36: ("foreign function: #x805DC00") 37: ("foreign function: #xB7FB696E") [2011-07-06 14:20:27 [ERROR]] Error while processing connection: #<SB-SYS:FD-STREAM for "a socket" {B8E0AA1}> is closed And here's the code for that service: (in-package :rounds-web) (defun split-no-last (subject by) (loop with last = 0 for i = (position by subject :start last) if i collect (subseq subject last i) into parts and do (setf last (1+ i)) else when (> (1- (length subject)) last) collect (subseq subject last) into parts end and return parts end)) (defun time-value (raw-value) (let* ((pos (position #\. raw-value)) (result (list (subseq raw-value 0 pos))) (raw-length (length raw-value))) (if (< pos raw-length) (append result (list (subseq raw-value (1+ pos) raw-length))) result))) (defun parse-name (attribute) ;; *round*, *category* and *event-type* are hash tables defined elsewhere `(,(gethash (subseq attribute 0 1) *round*) ,(gethash (subseq attribute 1 3) *category*) ,(gethash (subseq attribute 3) *event-type*))) (defun print-attribute (stream attribute val) (format stream "Event: Round ~a | Category ~a | Type ~a~&" (first attribute) (second attribute) (third attribute)) (loop for v in val for split = (time-value v) for time = (first split) for value = (if (> (length split) 1) (second split) nil) do (format stream " Time: ~a, Value: ~a~&" time value))) (defun save-parsed-events (xml) (with-open-file (stream "./analytics.log" :direction :output :if-exists :supersede) (loop for node across (dom:child-nodes xml) do (when (eq (dom:node-type node) :element) (format stream "VideoSessionId: ~a =========================================~&" (subseq (dom:node-name node) 1)) (when (dom:attributes node) (loop with attributes = (dom:attributes node) for n from 0 upto (1- (dom:length attributes)) for attribute = (dom:item attributes n) do (print-attribute stream (parse-name (dom:name attribute)) (split-no-last (dom:get-attribute node (dom:name attribute)) #\,)))))))) (defun analytics-service () (save-parsed-events (dom:first-child (cxml:parse-stream (hunchentoot:raw-post-data :request hunchentoot:*request* :want-stream t) (cxml-dom:make-dom-builder))))) This code isn't meant for production, all it does it mocks a real service which is meant to do the same thing so I could use it for testing, yet, I'd of course like it to work :) Please, if you have anything to suggest, I'd be very happy to hear! Thanks! Oleg

4 9