December 2005 - tbnl-devel@common-lisp.net

[tbnl-devel] tbnl freeze
by Steve Huffman 30 Apr '06

30 Apr '06

Hi all, I'm working on reddit.com, which uses TBNL as its web framework. For the most part, all is well, but I've been running into an error lately that is causing me to worry. Basically, the symptom of the problem is that our website appears to pause, and all requests just hang (eventually they'll timeout on the browser side). I work against our server using remote SLIME. The SLIME session does not hang, but in order to bring the website back to life I log into our web server, use attachtty to view the running lisp process, and hit ctrl + c a few times, after which a debugger will pop open in Emacs that I can abort. I'm running CMUCL 19a on a FreeBSD box. I use apache and mod_lisp in front of TBNL. I haven't been able to find out what causes this lockup, nor do I understand why I must open the process on the web server (when I'm already cruising in SLIME) to wake it up. Has anyone seen any issues like this before? Or perhaps know if this is even a TBNL issue? I've pasted the output I see from the lisp process (via attachtty) and in my emacs session (after I send an interrupt). Any help would be greatly appreciated, Steve Huffman In attachtty I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY ] Destroy the process Debug (type H for help) (UNIX::SIGINT-HANDLER #<unused-arg> #<unused-arg> #.(SYSTEM:INT-SAP #x47FD789C)) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/signal.lisp. 0] and in emacs I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY] Destroy the process Backtrace: 0: (UNIX::SIGINT-HANDLER #<#1=unused-arg> #<#1#> #.(SYSTEM:INT-SAP #x47FD789C)) 1: (UNIX::SIGINT-HANDLER 3 #<#1=unused-arg> #<#1#> #.(SYSTEM:INT- SAP #x47FD789C))[:EXTERNAL] 2: ("Foreign function call land") 3: ("Foreign function call land") 4: ("Foreign function call land") 5: ("Foreign function call land") 6: ("Foreign function call land") 7: (LISP::FD-STREAM-READ-N-BYTES #<Stream for descriptor 8> "" 0 14 ...) 8: (LISP::FD-STREAM-READ-N-BYTES 5 #<Stream for descriptor 8> "" 0 ...)[:EXTERNAL] 9: (LISP::READ-INTO-SIMPLE-STRING "" #<Stream for descriptor 8> 0 14) 10: (READ-SEQUENCE "" #<Stream for descriptor 8> :START 0 ...) 11: ((METHOD INITIALIZE-INSTANCE (:AFTER) (TBNL::REQUEST)) (#(1 2 0 3 5 ...) . #()) #<#1=unused-arg> #<TBNL::REQUEST {49094BA5}> #<#1#>) 12: ((MAKE-INSTANCE TBNL::REQUEST :HEADERS-IN PCL::|.P0.|) (("content- stream" . #<Stream for descriptor 8>) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip- addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 13: (TBNL::PROCESS-REQUEST (("content-stream" . #<Stream for descriptor 8>) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip-addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 14: (TBNL::LISTEN-FOR-REQUEST #<Stream for descriptor 8> TBNL::PROCESS-REQUEST)

4 5

[tbnl-devel] Announcement: Hunchentoot
by Edi Weitz 31 Dec '05

31 Dec '05

Hunchentoot is yet another Common Lisp web server. It is implemented atop TBNL and adds HTTP/1.1 chunking, persistent connections (keep-alive), and SSL to the mix. More info here: <http://weitz.de/hunchentoot/> Cheers, Edi.

2 1

[tbnl-devel] New version 0.9.2
by Edi Weitz 31 Dec '05

31 Dec '05

ChangeLog: Version 0.9.2 2005-12-31 Protocol of reply is HTTP/1.1 now Made HTTP/0.9 default protocol of request if none was provided Some preparations for Hunchentoot Various minor changes Small fixes in docs Download: <http://weitz.de/files/tbnl.tar.gz> Cheers, Edi.

1 0

[tbnl-devel] Re: default handler
by Edi Weitz 27 Dec '05

27 Dec '05

Hi Jim! On Thu, 22 Dec 2005 07:11:18 -0700 (MST), Jim Prewett <download(a)hpc.unm.edu> wrote: > I've been pumping my co-worker, Jared, for information on dynamic > websites. Jared is the primary author of our > trouble-ticket/accounting/inventory/testing/everything-but-the-kitchen-sink > web application called AIRS. AIRS is a Perl/Mason/MySQL based > application. (its on Sourceforge if you ever something like that > FWIW) > > Jared was telling me that in AIRS, one of the tricks they do to > ensure that a user is logged in with a valid session is to have what > Jared calls a "default handler" which is invoked with every page (by > Mason apparently). I guess it serves to validate anything you need > validated before a page is displayed; Jared notes that it can be > used to temporarily "hijack" the functionality of the page (so, in > AIRS's case, the list-ticket page will display the list of tickets > to a logged in user, but a user without a valid session will be sent > to the login page and after being successfully logged in, she will > be shown the list of tickets). > > Does TBNL support anything like this "out of the box"? If not, do > you have any thoughts on how one might implement such a feature? > Would you, maybe, define a macro like (with-valid-session ... ) > that would be called inside of each function that spits back a > webpage? > > I like the idea, but i'm not necessarily after a 1:1 mapping to the > concepts in Mason, I just want to implement the general idea. Generally, I think that questions like these are better asked on the TBNL mailing list (on Cc now, you need to be subscribed to post something). Other users might already have a solution for this and people who have similar questions can later scan the archives. No, TBNL doesn't have this out of the box but it should be flexible enough to add whatever scheme you like. I myself usually use macros like WITH-VALID-SESSION, yes. Cheers, Edi.

2 3

[tbnl-devel] destroying a session?
by Jim Prewett 27 Dec '05

27 Dec '05

I'm trying to implement some sort of "logout" functionality right now, but don't see a clear way to do it. What I've come up with is setting the session-max-time to 0. Is there a better way to do this? Thanks, Jim James E. Prewett Jim(a)Prewett.org download(a)hpc.unm.edu Systems Team Leader LoGS: http://www.hpc.unm.edu/~download/LoGS/ Designated Security Officer OpenPGP key: pub 1024D/31816D93 HPC Systems Engineer III UNM HPC 505.277.8210

2 2

[tbnl-devel] Merged patches to Araneida (with one change)
by Alan Shields 26 Dec '05

26 Dec '05

Hello everyone, As Edi is out for a few days, he asked me to email the devel list instead. I've merged in the patches to Araneida to enable TBNL to work. I made a few modifications, however. I'd like someone to test out the changes to make sure they work. To test out the changes, do: darcs get http://code.microarray.omrf.org/darcs/alan-araneida-testing/ On the TBNL-side, instead of setting araneida::*standard-araneida* to nil, you need to set araneida:*araneida-mode* to :tbnl . Let me know if it works! Alan Shields

3 3

[tbnl-devel] Re: [mod-lisp-devel] mod_lisp2.c: Patch for Lisp data w/o Content Length header
by Edi Weitz 26 Dec '05

26 Dec '05

On Sat, 24 Dec 2005 23:09:36 +0100, Edi Weitz <edi(a)agharta.de> wrote: > From the ChangeLog it looks like this was already fixed 2004-11-15 > but it doesn't work in the current version. Patch attached. I have, BTW, uploaded corresponding mod_lisp2 binaries for Win32 to this location <http://weitz.de/files/mod_lisp2.zip> as it seems that the ones at <http://www.fractalconcept.com:8000/public/open-source/mod_lisp/windows/> were quite old. Cheers, Edi.

1 0

[tbnl-devel] mime-types.lisp
by Hilverd Reker 25 Dec '05

25 Dec '05

Hello, I just downloaded TBNL 0.9.0, and it seems like the file mime-types.lisp is missing (to which tbnl.asd refers). Could you please take a look? Hilverd Reker

2 1

[tbnl-devel] New version 0.9.0
by Edi Weitz 24 Dec '05

24 Dec '05

ChangeLog: Version 0.9.0 2005-12-24 Experimental support for writing directly to the front-end (see SEND-HEADERS) Added HANDLE-STATIC-FILE Changed CREATE-STATIC-FILE-DISPATCHER-AND-HANDLER to use new facilities Added CREATE-FOLDER-DISPATCHER-AND-HANDLER Added link to Travis Cross' message w.r.t. SBCL Download: <http://weitz.de/files/tbnl.tar.gz> Happy Holidays, Edi.

2 1

[tbnl-devel] fixed: tbnl / sbcl binary upload / download issues
by Travis Cross 20 Dec '05

20 Dec '05

TBNL has played poorly with SBCL up to this point with regard to binary data. Uploading binary files never worked at all, and only 1/2 of the binary download examples appeared to work out of the box. As it turns out, these problems are not related to code in current versions of TBNL itself, but in TBNL's dependencies: KMRCL and rfc2388. Attached are the necessary patches. *Changes* -- kmrcl: Prior to this patch, kmrcl opened streams in SBCL with :element-type 'base-char. This effectively prevented streams from writing character values above 127 or operating in a bivalent fashion. SBCL will open bivalent streams when :element-type :default is specified. -- rfc2388 rfc2388 opens a file stream to save attachments. If the :external-format option is not given to OPEN, SBCL will default to :ascii and only write characters up to 127. The :external-format :latin-1 option needs to be passed to OPEN. *Testing* The "Binary data, delivered from file", "Binary data, delivered from RAM", and "File Upload" examples in the tbnl-test package all work correctly now with SBCL-0.9.7, both when using TBNL directly (without a front-end) and with mod_lisp2 / Apache2. For binary upload testing, I uploaded a 165MB video. I then downloaded the video through the tbnl interface (with Firefox) and ensured that the hashes of the two files matched. I was a bit surprised, actually, that TBNL / SBCL didn't choke on the size of the file ;) To make sure these changes were production-ready, I ran ApacheBench2 against the binary image downloads: /usr/sbin/ab2 -c 50 -n 1000 http://server/tbnl/test/image-ram.jpg Complete requests: 1000 Failed requests: 0 Requests per second: 343.94 [#/sec] (mean) Time per request: 145.373 [ms] (mean) /usr/sbin/ab2 -c 50 -n 1000 http://server/tbnl/test/image.jpg Complete requests: 1000 Failed requests: 0 Requests per second: 237.63 [#/sec] (mean) Time per request: 210.412 [ms] (mean) As far as I can tell, these changes make SBCL a fully supported platform for TBNL. Cheers, -- Travis PS - I should note a few things here for reference: -- :tbnl-bivalent-streams needs to be enabled in *features*. -- The unicode tests in the TBNL suite appear to work correctly. -- Contrary to our prior expectations, the underlying issue was not related to handling sb-unicode appropriately, and in fact, binary handling does not seem to rely on sb-unicode at all. diff -u kmrcl-1.84/processes.lisp kmrcl/processes.lisp --- kmrcl-1.84/processes.lisp 2005-11-03 12:43:52.000000000 -0500 +++ kmrcl/processes.lisp 2005-12-02 19:33:15.000000000 -0500 @@ -17,7 +17,7 @@ #+allegro (mp:process-run-function name func) #+cmu (mp:make-process func :name name) #+lispworks (mp:process-run-function name nil func) - #+sb-thread (sb-thread:make-thread func) + #+sb-thread (sb-thread:make-thread func :name name) #+openmcl (ccl:process-run-function name func) #-(or allegro cmu lispworks sb-thread openmcl) (funcall func) ) diff -u kmrcl-1.84/sockets.lisp kmrcl/sockets.lisp --- kmrcl-1.84/sockets.lisp 2005-11-03 12:43:52.000000000 -0500 +++ kmrcl/sockets.lisp 2005-12-19 20:20:27.000000000 -0500 @@ -84,7 +84,7 @@ (let ((sock (sb-bsd-sockets:socket-accept listener))) (values (sb-bsd-sockets:socket-make-stream - sock :element-type 'base-char :input t :output t) + sock :element-type :default :input t :output t) sock))) #+openmcl (let ((sock (ccl:accept-connection listener :wait t))) @@ -115,7 +115,8 @@ (defun close-active-socket (socket) - (close socket)) + #+sbcl (sb-bsd-sockets:socket-close socket) + #-sbcl (close socket)) (defun ipaddr-to-dotted (ipaddr &key values) "Convert from 32-bit integer to dotted string." @@ -178,7 +179,7 @@ (sb-bsd-sockets:socket-connect sock (lookup-hostname server) port) (values (sb-bsd-sockets:socket-make-stream - sock :input t :output t :element-type 'base-char) + sock :input t :output t :element-type :default) sock)) #+cmu (let ((sock (ext:connect-to-inet-socket server port))) diff -u rfc2388.orig/rfc2388.lisp rfc2388/rfc2388.lisp --- rfc2388.orig/rfc2388.lisp 2005-12-19 20:12:42.000000000 -0500 +++ rfc2388/rfc2388.lisp 2005-12-19 20:29:50.000000000 -0500 @@ -441,8 +441,9 @@ :direction :output ;; external format for faithful I/O ;; see <http://cl-cookbook.sourceforge.net/io.html#faith> - #+(or :lispworks :allegro) + #+(or :sbcl :lispworks :allegro) :external-format + #+:sbcl :latin-1 #+:lispworks '(:latin-1 :eol-style :lf) #+:allegro (excl:crlf-base-ef :latin1)) (read-until-next-boundary input boundary nil out-file))

2 1