tbnl-devel January 2006

tbnl-devel@common-lisp.net

6 participants
9 discussions

[tbnl-devel] tbnl freeze
by Steve Huffman 30 Apr '06

30 Apr '06

Hi all, I'm working on reddit.com, which uses TBNL as its web framework. For the most part, all is well, but I've been running into an error lately that is causing me to worry. Basically, the symptom of the problem is that our website appears to pause, and all requests just hang (eventually they'll timeout on the browser side). I work against our server using remote SLIME. The SLIME session does not hang, but in order to bring the website back to life I log into our web server, use attachtty to view the running lisp process, and hit ctrl + c a few times, after which a debugger will pop open in Emacs that I can abort. I'm running CMUCL 19a on a FreeBSD box. I use apache and mod_lisp in front of TBNL. I haven't been able to find out what causes this lockup, nor do I understand why I must open the process on the web server (when I'm already cruising in SLIME) to wake it up. Has anyone seen any issues like this before? Or perhaps know if this is even a TBNL issue? I've pasted the output I see from the lisp process (via attachtty) and in my emacs session (after I send an interrupt). Any help would be greatly appreciated, Steve Huffman In attachtty I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY ] Destroy the process Debug (type H for help) (UNIX::SIGINT-HANDLER #<unused-arg> #<unused-arg> #.(SYSTEM:INT-SAP #x47FD789C)) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/signal.lisp. 0] and in emacs I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY] Destroy the process Backtrace: 0: (UNIX::SIGINT-HANDLER #<#1=unused-arg> #<#1#> #.(SYSTEM:INT-SAP #x47FD789C)) 1: (UNIX::SIGINT-HANDLER 3 #<#1=unused-arg> #<#1#> #.(SYSTEM:INT- SAP #x47FD789C))[:EXTERNAL] 2: ("Foreign function call land") 3: ("Foreign function call land") 4: ("Foreign function call land") 5: ("Foreign function call land") 6: ("Foreign function call land") 7: (LISP::FD-STREAM-READ-N-BYTES #<Stream for descriptor 8> "" 0 14 ...) 8: (LISP::FD-STREAM-READ-N-BYTES 5 #<Stream for descriptor 8> "" 0 ...)[:EXTERNAL] 9: (LISP::READ-INTO-SIMPLE-STRING "" #<Stream for descriptor 8> 0 14) 10: (READ-SEQUENCE "" #<Stream for descriptor 8> :START 0 ...) 11: ((METHOD INITIALIZE-INSTANCE (:AFTER) (TBNL::REQUEST)) (#(1 2 0 3 5 ...) . #()) #<#1=unused-arg> #<TBNL::REQUEST {49094BA5}> #<#1#>) 12: ((MAKE-INSTANCE TBNL::REQUEST :HEADERS-IN PCL::|.P0.|) (("content- stream" . #<Stream for descriptor 8>) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip- addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 13: (TBNL::PROCESS-REQUEST (("content-stream" . #<Stream for descriptor 8>) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip-addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 14: (TBNL::LISTEN-FOR-REQUEST #<Stream for descriptor 8> TBNL::PROCESS-REQUEST)

4 5

Re: [tbnl-devel] Re: TBNL file descriptor leak
by Edi Weitz 02 Feb '06

02 Feb '06

On Sun, 22 Jan 2006 10:20:04 -0700, "Robert J. Macomber" <tbnl(a)rojoma.com> wrote: > In my apache2 config, I've got this [simplified]: > > LispServer 127.0.0.1 3000 lisp > <VirtualHost *:80 *:54321> > ServerName lisp.rojoma.com > DocumentRoot /home/robertm/mod_lisp > <LocationMatch /.*\.l> > SetHandler lisp-handler > </LocationMatch> > </VirtualHost> > > If you visit the url http://lisp.rojoma.com/%ef.l apache sends bytes > for the script-filename parameter which can be interpreted as the > latin-1 string "/home/robertm/mod_lisp/ï.l". If they're interpreted > as UTF-8, like sbcl's READ-LINE tries to do when running in an UTF-8 > locale (since kmrcl doesn't specify an explicit :external-format in > ACCEPT-TCP-CONNECTION) an error is thrown in GET-REQUEST-DATA. > > In fact, a user can make apache send arbitrary bytes to lisp while > it's reading the request headers in other ways too (e.g., sending a > "Mumble: {random octets}" header) but script-filename is the only > one where Apache will do so more or less on its own. > > I changed kmrcl here rather than making tbnl send a "bad request" > reply because you never know what random junk proxies and such will > throw into users' HTTP request headers, and I didn't want to deny > service just because some proxy is adding the (accented) name of > their ISP to user-agent strings or something. I haven't actually > seen that happen yet, but the net is vast. OK, thanks for the info. I've been thinking about extracting the relevant bits from KMRCL and integrating them into TBNL directly anyway. We only use a small fraction of what's in there and it might make sense to make some of this user-configurable, not only because of the external format but also because of the interface(s) TBNL will listen on, for example. I'll do that one day... :) Cheers, Edi.

2 1

[tbnl-devel] tbnl as unix daemon
by Keith Irwin 31 Jan '06

31 Jan '06

Folks-- I want to be able to start/stop a tbnl based app via a script. I've done this before with my own server apps, so it all seems relatively straight forward. In other words, I have a Makefile which builds a core, copies shared-libs, packages it all up in a tarball, etc, etc, such that I can deploy the app on any machine, regardless of whether or not SBCL is installed. The problem is that (start-tbnl) starts up a thread, then returns, and with that return, sbcl quits, and I got no daemon. Is there a way to start tbnl without it returning? I suppose I can do something like: (defun daemon-start () (tbnl:start-tbnl) (wait-forever)) and then let my init.d script send a kill signal as I've already planned. Is that the recommended method (assuming I can find a wait-forever function, or just (read) or something)? In case it needs be said, I really value running tbnl inside a repl with slime and all that, but this is for the case where that's not desired or necessary. Thanks for any lore about what you all do in a similar case! ;) Keith

3 2

[tbnl-devel] Re: TBNL file descriptor leak
by Robert J. Macomber 22 Jan '06

22 Jan '06

On Sun, Jan 22, 2006 at 12:41:15AM +0100, Edi Weitz wrote: > > [not subscribed to tbnl-devel] > > You should be... :) All right; now I am. > I don't really like that idea. Instead, I now decided to send a 400 > (Bad Request) reply to the client. Check out TBNL 0.9.5 and see if it > works for you. It fixes the problem I was seeing, but I've uncovered a new one. However, this one's not really TBNL's fault. If you send a non-UTF8 urlencoded sequence _before_ the `?' in the request, TBNL dies the same as before, because headers are read from the bivalent stream with read-line (that is, in character mode) but Apache(*) de-url-encodes names before passing them on for the `script-filename' header. It can be fixed by either making kmrcl open the bivalent stream explicitly with :external-format :latin-1 or by running sbcl in a latin-1 locale. In any case, I haven't hit any more socket leaks now that that one's been tracked down, so it looks like all's right in the world, at least as far as TBNL's concerned. (*) now I'm using apache2 and mod_lisp2 version 1.3 -- Robert Macomber tbnl(a)rojoma.com

2 1

[tbnl-devel] New release 0.9.5
by Edi Weitz 22 Jan '06

22 Jan '06

ChangeLog: Version 0.9.5 2006-01-22 Made creation of REQUEST object safer (thanks to Robert J. Macomber) Replaced some erroneous DECLAIMs with DECLAREs (thanks to SBCL's style warnings) Slight documentation enhancements Download: <http://weitz.de/files/tbnl.tar.gz> Cheers, Edi.

1 1

[tbnl-devel] Error reading from Lisp (?)
by Edi Weitz 08 Jan '06

08 Jan '06

Hi! I'm currently having some trouble with a long-running TBNL/mod_lisp application on a public website which gets about 500,000 requests per month. It's an application delivered with LispWorks professional 4.4.6 and it runs on Linux behind Apache 2.0.54. >From time to time I see errors in the server log file and sometimes (very rarely) users also have reported errors. It turns out they're all of the "error reading from Lisp" kind. I hacked mod_lisp2.c a bit to make this more specific and I now get one of "error reading from Lisp (fill status)" or "error reading header name." This means that the error happens either when the reply entity is copied from Lisp to the client or when mod_lisp tries to read a header name from Lisp. I've attached a couple of these error message to the end of this email. What's suspicious for me is that they almost always come in chunks - half a dozen of them or more in a couple of seconds, then several hours without errors. I'm calling MARK-AND-SWEEP from time to time but at intervals that don't match the time of the error messages. (My first thought was that a full GC blocks LW long enough for mod_lisp to time out but that doesn't seem to be the case.) Other than that I only use locks in admin parts of the website which are clearly not used in the middle of the night when some of these errors happened. I'm lost. Does anyone have an idea where I should look to find the cause for these problems? Thanks, Edi. [Fri Jan 06 00:50:05 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:06 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:07 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:09 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:10 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:11 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:13 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:14 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:15 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 00:50:17 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:34 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:36 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:37 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:38 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:40 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:41 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:43 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:44 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:45 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 03:41:46 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:00 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:01 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:05 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:06 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:07 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:08 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:09 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:10 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:11 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 16:52:12 2006] [error] (70007)The timeout specified has expired: error reading header name [Fri Jan 06 18:07:26 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Fri Jan 06 21:53:49 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 04:31:45 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 05:03:53 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 06:04:10 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 07:05:02 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 07:56:04 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 07:56:05 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 07:56:06 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 10:03:19 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 10:28:26 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 11:22:53 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 11:23:57 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 11:37:46 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 11:48:43 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 11:52:56 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 13:03:12 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 13:24:57 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 13:32:49 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:04:45 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:10:00 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:11:04 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:12:08 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:13:14 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:14:19 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:15:26 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 14:58:12 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 15:25:11 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 15:29:14 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 15:30:45 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 15:31:49 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 15:46:11 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 15:49:50 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 15:53:46 2006] [error] (70007)The timeout specified has expired: error reading from Lisp (fill status) [Sat Jan 07 21:05:03 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:04 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:06 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:07 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:11 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:11 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:12 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:14 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:15 2006] [error] (70007)The timeout specified has expired: error reading header name [Sat Jan 07 21:05:19 2006] [error] (70007)The timeout specified has expired: error reading header name

2 2

[tbnl-devel] New: TBNL 0.9.4, Hunchentoot 0.1.2
by Edi Weitz 03 Jan '06

03 Jan '06

ChangeLog TBNL: Version 0.9.4 2006-01-03 Handle "Expect: 100-continue" for non-Apache front-ends Re-introduced IGNORE-ERRORS in GET-REQUEST-DATA ChangeLog Hunchentoot: Version 0.1.2 2006-01-03 Mention TBNL version number in server name header Download: <http://weitz.de/files/tbnl.tar.gz> <http://weitz.de/files/hunchentoot.tar.gz> Have fun, Edi.

1 0

[tbnl-devel] New Hunchentoot release 0.1.1
by Edi Weitz 01 Jan '06

01 Jan '06

ChangeLog: Version 0.1.1 2005-12-31 Fixed package stuff and HYPERDOC support Download: <http://weitz.de/files/hunchentoot.tar.gz> Cheers, Edi.

1 0

[tbnl-devel] New release 0.9.3
by Edi Weitz 01 Jan '06

01 Jan '06

Hi and Happy New Year! ChangeLog: Version 0.9.3 2006-01-01 Fixed bug in READ-HTTP-REQUEST Download: <http://weitz.de/files/tbnl.tar.gz> Cheers, Edi.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

tbnl-devel January 2006