tbnl-devel February 2006

tbnl-devel@common-lisp.net

6 participants
9 discussions

[tbnl-devel] tbnl freeze
by Steve Huffman 30 Apr '06

30 Apr '06

Hi all, I'm working on reddit.com, which uses TBNL as its web framework. For the most part, all is well, but I've been running into an error lately that is causing me to worry. Basically, the symptom of the problem is that our website appears to pause, and all requests just hang (eventually they'll timeout on the browser side). I work against our server using remote SLIME. The SLIME session does not hang, but in order to bring the website back to life I log into our web server, use attachtty to view the running lisp process, and hit ctrl + c a few times, after which a debugger will pop open in Emacs that I can abort. I'm running CMUCL 19a on a FreeBSD box. I use apache and mod_lisp in front of TBNL. I haven't been able to find out what causes this lockup, nor do I understand why I must open the process on the web server (when I'm already cruising in SLIME) to wake it up. Has anyone seen any issues like this before? Or perhaps know if this is even a TBNL issue? I've pasted the output I see from the lisp process (via attachtty) and in my emacs session (after I send an interrupt). Any help would be greatly appreciated, Steve Huffman In attachtty I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY ] Destroy the process Debug (type H for help) (UNIX::SIGINT-HANDLER #<unused-arg> #<unused-arg> #.(SYSTEM:INT-SAP #x47FD789C)) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/signal.lisp. 0] and in emacs I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY] Destroy the process Backtrace: 0: (UNIX::SIGINT-HANDLER #<#1=unused-arg> #<#1#> #.(SYSTEM:INT-SAP #x47FD789C)) 1: (UNIX::SIGINT-HANDLER 3 #<#1=unused-arg> #<#1#> #.(SYSTEM:INT- SAP #x47FD789C))[:EXTERNAL] 2: ("Foreign function call land") 3: ("Foreign function call land") 4: ("Foreign function call land") 5: ("Foreign function call land") 6: ("Foreign function call land") 7: (LISP::FD-STREAM-READ-N-BYTES #<Stream for descriptor 8> "" 0 14 ...) 8: (LISP::FD-STREAM-READ-N-BYTES 5 #<Stream for descriptor 8> "" 0 ...)[:EXTERNAL] 9: (LISP::READ-INTO-SIMPLE-STRING "" #<Stream for descriptor 8> 0 14) 10: (READ-SEQUENCE "" #<Stream for descriptor 8> :START 0 ...) 11: ((METHOD INITIALIZE-INSTANCE (:AFTER) (TBNL::REQUEST)) (#(1 2 0 3 5 ...) . #()) #<#1=unused-arg> #<TBNL::REQUEST {49094BA5}> #<#1#>) 12: ((MAKE-INSTANCE TBNL::REQUEST :HEADERS-IN PCL::|.P0.|) (("content- stream" . #<Stream for descriptor 8>) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip- addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 13: (TBNL::PROCESS-REQUEST (("content-stream" . #<Stream for descriptor 8>) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip-addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 14: (TBNL::LISTEN-FOR-REQUEST #<Stream for descriptor 8> TBNL::PROCESS-REQUEST)

4 5

[tbnl-devel] uploading a huge file / ajax progress bar
by Mac Chan 14 Feb '06

14 Feb '06

Hi, I'm trying to see if it's feasible to use lisp + tbnl to build an internal documentation system that allows user to upload huge files. Typically users might upload files with size ranges from 20mb to 600mb. I'm using lispworks as the development platform, but intend to deploy the site in freebsd/linux + sbcl/cmucl. I notice that by uploading a 600mb file on win32 + lispworks + tbnl mod-lisp, the memory usage remains constant (ie. Lispworks is streaming the content directly to disk). However, with either freebsd/cmucl or linux/sbcl, top shows that all of the uploaded contents are first stored in memory and then get written to disk. This is a major scalability problem, I'm not sure if this is a design issue that can be fixed. Since uploading huge files take a bit of time, displaying a progress is a must; otherwise user might think that the browser hangs. It is also good to let them know how long will it take to upload the file so they can check back in couple minutes instead of staring at a stalled screen. There exist some open source implementations for php and ruby. http://www.raditha.com/php/progress.php http://sean.treadway.info/demo/upload/ I quickly went through the tbnl request post initialization routine and the rfc2388 module. It looks like what we need to do is to change rfc2388 to continuously write a report of the current upload progress to a file, and then add an entry point in tbnl to return a html page showing the current progress. This page can be launched by javascript when the upload form is submitted. Am I missing any important piece of information in this picture? If anyone can give me more pointers I would try my hands on this as an exercise. I'm not sure if anyone would find this useful though. Thanks in advance. -- Mac

3 4

Re: [tbnl-devel] tbnl configuration
by Edi Weitz 12 Feb '06

12 Feb '06

Please use the mailing list. On Sun, 12 Feb 2006 15:13:43 -0800, CyBerHigh <cyberhigh(a)gurusnetwork.org> wrote: > Does CLISP offer threads on Freebsd and on linux? No, CLISP doesn't have threads on any platform.

1 0

[tbnl-devel] Re: [Lispweb] tbnl: what package?
by Edi Weitz 10 Feb '06

10 Feb '06

[This is probably better asked on the TBNL mailing list.] On Fri, 10 Feb 2006 21:15:19 +1000, Igor Plekhov <penguin(a)ocean.vvo.ru> wrote: > In TBNL some functions are created to handle incoming requests. Functions are created? You're talking about your own handlers or something else? > What package are they executed in? I.e. what package *PACKAGE* is > bound to? How can I control it? In your own functions you can rebind *PACKAGE* to your liking. (let ((*package* (my-function))) ;; do something ) For existing TBNL functions the package shouldn't matter with the exception of READ-FROM-STRING* the behaviour of which is documented: <http://weitz.de/tbnl/#read-from-string*> Does that answer your question or am I misunderstanding?

3 8

Re: [tbnl-devel] tbnl configuration
by Edi Weitz 10 Feb '06

10 Feb '06

[Please use the mailing list.] On Fri, 10 Feb 2006 04:31:27 -0800, CyBerHigh <cyberhigh(a)gurusnetwork.org> wrote: > Sorry I ment Apache 13 with mod_lisp. There is no Apache 13. You probably mean 1.3. > I am on Freebsd 5.4 with SBCL .0.9.9 OK, then you don't have threads. That explains your problems. > My handler looks like this > LispServer 127.0.0.1 3000 "tbnl" > AddHandler lisp-handler .lisp > <Location /tbnl> > SetHandler lisp-handler > </Location> That's not the kind of handler I was talking about. I meant your Lisp handlers. <http://weitz.de/tbnl/#handlers> > Now, adding the .lisp part helped out, I can now get it to error out > saying something isn't configured write, when I try to go to a .lisp > file. I tryed to put a simple (format t "hello, world") into the > file but it gave me the same thing. Where do you expect the output of FORMAT to go? > I can't tell where it is erroring out at, because it doesn't put it > into the error log nor does it print it to the screen. What do you see in Apache's access log? > Any idea as to how I can get it printed out? Get what printed out? You can use LOG-MESSAGE to write to the error log. > It is possible that the slowness I was experincing could have been > from it being misconfigured. I think it's because you don't have threads. On FreeBSD use CMUCL. Cheers, Edi.

1 0

[tbnl-devel] tbnl configuration
by CyBerHigh 10 Feb '06

10 Feb '06

Hello I am trying to set up my first running of tbnl. Well, I got everything working to a point. I am using apache 12 and SBCL as my lisp. TBNL works, I can go to the test page and everything works on it. The problem with my set up is that it doesn't seem to be able to do anything else besides the default pages. Like, I tried to make it so that everything inside of /asp would be ran with TBNL server, however it just gives me a page saying that TBNL isn't proberly configured yet. Any idea as to how I can set it so this will work? Also is is possible for me to be able to set up apache to handle any file with the .lisp extention as a lisp file and send it to TBNL server? As I was playing with the test page I noticed that the server didn't seem very responsive. Well, it was on the first load but as soon as I would click on a link the page would like hang. Infact it never would load the page, I am not sure if this is because I havn't configured something right. If it is always like this could you let me know, because it wouldn't be very suited for production use if I can't get past the first page easyly. I am not saying that I couldn't ever, I had to click on the link a bunch and finally it would switch. Or I would have to wait like 5 mins then I could change the page. That IMO isn't very usfull for production. Is there a fix to this slow responsivness? Thank you

2 1

[tbnl-devel] Hunchentoot 0.1.3 released
by Edi Weitz 08 Feb '06

08 Feb '06

ChangeLog: Version 0.1.3 2006-02-08 Removed KMRCL workaround Download: <http://weitz.de/files/hunchentoot.tar.gz>

1 0

[tbnl-devel] New version 0.9.7
by Edi Weitz 08 Feb '06

08 Feb '06

ChangeLog: Version 0.9.7 2006-02-06 More robust computation of content length Download: <http://weitz.de/files/tbnl.tar.gz> Cheers, Edi.

1 0

Re: [tbnl-devel] Re: TBNL file descriptor leak
by Edi Weitz 02 Feb '06

02 Feb '06

On Sun, 22 Jan 2006 10:20:04 -0700, "Robert J. Macomber" <tbnl(a)rojoma.com> wrote: > In my apache2 config, I've got this [simplified]: > > LispServer 127.0.0.1 3000 lisp > <VirtualHost *:80 *:54321> > ServerName lisp.rojoma.com > DocumentRoot /home/robertm/mod_lisp > <LocationMatch /.*\.l> > SetHandler lisp-handler > </LocationMatch> > </VirtualHost> > > If you visit the url http://lisp.rojoma.com/%ef.l apache sends bytes > for the script-filename parameter which can be interpreted as the > latin-1 string "/home/robertm/mod_lisp/ï.l". If they're interpreted > as UTF-8, like sbcl's READ-LINE tries to do when running in an UTF-8 > locale (since kmrcl doesn't specify an explicit :external-format in > ACCEPT-TCP-CONNECTION) an error is thrown in GET-REQUEST-DATA. > > In fact, a user can make apache send arbitrary bytes to lisp while > it's reading the request headers in other ways too (e.g., sending a > "Mumble: {random octets}" header) but script-filename is the only > one where Apache will do so more or less on its own. > > I changed kmrcl here rather than making tbnl send a "bad request" > reply because you never know what random junk proxies and such will > throw into users' HTTP request headers, and I didn't want to deny > service just because some proxy is adding the (accented) name of > their ISP to user-agent strings or something. I haven't actually > seen that happen yet, but the net is vast. OK, thanks for the info. I've been thinking about extracting the relevant bits from KMRCL and integrating them into TBNL directly anyway. We only use a small fraction of what's in there and it might make sense to make some of this user-configurable, not only because of the external format but also because of the interface(s) TBNL will listen on, for example. I'll do that one day... :) Cheers, Edi.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

tbnl-devel February 2006