Hi,
I'm trying to see if it's feasible to use lisp + tbnl to build an internal documentation system that allows user to upload huge files.
Typically users might upload files with size ranges from 20mb to 600mb.
I'm using lispworks as the development platform, but intend to deploy the site in freebsd/linux + sbcl/cmucl.
I notice that by uploading a 600mb file on win32 + lispworks + tbnl mod-lisp, the memory usage remains constant (ie. Lispworks is streaming the content directly to disk).
However, with either freebsd/cmucl or linux/sbcl, top shows that all of the uploaded contents are first stored in memory and then get written to disk. This is a major scalability problem, I'm not sure if this is a design issue that can be fixed.
Since uploading huge files take a bit of time, displaying a progress is a must; otherwise user might think that the browser hangs. It is also good to let them know how long will it take to upload the file so they can check back in couple minutes instead of staring at a stalled screen.
There exist some open source implementations for php and ruby. http://www.raditha.com/php/progress.php http://sean.treadway.info/demo/upload/
I quickly went through the tbnl request post initialization routine and the rfc2388 module. It looks like what we need to do is to change rfc2388 to continuously write a report of the current upload progress to a file, and then add an entry point in tbnl to return a html page showing the current progress. This page can be launched by javascript when the upload form is submitted.
Am I missing any important piece of information in this picture? If anyone can give me more pointers I would try my hands on this as an exercise. I'm not sure if anyone would find this useful though.
Thanks in advance. -- Mac