Hi!
On Thu, 9 Feb 2006 01:38:13 -0800, Mac Chan emailmac@gmail.com wrote:
I notice that by uploading a 600mb file on win32 + lispworks + tbnl mod-lisp, the memory usage remains constant (ie. Lispworks is streaming the content directly to disk).
However, with either freebsd/cmucl or linux/sbcl, top shows that all of the uploaded contents are first stored in memory and then get written to disk. This is a major scalability problem, I'm not sure if this is a design issue that can be fixed.
This should not happen. Maybe you're using older versions of TBNL here? The code which is responsible for the streaming was inadvertently left out from some released versions.
I quickly went through the tbnl request post initialization routine and the rfc2388 module. It looks like what we need to do is to change rfc2388 to continuously write a report of the current upload progress to a file, and then add an entry point in tbnl to return a html page showing the current progress. This page can be launched by javascript when the upload form is submitted.
Am I missing any important piece of information in this picture? If anyone can give me more pointers I would try my hands on this as an exercise. I'm not sure if anyone would find this useful though.
What you describe seems to be the usual way to do it. I also did something similar once for a mod_perl application.
As far as changing RFC2388 is concerned, it would be nice if we could convince Janis to add the changes to his distributions (and therefore make them unobtrusive so they won't bother non-TBNL users).
Thanks, Edi.