[hunchentoot-devel] Publishing stream data?
Hello, I want to return some data in CSV format (or also PDF format in another application). At the moment I am using something like (let ((file "output.csv")) (with-open-file (stream file :direction :output :if-exists :supersede) (csv-output stream ...)) (handle-static-file file) (delete-file file)) However, this is clearly not good in a multiprocessing situation, and I would like to get rid of the intermediate file. Is it possible to publish the stream output directly? Thank you very much, Nicolas
Nicolas, Take a look at the documentation for HUNCHENTOOT:SEND-HEADERS (http://weitz.de/hunchentoot/#send-headers). The STREAM returned by SEND-HEADERS can be bound using WITH-OPEN-STREAM and written to using standard Common Lisp I/O functions, e.g., (with-open-stream (out (hunchentoot:send-headers)) (write-sequence (make-array pad-len :element-type '(unsigned-byte 8) :initial-element 0) out)) Hope this helps. -ram On 9/28/07, Nicolas Neuss <neuss@math.uni-karlsruhe.de> wrote:
Hello,
I want to return some data in CSV format (or also PDF format in another application). At the moment I am using something like
(let ((file "output.csv")) (with-open-file (stream file :direction :output :if-exists :supersede) (csv-output stream ...)) (handle-static-file file) (delete-file file))
However, this is clearly not good in a multiprocessing situation, and I would like to get rid of the intermediate file. Is it possible to publish the stream output directly?
Thank you very much, Nicolas _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
On Fri, 28 Sep 2007 09:40:15 -0700, "Ram Krishnan" <rkris@kriyative.net> wrote:
The STREAM returned by SEND-HEADERS can be bound using WITH-OPEN-STREAM and written to using standard Common Lisp I/O functions, e.g.,
(with-open-stream (out (hunchentoot:send-headers)) (write-sequence (make-array pad-len :element-type '(unsigned-byte 8) :initial-element 0) out))
You shouldn't use WITH-OPEN-STREAM here as that will unconditionally close the stream. Hunchentoot must be able to decide whether the stream should be closed or not.
Ah, you're absolutely right. Simply binding using LET, instead of WITH-OPEN-STREAM would do the trick then. Thanks. -ram On 9/29/07, Edi Weitz <edi@agharta.de> wrote:
On Fri, 28 Sep 2007 09:40:15 -0700, "Ram Krishnan" <rkris@kriyative.net> wrote:
The STREAM returned by SEND-HEADERS can be bound using WITH-OPEN-STREAM and written to using standard Common Lisp I/O functions, e.g.,
(with-open-stream (out (hunchentoot:send-headers)) (write-sequence (make-array pad-len :element-type '(unsigned-byte 8) :initial-element 0) out))
You shouldn't use WITH-OPEN-STREAM here as that will unconditionally close the stream. Hunchentoot must be able to decide whether the stream should be closed or not. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
"Ram Krishnan" <rkris@kriyative.net> writes:
Ah, you're absolutely right.
Simply binding using LET, instead of WITH-OPEN-STREAM would do the trick then.
Thanks.
-ram
OK, thanks to everyone who responded (and especially to Edi for Hunchentoot and all the other nice software). I have now used something like the following: (define-easy-handler (csv-export-page :uri "/csv-export") ((csv :parameter-type 'boolean) ...) (when csv (setf (content-type) "text/comma-separated-values" (header-out "Last-Modified") (rfc-1123-date (get-universal-time)) (content-length) nil) (let ((out (send-headers))) (csv-output :stream out ...))) (with-output-to-string (out) (fill-and-print-template "csv-export.html" values :stream out))) As much as I can see, this works. Yours, Nicolas
Nicolas Neuss <neuss@math.uni-karlsruhe.de> writes:
(define-easy-handler (csv-export-page :uri "/csv-export") ((csv :parameter-type 'boolean) ...) (when csv (setf (content-type) "text/comma-separated-values" (header-out "Last-Modified") (rfc-1123-date (get-universal-time)) (content-length) nil) (let ((out (send-headers))) (csv-output :stream out ...))) (with-output-to-string (out) (fill-and-print-template "csv-export.html" values :stream out)))
As much as I can see, this works.
In the code, ... denote some further parameters which can be configured via the csv-export page. Nicolas
Hello everbody, Coincidently playing with hunchentoot yesterday I encountered the same problem with streams. Based in the code of HANDLE-STATIC-FILE a made a path with 2 functions and one macro: make-http-output-stream witch send the appropriate headers and return a stream. This function deal with content-type and content-disposition headers. with-http-output-stream Create an http-output-stream, run the code and call finish-output handle-stream Copy the stream to the http-output I hope to contribute to the hunchetoot development. Att. Leonardo Varuzza. USP - Brazil. PS: A interesting use of this patch is to send images created by Zach Brene's Vecto library. I just send another patch to him to enable saving the result png to a stream. On 10/1/07, Nicolas Neuss <neuss@math.uni-karlsruhe.de> wrote:
Nicolas Neuss <neuss@math.uni-karlsruhe.de> writes:
(define-easy-handler (csv-export-page :uri "/csv-export") ((csv :parameter-type 'boolean) ...) (when csv (setf (content-type) "text/comma-separated-values" (header-out "Last-Modified") (rfc-1123-date (get-universal-time)) (content-length) nil) (let ((out (send-headers))) (csv-output :stream out ...))) (with-output-to-string (out) (fill-and-print-template "csv-export.html" values :stream out)))
As much as I can see, this works.
In the code, ... denote some further parameters which can be configured via the csv-export page.
Nicolas _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
-- The most fundamental particles in this product are held together by a "gluing" force about which little is currently known and whose adhesive power can therefore not be permanently guaranteed.
On Mon, 1 Oct 2007 09:13:47 -0300, "Leonardo Varuzza" <varuzza@gmail.com> wrote:
Coincidently playing with hunchentoot yesterday I encountered the same problem with streams. Based in the code of HANDLE-STATIC-FILE a made a path with 2 functions and one macro:
make-http-output-stream witch send the appropriate headers and return a stream. This function deal with content-type and content-disposition headers.
with-http-output-stream Create an http-output-stream, run the code and call finish-output
handle-stream Copy the stream to the http-output
I hope to contribute to the hunchetoot development.
Thanks for the patch, but I hope you don't mind if I think that this is in the realm of things that can and should be done by Hunchentoot users and not be provided by the library. Cheers, Edi.
Edi Weitz <edi@agharta.de> writes:
Coincidently playing with hunchentoot yesterday I encountered the same problem with streams. Based in the code of HANDLE-STATIC-FILE a made a path with 2 functions and one macro:
make-http-output-stream witch send the appropriate headers and return a stream. This function deal with content-type and content-disposition headers.
with-http-output-stream Create an http-output-stream, run the code and call finish-output
handle-stream Copy the stream to the http-output
I hope to contribute to the hunchetoot development.
Thanks for the patch, but I hope you don't mind if I think that this is in the realm of things that can and should be done by Hunchentoot users and not be provided by the library.
Perhaps there should be a HUNCHENTOOT-CONTRIB which contains such things, and if they become standard/useful enough then they might be merged into Hunchentoot proper? -- Robert Uhl <http://public.xdi.org/=ruhl> Whoa, there. Those are some strong words for somebody who doesn't even own a machine gun. --Milkman Dan
On Thu, 04 Oct 2007 10:17:31 -0600, Robert Uhl <eadmund42@gmail.com> wrote:
Perhaps there should be a HUNCHENTOOT-CONTRIB which contains such things, and if they become standard/useful enough then they might be merged into Hunchentoot proper?
In TBNL we had a contrib repository included with the distribution, but I deemed it too much work (for me) to maintain it together with the main source. We now have this http://common-lisp.net/websvn/listing.php?repname=tbnl&path=%2Fcontrib%2Flsp%2F which currently contains code by Mac Chan. If someone else wants access to this repository for add-ons for Hunchentoot, let me know. Cheers, Edi.
On a related note, in the absence of a hucnhentoot-contrib, I'll take a moment to plug some of my own hunchentoot-using packages. Two are general purpose libraries that could, in theory, be used by any hunchentoot application, hunchentoot-vhost for using a single hunchentoot server to serve multiple "hosts", and hunchentoot-auth for managing users. The other package is nuclblog, which is a relatively simple blog that runs on top of hunchentoot. They can be found at: http://cyrusharmon.org/projects Comments/suggestions/patches-to-implement-all-the-neat-features-I- never-got-around-to-implementing gladly accepted. In particular, I'd be curious as to how others handle user authentication and what not. It starts off trivial, but, at least from my experience, got to be a bit more complicated than I would have thought at the beginning. Things like redirecting non-authorized users, html pages for logging in vs. browser authentication, password salts, persistent storage for users, an interface for adding new users, etc... all make this a harder problem than one might think at first glance. Perhaps it's just "trivial" and everyone should roll their own solution here, but, comments on the hunchentoot-auth approach would be greatly appreciated. Thanks, Cyrus On Oct 4, 2007, at 12:26 PM, Edi Weitz wrote:
On Thu, 04 Oct 2007 10:17:31 -0600, Robert Uhl <eadmund42@gmail.com> wrote:
Perhaps there should be a HUNCHENTOOT-CONTRIB which contains such things, and if they become standard/useful enough then they might be merged into Hunchentoot proper?
In TBNL we had a contrib repository included with the distribution, but I deemed it too much work (for me) to maintain it together with the main source. We now have this
http://common-lisp.net/websvn/listing.php?repname=tbnl&path=% 2Fcontrib%2Flsp%2F
which currently contains code by Mac Chan. If someone else wants access to this repository for add-ons for Hunchentoot, let me know.
Cheers, Edi. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
Nicolas Neuss <neuss@math.uni-karlsruhe.de> writes:
I want to return some data in CSV format (or also PDF format in another application). At the moment I am using something like
(let ((file "output.csv")) (with-open-file (stream file :direction :output :if-exists :supersede) (csv-output stream ...)) (handle-static-file file) (delete-file file))
However, this is clearly not good in a multiprocessing situation, and I would like to get rid of the intermediate file. Is it possible to publish the stream output directly?
If I were you, I'd start with looking at the source code of HANDLE-STATIC-FILE: (defun handle-static-file (path &optional content-type) "A function which acts like a Hunchentoot handler for the file denoted by PATH. Send a content type header corresponding to CONTENT-TYPE or \(if that is NIL) tries to determine the content type via the file's suffix." (unless (or (pathname-name path) (pathname-type path)) ;; not a file (setf (return-code) +http-bad-request+) (throw 'handler-done nil)) (unless (probe-file path) ;; does not exist (setf (return-code) +http-not-found+) (throw 'handler-done nil)) (let ((time (or (file-write-date path) (get-universal-time)))) (setf (content-type) (or content-type (mime-type path) "application/octet-stream")) (handle-if-modified-since time) (with-open-file (file path :direction :input :element-type '(unsigned-byte 8) :if-does-not-exist nil) (setf (header-out "Last-Modified") (rfc-1123-date time) (content-length) (file-length file)) (let ((out (send-headers))) (loop with buf = (make-array +buffer-length+ :element-type '(unsigned-byte 8)) for pos = (read-sequence buf file) until (zerop pos) do (write-sequence buf out :end pos) (finish-output out)))))) The trick is to send the will be transfered content to the stream returned by call to SEND-HEADERS. Also, don't forget to check the SEND-HEADERS documentation from the Hunchentoot manual: ... Returns a flexi stream to which the body of the reply can be written. Once this function has been called, further changes to *REPLY* don't have any effect. Also, automatic handling of errors (i.e. sending the corresponding status code to the browser, etc.) is turned off for this request. ... (More is in the documentation.) Regards.
participants (7)
-
Cyrus Harmon -
Edi Weitz -
Leonardo Varuzza -
Nicolas Neuss -
Ram Krishnan -
Robert Uhl -
Volkan YAZICI