On Sat, 26 May 2007 03:24:31 -0700, "Mac Chan" emailmac@gmail.com wrote:
Using the network monitor in FireBug , my initial testing (with a 790 KB javascript file) shows that the response time is slower with gzip turned on (I'm guessing that it might be related to flexi-stream.)
I suppose gzip itself will also take some time, won't it? Why do you think flexi-streams is the culprit in this case?
Should we add this to hunchentoot or make it an extension module? (this will introduce two new dependencies to hunchentoot)
I'm not against it per se, but I wouldn't want to add it just for static files. I think static files aren't something we should be particularly concerned with. If you have lots of static files and Hunchentoot can't serve them fast enough for your purposes, let Apache serve them, or put them behind mod_lisp and use mod_gzip. Besides, clients will cache Javascript files, so they'll have to read your big Ajax library only once.
If, OTOH, you also want to allow gzip encoding for dynamic content generated with Hunchentoot, it gets a lot more complicated, particularly because gzip-streams are binary only. I thought about this for some time, and I don't see an easy and elegant way around this - unless you want to add yet another flexi-streams layer which would result in four (!) layers of Gray streams atop each other.
If we add this to hunchentoot, should we add a flag to turn on/off gzip encoding? (probably yes)
Yes, definitely.
And again, since there's flexi-stream overhead (my guess), should we add a flag to control what content-type we should disable gzip encoding? (probably wasting cpu cycle to gzip jpeg, mp3 and png, etc).
Yes. This has nothing to do with whether flexi-streams adds overhead or not. It is generally a bad idea to compress formats that are already compressed... :)
(accept-gzip-p (search "gzip" (header-in :accept-encoding)))
I think that's a bit too simplistic. The header could be one of
Accept-Encoding: x-frogzipper Accept-Encoding: gzip; q=0 Accept-Encoding: *
Something like this maybe (untested):
(defun parse-qvalue (string) "Checks if the string STRING is a `qvalue' according to RFC 2616 and returns the corresponding numeric value. Signals an error otherwise." (unless (ppcre:scan "^(?:[0](?:\.[0-9]{0,3})?|[1](?:\.0{0,3})?)$" string) (error "Not a qvalue: ~S" string)) (case (char string 0) (#\1 1) (#\0 (let ((length (length string))) (case length ((1 2) 0) (otherwise (/ (parse-integer string :start 2) (case length (3 10) (4 100) (5 1000)))))))))
(defun accepts-gzip-p () "Checks whether the user agent accepts gzip encoding according to the incoming headers." (when-let (accept-encoding-header (header-in :accept-encoding)) (dolist (part (ppcre:split "," accept-encoding-header)) (with-input-from-string (stream part) (let* ((*current-error-message* "Corrupted Accept-Encoding header:") (coding (read-token stream)) (parameters (read-name-value-pairs stream)) (quality (cdr (assoc "q" parameters)))) (when (and (member coding '("gzip" "*") :test #'string=) (or (null quality) (plusp (handler-case (parse-qvalue quality) (error (condition) (log-message :warn "While parsing Accept-Encoding header: ~A" condition) 0))))) (return-from accepts-gzip-p t)))))))
Cheers, Edi.