[hunchentoot-devel] external-format and content-type question
I recently had this problem: I use sbcl + hunchentoot (current quicklisp supplied version) to serve cl-who generated content that contains non-ascii range utf-8 chars (greek letters). Initially i got the usual encoding error while processing a request. So I found the well known canonical solution: (setf *hunchentoot-default-external-format* (flex:make-external-format :utf8 :eol-style :lf)). The result is that the browser (chrome) displayed garbage where the greek letters should appear, although i set the default encoding of chrome to UTF-8 and i include the utf-8 meta tag in the content. I saw in the chrome tools that hunchentoot sent Latin-1 Http content-type header. This showed me that I needed to send the content-type-header manually and that Chrome (correctly IMO) ignores the user setting for the default encoding, if it receives a different HTTP content-type type header. So I set *default-content-type* and all is well now. But I came across the (reply-external-format*) function which confused me. Now, i am trying to clarify the functional co-relation or/and overlapping between those three: 1.setting *hunchentoot-default-external-format* 2.setting (reply-external-format*) --> is this just a per-request override of the *hunchentoot-default-external-format*? 3.sending the content-type header. --> Is there any legal scenario where we should report to the client something different than what is implied by (reply-external-format*)? What is the point of a default content-type header if there is a directly set default-external-format which also implicitly designates the header we should send? thanks in advance, Bill.
Hi Bill, Let me try to answer your question with a description of how external formats in Hunchentoot are supposed to work. I am talking about Hunchentoot 1.2.0 specifically, as the behavior has changed since the 1.1 release family in the hope of making it easier to understand and use. External formats are meant to mostly work automatically. It should be enough to select a proper external format for the character set that you normally use and set *hunchentoot-default-external-format* to that. The new default character set since 1.2.0 is :utf-8, so most standard use cases should be covered. Also new with 1.2.0 is Hunchentoot's ability to automatically report the encoding used in replies sent. This is done by adding a charset= attribute to the reply content type, but only if the content type is a subtype of "text" (i.e. it does not affect, say, application/xml). This is independent of the *hunchentoot-default-external-format* setting, i.e. you can set (reply-external-format*) in your handler to something different from the default and still see the right charset be added to the content-type, if it is text. All this works for handlers that return strings, i.e. if you handler returns an octet vector or directly writes to the response stream, you are responsible for setting up the content-type in the reply properly yourself. If this does not work, please supply us with a specific case where and how it does not. Thanks! Hans On Thu, Nov 10, 2011 at 2:42 PM, Vassilis Radis <radisb@gmail.com> wrote:
I recently had this problem: I use sbcl + hunchentoot (current quicklisp supplied version) to serve cl-who generated content that contains non-ascii range utf-8 chars (greek letters). Initially i got the usual encoding error while processing a request. So I found the well known canonical solution: (setf *hunchentoot-default-external-format* (flex:make-external-format :utf8 :eol-style :lf)). The result is that the browser (chrome) displayed garbage where the greek letters should appear, although i set the default encoding of chrome to UTF-8 and i include the utf-8 meta tag in the content. I saw in the chrome tools that hunchentoot sent Latin-1 Http content-type header. This showed me that I needed to send the content-type-header manually and that Chrome (correctly IMO) ignores the user setting for the default encoding, if it receives a different HTTP content-type type header. So I set *default-content-type* and all is well now. But I came across the (reply-external-format*) function which confused me. Now, i am trying to clarify the functional co-relation or/and overlapping between those three: 1.setting *hunchentoot-default-external-format* 2.setting (reply-external-format*) --> is this just a per-request override of the *hunchentoot-default-external-format*? 3.sending the content-type header. --> Is there any legal scenario where we should report to the client something different than what is implied by (reply-external-format*)? What is the point of a default content-type header if there is a directly set default-external-format which also implicitly designates the header we should send? thanks in advance, Bill. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
Hans, Thank you for your quick answer. The only blur thing for me now is: How exactly hunchentoot understands what kind of content-type my handler's return string contains? Thanks, Bill On Thu, Nov 10, 2011 at 4:12 PM, Hans Hübner <hans.huebner@gmail.com> wrote:
Hi Bill,
Let me try to answer your question with a description of how external formats in Hunchentoot are supposed to work. I am talking about Hunchentoot 1.2.0 specifically, as the behavior has changed since the 1.1 release family in the hope of making it easier to understand and use.
External formats are meant to mostly work automatically. It should be enough to select a proper external format for the character set that you normally use and set *hunchentoot-default-external-format* to that. The new default character set since 1.2.0 is :utf-8, so most standard use cases should be covered.
Also new with 1.2.0 is Hunchentoot's ability to automatically report the encoding used in replies sent. This is done by adding a charset= attribute to the reply content type, but only if the content type is a subtype of "text" (i.e. it does not affect, say, application/xml). This is independent of the *hunchentoot-default-external-format* setting, i.e. you can set (reply-external-format*) in your handler to something different from the default and still see the right charset be added to the content-type, if it is text.
All this works for handlers that return strings, i.e. if you handler returns an octet vector or directly writes to the response stream, you are responsible for setting up the content-type in the reply properly yourself.
If this does not work, please supply us with a specific case where and how it does not.
Thanks! Hans
I recently had this problem: I use sbcl + hunchentoot (current quicklisp supplied version) to serve cl-who generated content that contains non-ascii range utf-8 chars (greek letters). Initially i got the usual encoding error while processing a request. So I found the well known canonical solution: (setf *hunchentoot-default-external-format* (flex:make-external-format :utf8 :eol-style :lf)). The result is that the browser (chrome) displayed garbage where the greek letters should appear, although i set the default encoding of chrome to UTF-8 and i include the utf-8 meta tag in the content. I saw in the chrome tools that hunchentoot sent Latin-1 Http content-type header. This showed me that I needed to send the content-type-header manually and that Chrome (correctly IMO) ignores the user setting for the default encoding, if it receives a different HTTP content-type type header. So I set *default-content-type* and all is well now. But I came across the (reply-external-format*) function which confused me. Now, i am trying to clarify the functional co-relation or/and overlapping between those three: 1.setting *hunchentoot-default-external-format* 2.setting (reply-external-format*) --> is this just a per-request override of the *hunchentoot-default-external-format*? 3.sending the content-type header. --> Is there any legal scenario where we should report to the client something different than what is implied by (reply-external-format*)? What is the point of a default content-type
On Thu, Nov 10, 2011 at 2:42 PM, Vassilis Radis <radisb@gmail.com> wrote: header
if there is a directly set default-external-format which also implicitly designates the header we should send? thanks in advance, Bill. _______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
_______________________________________________ tbnl-devel site list tbnl-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/tbnl-devel
On Thu, Nov 10, 2011 at 4:37 PM, Vassilis Radis <radisb@gmail.com> wrote:
How exactly hunchentoot understands what kind of content-type my handler's return string contains?
It understands nothing about the content type. All it does is add a "charset" attribute to the content-type header if the returned content is a string and the content type begins with "text/". Your handlers still have to set the content type to the appropriate base type, but they don't need (and should not) set a charset attribute because that is automatically determined by Hunchentoot (through the external format). Let us know if you have further questions. -Hans
participants (2)
-
Hans Hübner -
Vassilis Radis