From dl@znain.net Fri May 18 10:01:15 2007
From: Dimitre Liotev <dl@znain.net>
To: html-template-devel@common-lisp.net
Subject: [html-template-devel] problem with UTF characters in template
Date: Fri, 18 May 2007 10:01:06 -0400
Message-ID: <81bqgigrj1.fsf@oh177.znain.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============1457240001690606559=="

--===============1457240001690606559==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


Hi,

when I have UTF characters in a template I get this error:

#\? is not a LATIN-1 character.
   [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]

Is there a way to fix this?

I use Allegro CL Express on Windows. 

-- 
Dimitre Liotev

--===============1457240001690606559==--


From edi@agharta.de Fri May 18 10:08:49 2007
From: Edi Weitz <edi@agharta.de>
To: html-template-devel@common-lisp.net
Subject: Re: [html-template-devel] problem with UTF characters in template
Date: Fri, 18 May 2007 16:08:51 +0200
Message-ID: <ud50y6x70.fsf@agharta.de>
In-Reply-To: <81bqgigrj1.fsf@oh177.znain.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============6833688922881280911=="

--===============6833688922881280911==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit

On Fri, 18 May 2007 10:01:06 -0400, Dimitre Liotev <dl(a)znain.net> wrote:

> when I have UTF characters in a template I get this error:
>
> #\? is not a LATIN-1 character.
>    [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]

What external format did you specify?  How about showing us the code
you tried and providing a backtrace?

Cheers,
Edi.

--===============6833688922881280911==--


From dl@znain.net Fri May 18 11:24:22 2007
From: Dimitre Liotev <dl@znain.net>
To: html-template-devel@common-lisp.net
Subject: Re: [html-template-devel] problem with UTF characters in template
Date: Fri, 18 May 2007 11:24:15 -0400
Message-ID: <814pmagnog.fsf@oh177.znain.com>
In-Reply-To: <ud50y6x70.fsf@agharta.de>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============8255216617068270961=="

--===============8255216617068270961==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Edi Weitz <edi(a)agharta.de> writes:

> On Fri, 18 May 2007 10:01:06 -0400, Dimitre Liotev <dl(a)znain.net> wrote:
>
>> when I have UTF characters in a template I get this error:
>>
>> #\? is not a LATIN-1 character.
>>    [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]
>
> What external format did you specify?  How about showing us the code
> you tried and providing a backtrace?
>
> Cheers,
> Edi.


Hi Edi,

I am using the "Simple example" from your page
http://weitz.de/html-template/, specifying :external-format "UTF-8" in
the call to create-template-printer:

(defparameter *template-dispatcher*
  (hunchentoot:create-prefix-dispatcher "/template-test/"
   (lambda ()
     (let* ((html-template:*string-modifier* #'identity)
           (rows (loop for i below 49 by 7
                   collect
                     (list :cols
                           (loop for j from i below (+ i 7)
                              for string =3D (format nil "~R" j)
                              collect (list :content string=20
                                            :colorful-style (oddp j))))))
          (values (list :rows rows)))
       (with-output-to-string (html-template:*default-template-output*)
         (html-template:fill-and-print-template
          (html-template:create-template-printer #p"foo.tmpl" :external-forma=
t "UTF-8")
          values))))))                          =20


(push *template-dispatcher* hunchentoot:*dispatch-table*)


The template:

<table border=3D1>
  <!-- TMPL_LOOP rows -->
    <tr>
      <!-- TMPL_LOOP cols -->
        <!-- TMPL_IF colorful-style -->
          <td align=3D"right" bgcolor=3D"blue">some cyrullic chars: =D0=94=D0=
=96=D0=93=D0=91<!-- TMPL_VAR content --></td>
        <!-- TMPL_ELSE -->
          <td align=3D"right" ><!-- TMPL_VAR content --></td>
        <!-- /TMPL_IF -->
      <!-- /TMPL_LOOP -->
    </tr>
  <!-- /TMPL_LOOP -->
</table>


Now if I point the browser to http://localhost:8082/template-test/ it
works fine if the UTF chars are not there, but if I put any cyrillic UTF
encoded character in the template I get the error:


#\? is not a LATIN-1 character.
   [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]

Restarts:
 0: [ABORT] Abort entirely from this (lisp) process.

Backtrace:
  0: ((:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0))
      Locals:
        CONDITION =3D :UNKNOWN
  1: ((:INTERNAL (:TOP-LEVEL-FORM "swank-backend.lisp" 20112) 0)
      #<Function SWANK-DEBUGGER-HOOK>
      #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK-BACKEND::HOOK =3D #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN =3D #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        SWANK-BACKEND::FUN =3D #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
  2: (SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK
       #<Function SWANK-DEBUGGER-HOOK>
       #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK-BACKEND::HOOK =3D #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN =3D #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        SWANK-BACKEND::HOOK =3D #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN =3D #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
  3: ((:INTERNAL SWANK::CALL-WITH-CONNECTION 0))
      Locals:
        SWANK::FUN =3D #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
  4: (SWANK::CALL-WITH-REDIRECTED-IO #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>)
      Locals:
        SWANK::CONNECTION =3D #<SWANK::CONNECTION @ #x207161da>
        FUNCTION =3D #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>
        FUNCTION =3D #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>
        SWANK::IO =3D #<TWO-WAY-STREAM "" @ #x20726182>
        SWANK::IN =3D #<SWANK-BACKEND::SLIME-INPUT-STREAM @ #x20726192>
        SWANK::CONNECTION =3D #<SWANK::CONNECTION @ #x207161da>
        SWANK::OUT =3D #<SWANK-BACKEND::SLIME-OUTPUT-STREAM @ #x207161ca>
  5: (SWANK::MAYBE-CALL-WITH-IO-REDIRECTION
       #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>)
      Locals:
        SWANK::CONNECTION =3D #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN =3D #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0)
  @ #x2127a6aa>
        SWANK::CONNECTION =3D #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN =3D #(#<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0)
  @ #x2127a67a>)
  6: (SWANK::CALL-WITH-CONNECTION #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK::CONNECTION =3D #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN =3D #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
        SWANK::CONNECTION =3D #<SWANK::CONNECTION @ #x207161da>
        SYSTEM::.LAMBDA-LEXICAL-ENV. =3D #(#<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
        SWANK::FUN =3D #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
  7: (SWANK:SWANK-DEBUGGER-HOOK
       #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @ #x2127a602>
       #<Function SWANK-DEBUGGER-HOOK>)
      Locals:
        CONDITION =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        SWANK::HOOK =3D #<Function SWANK-DEBUGGER-HOOK>
        SYSTEM::.LAMBDA-LEXICAL-ENV. =3D #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        CONDITION =3D :UNKNOWN
  8: (INVOKE-DEBUGGER #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
                        #x2127a602>)
      Locals:
        CONDITION =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-0 =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-1 =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-2 =3D T
  9: (SIGNAL #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @ #x2127a602>)
      Locals:
        EXCL::DATUM =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::ARGUMENTS =3D NIL
        EXCL::LOCAL-0 =3D NIL
        EXCL::LOCAL-1 =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-2 =3D ((ERROR . #<Function MAYBE-INVOKE-DEBUGGER>))
        EXCL::LOCAL-3 =3D (ERROR . #<Function MAYBE-INVOKE-DEBUGGER>)
        EXCL::LOCAL-4 =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-5 =3D #\null
        EXCL::LOCAL-6 =3D #<Function MAYBE-INVOKE-DEBUGGER>
 10: (ERROR FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR :FORMAT-CONTROL
            "~S is not a LATIN-1 character." :FORMAT-ARGUMENTS (#\?)
            :STREAM
            #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>)
      Locals:
        EXCL::DATUM =3D FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR
        EXCL::ARGUMENTS =3D (:FORMAT-CONTROL "~S is not a LATIN-1
  character." :FORMAT-ARGUMENTS
                           (#\?) :STREAM
                           #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
  @ #x21279e5a>)
        EXCL::LOCAL-0 =3D (:FORMAT-CONTROL "~S is not a LATIN-1
  character." :FORMAT-ARGUMENTS
                         (#\?) :STREAM
                         #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>)
        EXCL::LOCAL-1 =3D FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR
        EXCL::LOCAL-2 =3D #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-3 =3D 97
        EXCL::LOCAL-4 =3D 1
        EXCL::LOCAL-5 =3D #<non-lisp object @ #xfffffffd>
        EXCL::LOCAL-6 =3D #<non-lisp object @ #x63897f0f>
        :UNKNOWN =3D NIL
        :UNKNOWN =3D NIL
        :UNKNOWN =3D NIL
        :UNKNOWN =3D NIL
        :UNKNOWN =3D NIL
 11: (FLEXI-STREAMS::SIGNAL-ENCODING-ERROR
       #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
       "~S is not a LATIN-1 character." #\?)
      Locals:
        FLEXI-STREAMS:FLEXI-STREAM =3D
  #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        FLEXI-STREAMS::FORMAT-CONTROL =3D "~S is not a LATIN-1 character."
        FLEXI-STREAMS::FORMAT-ARGS =3D (#\?)
        FLEXI-STREAMS::FORMAT-ARGS =3D (#\?)
        FLEXI-STREAMS:FLEXI-STREAM =3D
  #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        FLEXI-STREAMS::FORMAT-CONTROL =3D "~S is not a LATIN-1 character."
 12: ((METHOD STREAM-WRITE-CHAR
       (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
      Locals:
        STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        CHAR =3D #\?
        CHAR =3D #\?
        FLEXI-STREAMS:OCTET =3D 1044
        STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
 13: ((:INTERNAL (:EFFECTIVE-METHOD 2 NIL NIL NIL NIL) 0)
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
      Locals:
        EXCL::METHARG0 =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        EXCL::METHARG1 =3D #\?
        EXCL::LOCAL-0 =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        EXCL::LOCAL-1 =3D #\?
        EXCL::LOCAL-2 =3D #<Closure (:INTERNAL (:EFFECTIVE-METHOD 2 NIL
  NIL NIL NIL) 0) @
                          #x20c1f10a>
        EXCL::LOCAL-3 =3D #<non-lisp object @ #x2>
        EXCL::LOCAL-4 =3D NIL
        EXCL::LOCAL-5 =3D 3
        EXCL::LOCAL-6 =3D 137387322
        EXCL::LOCAL-7 =3D #<Function (METHOD STREAM-WRITE-CHAR
                                    (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STRE=
AM
  T))>
        EXCL::LOCAL-8 =3D 2
 14: ((METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE
       (FLEXI-STREAMS:FLEXI-OUTPUT-STREAM T T T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
      "<table border=3D1>
      =20
         <tr>
          =20
            =20
               <td align=3D\"right\" >zero</td>
              ..)
      Locals:
        FLEXI-STREAMS:FLEXI-OUTPUT-STREAM =3D
              #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        SEQUENCE =3D "<table border=3D1>
                    =20
                       <tr>
                        =20
                          =20
                             <td align=3D\"right\" >zero</td>
                          =20
                        =20
                          =20
                             <td align=3D\"right\" bgcolor=3D\"blue\">some
                             cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::START =3D 173
        FLEXI-STREAMS::END =3D 4537
        SEQUENCE =3D "<table border=3D1>
                    =20
                       <tr>
                        =20
                          =20
                             <td align=3D\"right\" >zero</td>
                          =20
                        =20
                          =20
                             <td align=3D\"right\" bgcolor=3D\"blue\">some
                             cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS:FLEXI-OUTPUT-STREAM =3D
                             #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
                             @ #x21279e5a>
        #:|g1798| =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
                             #x21279e5a>
 15: ((METHOD STREAM-WRITE-STRING
       (TRIVIAL-GRAY-STREAMS:TRIVIAL-GRAY-STREAM-MIXIN T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
      "<table border=3D1>
      =20
         <tr>
          =20
            =20
               <td align=3D\"right\" >zero</td>
              ..)
      Locals:
        STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
              #x21279e5a>
        TRIVIAL-GRAY-STREAMS::SEQ =3D "<table border=3D1>
                                     =20
                                        <tr>
                                         =20
                                           =20
                                              <td align=3D\"right\"
                                              >zero</td>
                                           =20
                                         =20
                                           =20
                                              <td align=3D\"right\"
                                              bgcolor=3D\"blue\">some
                                              cyrillic chars: ????
                                              one</td> ..
        TRIVIAL-GRAY-STREAMS::START =3D 0
        TRIVIAL-GRAY-STREAMS::END =3D NIL
        TRIVIAL-GRAY-STREAMS::START =3D 0
        TRIVIAL-GRAY-STREAMS::END =3D NIL
        STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
                                              #x21279e5a>
 16: (FLEXI-STREAMS:STRING-TO-OCTETS "<table border=3D1>
      =20
         <tr>
          =20
            =20
               <td align=3D\"right\" >zero</td>
            =20
          =20
            =20
               <td align=3D\"right\" bgcolor=3D\"blue\">some cyrillic chars:
               ???? one</td> ..)
      Locals:
        STRING =3D "<table border=3D1>
                  =20
                     <tr>
                      =20
                        =20
                           <td align=3D\"right\" >zero</td>
                        =20
                      =20
                        =20
                           <td align=3D\"right\" bgcolor=3D\"blue\">some
                           cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::EXTERNAL-FORMAT =3D
                           #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                           :EOL-STYLE :LF) @
                                           #x20760e02>
        FLEXI-STREAMS::START =3D :UNSUPPLIED
        FLEXI-STREAMS::END =3D :UNSUPPLIED
        FLEXI-STREAMS::OUT =3D #<FLEXI-STREAMS::VECTOR-OUTPUT-STREAM @
                           #x21279d72>
        NIL =3D 0
        STRING =3D "<table border=3D1>
                  =20
                     <tr>
                      =20
                        =20
                           <td align=3D\"right\" >zero</td>
                        =20
                      =20
                        =20
                           <td align=3D\"right\" bgcolor=3D\"blue\">some
                           cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::START =3D 0
        FLEXI-STREAMS::END =3D NIL
        FLEXI-STREAMS::EXTERNAL-FORMAT =3D
                           #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                           :EOL-STYLE :LF) @
                                           #x20760e02>
 17: (HUNCHENTOOT::COMPUTE-LENGTH "<table border=3D1>
      =20
         <tr>
          =20
            =20
               <td align=3D\"right\" >zero</td>
            =20
          =20
            =20
               <td align=3D\"right\" bgcolor=3D\"blue\">some cyrillic chars:
               ???? one</td> ..)
      Locals:
        NIL =3D "<table border=3D1>
               =20
                  <tr>
                   =20
                     =20
                        <td align=3D\"right\" >zero</td>
                     =20
                   =20
                     =20
                        <td align=3D\"right\" bgcolor=3D\"blue\">some
                        cyrillic chars: ???? one</td> ..
        EXCL::LOCAL-0 =3D "<table border=3D1>
                         =20
                            <tr>
                             =20
                               =20
                                  <td align=3D\"right\" >zero</td>
                               =20
                             =20
                               =20
                                  <td align=3D\"right\"
                                  bgcolor=3D\"blue\">some cyrillic chars:
                                  ???? one</td> ..
        EXCL::LOCAL-1 =3D #<HUNCHENTOOT::REPLY @ #x2126e772>
        EXCL::LOCAL-2 =3D #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                                  :EOL-STYLE :LF) @
                          #x20760e02>
 18: (HUNCHENTOOT::START-OUTPUT "<table border=3D1>
      =20
         <tr>
          =20
            =20
               <td align=3D\"right\" >zero</td>
            =20
          =20
            =20
               <td align=3D\"right\" bgcolor=3D\"blue\">some cyrillic chars:
               ???? one</td> ..)
 19: (HUNCHENTOOT::PROCESS-REQUEST
       ((:HOST . "localhost:8082")
        (:USER-AGENT
         . "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)")
        (:ACCEPT
         . "text/xml,application/xml,application/xhtml+xml,text/html;q=3D0.9,=
text/plain;q=3D0.8,image/png,*/*;q=3D0.5")
        (:ACCEPT-LANGUAGE . "en-us,en;q=3D0.5")
        (:ACCEPT-ENCODING . "gzip,deflate")
        (:ACCEPT-CHARSET . "ISO-8859-1,utf-8;q=3D0.7,*;q=3D0.7")
        (:KEEP-ALIVE . "300") (:CONNECTION . "keep-alive") ..))
 20: (HUNCHENTOOT::PROCESS-CONNECTION #<HUNCHENTOOT::SERVER @
               #x20c14f9a>
       #<MULTIVALENT #1=3Dstream socket #1#connected from
               #2=3Dlocalhost/8082
         to #2#/4142 @ #x211e0dba>)

--===============8255216617068270961==--


From edi@agharta.de Fri May 18 11:58:15 2007
From: Edi Weitz <edi@agharta.de>
To: html-template-devel@common-lisp.net
Subject: Re: [html-template-devel] problem with UTF characters in template
Date: Fri, 18 May 2007 17:58:16 +0200
Message-ID: <uveeq5dk7.fsf@agharta.de>
In-Reply-To: <814pmagnog.fsf@oh177.znain.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============4545503506508825879=="

--===============4545503506508825879==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On Fri, 18 May 2007 11:24:15 -0400, Dimitre Liotev <dl(a)znain.net> wrote:

> I am using the "Simple example" from your page
> http://weitz.de/html-template/, specifying :external-format "UTF-8"
> in the call to create-template-printer:

Is the string "UTF-8" a valid external format specifier on AllegroCL?

> (defparameter *template-dispatcher*
>   (hunchentoot:create-prefix-dispatcher "/template-test/"
>    (lambda ()
>      (let* ((html-template:*string-modifier* #'identity)
>            (rows (loop for i below 49 by 7
>                    collect
>                      (list :cols
>                            (loop for j from i below (+ i 7)
>                               for string =3D (format nil "~R" j)
>                               collect (list :content string=20
>                                             :colorful-style (oddp j))))))
>           (values (list :rows rows)))
>        (with-output-to-string (html-template:*default-template-output*)
>          (html-template:fill-and-print-template
>           (html-template:create-template-printer #p"foo.tmpl" :external-for=
mat "UTF-8")
>           values))))))
>
> [...]
>
>  12: ((METHOD STREAM-WRITE-CHAR
>        (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM T))
>       #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
>       Locals:
>         STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>   #x21279e5a>
>         CHAR =3D #\?
>         CHAR =3D #\?
>         FLEXI-STREAMS:OCTET =3D 1044
>         STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>   #x21279e5a>

The condition is signalled when Hunchentoot is trying to write the
page to the client.  You have to make sure that Hunchentoot knows that
it's supposed to output UTF-8.  See the examples that come with
Hunchentoot for how this can be done.  (And you should also set the
headers accordingly, so the browser knows what to expect.)

--===============4545503506508825879==--


From dl@znain.net Fri May 18 14:55:40 2007
From: Dimitre Liotev <dl@znain.net>
To: html-template-devel@common-lisp.net
Subject: Re: [html-template-devel] problem with UTF characters in template
Date: Fri, 18 May 2007 14:55:33 -0400
Message-ID: <81y7jmezbu.fsf@oh177.znain.com>
In-Reply-To: <uveeq5dk7.fsf@agharta.de>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="===============3679058792614867831=="

--===============3679058792614867831==
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Edi Weitz <edi(a)agharta.de> writes:

> On Fri, 18 May 2007 11:24:15 -0400, Dimitre Liotev <dl(a)znain.net> wrote:
>
>> I am using the "Simple example" from your page
>> http://weitz.de/html-template/, specifying :external-format "UTF-8"
>> in the call to create-template-printer:
>
> Is the string "UTF-8" a valid external format specifier on AllegroCL?

I thought so, but it seems that the format must be specified as a keyword:

CL-USER> (describe (find-if (lambda (x)
                              (member :UTF-8 (excl:ef-nicknames x)))
                            (EXCL:ALL-EXTERNAL-FORMATS)))

#<EXTERNAL-FORMAT :UTF8 ['(:E-CRLF :UTF8-BASE)] @ #x20116f6a> is a
structure of type EXTERNAL-FORMAT.  It has these slots:
 NAME               (:E-CRLF :UTF8-BASE)
 NICKNAMES          (:UTF8 :UTF-8)
...


I thought the string "UTF-8" would also work because when I try WITH-OPEN-FIL=
E =20
with an invalid external format like "UTF-9", an error is thrown, while "UTF-=
8"
is accepted and no error is raised.


>> (defparameter *template-dispatcher*
>>   (hunchentoot:create-prefix-dispatcher "/template-test/"
>>    (lambda ()
>>      (let* ((html-template:*string-modifier* #'identity)
>>            (rows (loop for i below 49 by 7
>>                    collect
>>                      (list :cols
>>                            (loop for j from i below (+ i 7)
>>                               for string =3D (format nil "~R" j)
>>                               collect (list :content string=20
>>                                             :colorful-style (oddp j))))))
>>           (values (list :rows rows)))
>>        (with-output-to-string (html-template:*default-template-output*)
>>          (html-template:fill-and-print-template
>>           (html-template:create-template-printer #p"foo.tmpl" :external-fo=
rmat "UTF-8")
>>           values))))))
>>
>> [...]
>>
>>  12: ((METHOD STREAM-WRITE-CHAR
>>        (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM T))
>>       #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #\?)
>>       Locals:
>>         STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>>   #x21279e5a>
>>         CHAR =3D #\?
>>         CHAR =3D #\?
>>         FLEXI-STREAMS:OCTET =3D 1044
>>         STREAM =3D #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
>>   #x21279e5a>
>
> The condition is signalled when Hunchentoot is trying to write the
> page to the client.  You have to make sure that Hunchentoot knows that
> it's supposed to output UTF-8.  See the examples that come with
> Hunchentoot for how this can be done.  (And you should also set the
> headers accordingly, so the browser knows what to expect.)


Thanks!

For the sake of those who might encounter the same problem:
I added this to my handler function:

  (setf (hunchentoot:reply-external-format) :UTF-8)

This will set the reply-external-format only for the given handler. To
set it globaly use this:

  (setf hunchentoot:*hunchentoot-default-external-format* :UTF-8)



--===============3679058792614867831==--