Re: [html-template-devel] problem with UTF characters in template

18 May 2007


      Edi Weitz edi@agharta.de writes:
...
On Fri, 18 May 2007 10:01:06 -0400, Dimitre Liotev dl@znain.net wrote:
...
when I have UTF characters in a template I get this error:
#? is not a LATIN-1 character.
   [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]
What external format did you specify?  How about showing us the code
you tried and providing a backtrace?
Cheers,
Edi.
Hi Edi,
I am using the "Simple example" from your page
http://weitz.de/html-template/, specifying :external-format "UTF-8" in
the call to create-template-printer:
(defparameter *template-dispatcher*
  (hunchentoot:create-prefix-dispatcher "/template-test/"
   (lambda ()
     (let* ((html-template:*string-modifier* #'identity)
           (rows (loop for i below 49 by 7
                   collect
                     (list :cols
                           (loop for j from i below (+ i 7)
                              for string = (format nil "~R" j)
                              collect (list :content string 
                                            :colorful-style (oddp j))))))
          (values (list :rows rows)))
       (with-output-to-string (html-template:*default-template-output*)
         (html-template:fill-and-print-template
          (html-template:create-template-printer #p"foo.tmpl" :external-format "UTF-8")
          values))))))
(push *template-dispatcher* hunchentoot:*dispatch-table*)
The template:
<table border=1>
  <!-- TMPL_LOOP rows -->
    <tr>
      <!-- TMPL_LOOP cols -->
        <!-- TMPL_IF colorful-style -->
          <td align="right" bgcolor="blue">some cyrullic chars: ДЖГБ<!-- TMPL_VAR content --></td>
        <!-- TMPL_ELSE -->
          <td align="right" ><!-- TMPL_VAR content --></td>
        <!-- /TMPL_IF -->
      <!-- /TMPL_LOOP -->
    </tr>
  <!-- /TMPL_LOOP -->
</table>
Now if I point the browser to http://localhost:8082/template-test/ it
works fine if the UTF chars are not there, but if I put any cyrillic UTF
encoded character in the template I get the error:
#? is not a LATIN-1 character.
   [Condition of type FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR]
Restarts:
 0: [ABORT] Abort entirely from this (lisp) process.
Backtrace:
  0: ((:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0))
      Locals:
        CONDITION = :UNKNOWN
  1: ((:INTERNAL (:TOP-LEVEL-FORM "swank-backend.lisp" 20112) 0)
      #<Function SWANK-DEBUGGER-HOOK>
      #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK-BACKEND::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
  2: (SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK
       #<Function SWANK-DEBUGGER-HOOK>
       #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK-BACKEND::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        SWANK-BACKEND::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SWANK-BACKEND::FUN = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
  3: ((:INTERNAL SWANK::CALL-WITH-CONNECTION 0))
      Locals:
        SWANK::FUN = #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
  4: (SWANK::CALL-WITH-REDIRECTED-IO #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>)
      Locals:
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        FUNCTION = #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>
        FUNCTION = #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>
        SWANK::IO = #<TWO-WAY-STREAM "" @ #x20726182>
        SWANK::IN = #<SWANK-BACKEND::SLIME-INPUT-STREAM @ #x20726192>
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::OUT = #<SWANK-BACKEND::SLIME-OUTPUT-STREAM @ #x207161ca>
  5: (SWANK::MAYBE-CALL-WITH-IO-REDIRECTION
       #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0) @
  #x2127a6aa>)
      Locals:
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN = #<Closure (:INTERNAL SWANK::CALL-WITH-CONNECTION 0)
  @ #x2127a6aa>
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN = #(#<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0)
  @ #x2127a67a>)
  6: (SWANK::CALL-WITH-CONNECTION #<SWANK::CONNECTION @ #x207161da>
       #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
      Locals:
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SWANK::FUN = #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
        SWANK::CONNECTION = #<SWANK::CONNECTION @ #x207161da>
        SYSTEM::.LAMBDA-LEXICAL-ENV. = #(#<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>)
        SWANK::FUN = #<Closure (:INTERNAL SWANK:SWANK-DEBUGGER-HOOK 0) @
  #x2127a67a>
  7: (SWANK:SWANK-DEBUGGER-HOOK
       #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @ #x2127a602>
       #<Function SWANK-DEBUGGER-HOOK>)
      Locals:
        CONDITION = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        SWANK::HOOK = #<Function SWANK-DEBUGGER-HOOK>
        SYSTEM::.LAMBDA-LEXICAL-ENV. = #<Closure (:INTERNAL
  SWANK:SWANK-DEBUGGER-HOOK 0) @ #x2127a67a>
        CONDITION = :UNKNOWN
  8: (INVOKE-DEBUGGER #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
                        #x2127a602>)
      Locals:
        CONDITION = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-0 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-1 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-2 = T
  9: (SIGNAL #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @ #x2127a602>)
      Locals:
        EXCL::DATUM = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::ARGUMENTS = NIL
        EXCL::LOCAL-0 = NIL
        EXCL::LOCAL-1 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-2 = ((ERROR . #<Function MAYBE-INVOKE-DEBUGGER>))
        EXCL::LOCAL-3 = (ERROR . #<Function MAYBE-INVOKE-DEBUGGER>)
        EXCL::LOCAL-4 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-5 = #\null
        EXCL::LOCAL-6 = #<Function MAYBE-INVOKE-DEBUGGER>
 10: (ERROR FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR :FORMAT-CONTROL
            "~S is not a LATIN-1 character." :FORMAT-ARGUMENTS (#?)
            :STREAM
            #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>)
      Locals:
        EXCL::DATUM = FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR
        EXCL::ARGUMENTS = (:FORMAT-CONTROL "~S is not a LATIN-1
  character." :FORMAT-ARGUMENTS
                           (#?) :STREAM
                           #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
  @ #x21279e5a>)
        EXCL::LOCAL-0 = (:FORMAT-CONTROL "~S is not a LATIN-1
  character." :FORMAT-ARGUMENTS
                         (#?) :STREAM
                         #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>)
        EXCL::LOCAL-1 = FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR
        EXCL::LOCAL-2 = #<FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR @
  #x2127a602>
        EXCL::LOCAL-3 = 97
        EXCL::LOCAL-4 = 1
        EXCL::LOCAL-5 = #<non-lisp object @ #xfffffffd>
        EXCL::LOCAL-6 = #<non-lisp object @ #x63897f0f>
        :UNKNOWN = NIL
        :UNKNOWN = NIL
        :UNKNOWN = NIL
        :UNKNOWN = NIL
        :UNKNOWN = NIL
 11: (FLEXI-STREAMS::SIGNAL-ENCODING-ERROR
       #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
       "~S is not a LATIN-1 character." #?)
      Locals:
        FLEXI-STREAMS:FLEXI-STREAM =
  #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        FLEXI-STREAMS::FORMAT-CONTROL = "~S is not a LATIN-1 character."
        FLEXI-STREAMS::FORMAT-ARGS = (#?)
        FLEXI-STREAMS::FORMAT-ARGS = (#?)
        FLEXI-STREAMS:FLEXI-STREAM =
  #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        FLEXI-STREAMS::FORMAT-CONTROL = "~S is not a LATIN-1 character."
 12: ((METHOD STREAM-WRITE-CHAR
       (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #?)
      Locals:
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        CHAR = #?
        CHAR = #?
        FLEXI-STREAMS:OCTET = 1044
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
 13: ((:INTERNAL (:EFFECTIVE-METHOD 2 NIL NIL NIL NIL) 0)
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a> #?)
      Locals:
        EXCL::METHARG0 = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        EXCL::METHARG1 = #?
        EXCL::LOCAL-0 = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
  #x21279e5a>
        EXCL::LOCAL-1 = #?
        EXCL::LOCAL-2 = #<Closure (:INTERNAL (:EFFECTIVE-METHOD 2 NIL
  NIL NIL NIL) 0) @
                          #x20c1f10a>
        EXCL::LOCAL-3 = #<non-lisp object @ #x2>
        EXCL::LOCAL-4 = NIL
        EXCL::LOCAL-5 = 3
        EXCL::LOCAL-6 = 137387322
        EXCL::LOCAL-7 = #<Function (METHOD STREAM-WRITE-CHAR
                                    (FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
  T))>
        EXCL::LOCAL-8 = 2
 14: ((METHOD TRIVIAL-GRAY-STREAMS:STREAM-WRITE-SEQUENCE
       (FLEXI-STREAMS:FLEXI-OUTPUT-STREAM T T T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
      "<table border=1>
<tr>
<td align="right" >zero</td>
              ..)
      Locals:
        FLEXI-STREAMS:FLEXI-OUTPUT-STREAM =
              #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
        SEQUENCE = "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some
                             cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::START = 173
        FLEXI-STREAMS::END = 4537
        SEQUENCE = "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some
                             cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS:FLEXI-OUTPUT-STREAM =
                             #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM
                             @ #x21279e5a>
        #:|g1798| = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
                             #x21279e5a>
 15: ((METHOD STREAM-WRITE-STRING
       (TRIVIAL-GRAY-STREAMS:TRIVIAL-GRAY-STREAM-MIXIN T))
      #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @ #x21279e5a>
      "<table border=1>
<tr>
<td align="right" >zero</td>
              ..)
      Locals:
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
              #x21279e5a>
        TRIVIAL-GRAY-STREAMS::SEQ = "<table border=1>
<tr>
<td align="right"
                                              >zero</td>
<td align="right"
                                              bgcolor="blue">some
                                              cyrillic chars: ????
                                              one</td> ..
        TRIVIAL-GRAY-STREAMS::START = 0
        TRIVIAL-GRAY-STREAMS::END = NIL
        TRIVIAL-GRAY-STREAMS::START = 0
        TRIVIAL-GRAY-STREAMS::END = NIL
        STREAM = #<FLEXI-STREAMS::FLEXI-LATIN-1-OUTPUT-STREAM @
                                              #x21279e5a>
 16: (FLEXI-STREAMS:STRING-TO-OCTETS "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some cyrillic chars:
               ???? one</td> ..)
      Locals:
        STRING = "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some
                           cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::EXTERNAL-FORMAT =
                           #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                           :EOL-STYLE :LF) @
                                           #x20760e02>
        FLEXI-STREAMS::START = :UNSUPPLIED
        FLEXI-STREAMS::END = :UNSUPPLIED
        FLEXI-STREAMS::OUT = #<FLEXI-STREAMS::VECTOR-OUTPUT-STREAM @
                           #x21279d72>
        NIL = 0
        STRING = "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some
                           cyrillic chars: ???? one</td> ..
        FLEXI-STREAMS::START = 0
        FLEXI-STREAMS::END = NIL
        FLEXI-STREAMS::EXTERNAL-FORMAT =
                           #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                           :EOL-STYLE :LF) @
                                           #x20760e02>
 17: (HUNCHENTOOT::COMPUTE-LENGTH "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some cyrillic chars:
               ???? one</td> ..)
      Locals:
        NIL = "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some
                        cyrillic chars: ???? one</td> ..
        EXCL::LOCAL-0 = "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right"
                                  bgcolor="blue">some cyrillic chars:
                                  ???? one</td> ..
        EXCL::LOCAL-1 = #<HUNCHENTOOT::REPLY @ #x2126e772>
        EXCL::LOCAL-2 = #<FLEXI-STREAMS::EXTERNAL-FORMAT (:ISO-8859-1
                                  :EOL-STYLE :LF) @
                          #x20760e02>
 18: (HUNCHENTOOT::START-OUTPUT "<table border=1>
<tr>
<td align="right" >zero</td>
<td align="right" bgcolor="blue">some cyrillic chars:
               ???? one</td> ..)
 19: (HUNCHENTOOT::PROCESS-REQUEST
       ((:HOST . "localhost:8082")
        (:USER-AGENT
         . "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)")
        (:ACCEPT
         . "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5")
        (:ACCEPT-LANGUAGE . "en-us,en;q=0.5")
        (:ACCEPT-ENCODING . "gzip,deflate")
        (:ACCEPT-CHARSET . "ISO-8859-1,utf-8;q=0.7,*;q=0.7")
        (:KEEP-ALIVE . "300") (:CONNECTION . "keep-alive") ..))
 20: (HUNCHENTOOT::PROCESS-CONNECTION #<HUNCHENTOOT::SERVER @
               #x20c14f9a>
       #<MULTIVALENT #1=stream socket #1#connected from
               #2=localhost/8082
         to #2#/4142 @ #x211e0dba>)

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Re: [html-template-devel] problem with UTF characters in template