On Nov 5, 2009, at 12:04 AM, Samium Gromoff wrote:
From: Robert Strandh strandh@labri.fr
I have another question:
- What's the state of unicode support with McCLIM. I guess this is
backend specific, since SBCL supports unicode strings very nicely. I use McCLIM on Mac OS X and non ASCII keys don't produce anything in McCLIM. Perhaps this is a problem with Mac OS X's X11 server? I've looked into 'freetype' font rendering and (I think) it assumes that char-code is < 256. Is there any unicode support in McCLIM?
There are two distinct issues, input and output.
Output is relatively easy. If we use mcclim-freetype or mcclim-truetype, things work fine it seems. You just need a Unicode font with all the glyphs. For native X11 (bitmap I guess) fonts, it is more complicated, because such fonts are limited to 16-bit glyph numbers, so we need a system for creating groups of fonts and choices of fonts depending on code points that would render in a pleasant way, and I am told this is hard because it depends on the language.
Input is trickier and depends on the backend. Our current problem is that the CLX backend doesn't support the XKEYBOARD extension (I am just repeating what others have told me here), so the McCLIM CLX backend doesn't either. Perhaps the good thing to do is to extend CLX with XKEYBOARD support.
I have posted patches to both CLX and McCLIM enabling XKB and layout switching about a year ago. Unfortunately, while they still apply, they appear to have bitrotten, and while the X layout switching keystroke appears to have effect, inputting any foreign character freezes McCLIM.
If anyone wants to take it from where I left it, here goes.
Sorry if it is an offensive way to publish that, but, for what it's worth, the changes can be accessed via git:
git://git.feelingofgreen.ru/mcclim, 'xkb-support' branch
and
git://git.feelingofgreen.ru/clx, 'xkb-support' branch
regards, Samium Gromoff -- _deepfire-at-feelingofgreen.ru O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
I've looked at what you did in xkb-support. Turns out it was mostly a matter of defining additional keysyms in clx/keysyms.lisp. I don't know about XKB - from what I googled it doesn't matter - but I know near to zero about X in general.
Anyways - I've converted http://www.cl.cam.ac.uk/~mgk25/ucs/keysym2ucs.c to CLX form. Btw this C file is probably the standard solution for unicode input in X clientwise - I've seen it in GLFW library which has crossplatform unicode input and has X11 as one of its backends.
So when I add all these define-keysyms into clx/keysyms.lisp, unicode in McCLIM 'just works'. Let me elaborate: - if :mcclim-truetype is not loaded, you get an error ( .. not unsigned byte 8) - seems the default font rendering is not robust enough - it should just spit out black squares or something when (> char-code 255), rather than dying.. - if :mcclim-truetype is loaded, unicode input 'just works' in 'gadgets test's text editor! But it doesn't work in goatee test! I don't know enough about internals to tell what's wrong.
-- another thing regarding CLX's conversion of keysyms->chars - the ASCII region of keysyms doesn't require a hashtable lookup and is the used most of the time. The conversion for ASCII region is trivial. I don't know how important this is performance-wise but if you look at the end of http://www.cl.cam.ac.uk/~mgk25/ucs/keysym2ucs.c -- it's probably best to do a check for ASCII region and return immediately, why waste cycles and memory lookups. Perhaps CLX requires a more general approach - I don't know, please tell me. Otherwise the C file is probably a good example how this should be done. CLX has a more general approach - in the hashtables there are lists stored, which again is not that efficient. Well perhaps the cost is negligible (or it is for SBCL! other lisps might be less efficient).. Please comment on this if you have experience hacking CLX - I'm interested in the rationale behing CLX's approach.
I've attached the UTF-8 encoded converted C file .. the contents must be put at the end of clx/keysyms.lisp. Perhaps this should be included in the main CLX branch (if it even exists) since it looks like a pretty general solution for all unicode keys (better than every CLX user rolling his own version).
Regards, Ziga