babel-devel

babel-devel@common-lisp.net

51 discussions

[babel-devel] Problem with gbk-map.lisp
by Luís Oliveira 03 Jan '12

03 Jan '12

Hello Wenpeng, I missed an obvious problem with your patch, it depends on Lisps loading gbk-map.lisp using the UTF-8 encoding. AFAIK, there's no good way to portably enforce that via ASDF, so I've converted the file to an ASCII representation. It's not as pretty as the previous version, but it seems to work. Cheers, -- Luís Oliveira http://r42.eu/~luis/

2 1

Re: [babel-devel] Submit a GBK patch
by Xiaofeng Yang 29 Dec '11

29 Dec '11

I suggest this for both GB2312 and GBK encoding (this 2 encodings are very important for Chinese users): http://trac.clozure.com/ccl/changeset/14911 I've tested this for CCL using the encoding tables and tests from GNU's iconv. Most babel code came from CCL. I think to integrated this for babel is easily and more suitable though it just only for CCL now. Best regards, Xiaofeng Yang 2011/12/21 levin <levin108(a)gmail.com> > HI,all > > Days ago, I found this project babel, and tried to use it in my project, > as I'm Chinese, > I found it doesn't support GBK encoding in babel, so I wrote a patch for > babel to make > it support GBK. > > I just used common lisp for not a long time, so if there was something not > good enough, please let me know, I also hope you can accept this patch or > help me > modify it to make babel support GBK, so we can use it freely to process > Chinese text. > > Thanks. > > -- > levin > twitter: @levin108 <http://twitter.com/levin108> > blog: basiccoder.com > > > _______________________________________________ > babel-devel mailing list > babel-devel(a)common-lisp.net > http://lists.common-lisp.net/cgi-bin/mailman/listinfo/babel-devel > >

2 1

[babel-devel] Unit tests failures on different lisps
by Anton Vodonosov 28 Dec '11

28 Dec '11

Hello. I am running tests of some most often Quicklisp-downloaded libraries, including babel. Babel tests have different number of failures/errors on different Lisps (about 8, 9 or 5). You may find the results here: http://common-lisp.net/project/cl-test-grid/pivot_ql-lib_lisp.html Clicking the ok/fail status refer to the library test logs where you may find what failures occurred. Best regards, - Anton

1 0

[babel-devel] Submit a GBK patch
by levin 21 Dec '11

21 Dec '11

HI,all Days ago, I found this project babel, and tried to use it in my project, as I'm Chinese, I found it doesn't support GBK encoding in babel, so I wrote a patch for babel to make it support GBK. I just used common lisp for not a long time, so if there was something not good enough, please let me know, I also hope you can accept this patch or help me modify it to make babel support GBK, so we can use it freely to process Chinese text. Thanks. -- levin twitter: @levin108 <http://twitter.com/levin108> blog: basiccoder.com

1 0

[babel-devel] Few fixes for ucs-2 and utf-32
by Dmitry Ignatiev 14 Nov '11

14 Nov '11

Hi there. There were some BOM-related bugs in unicode decoders. patch attached

2 3

[babel-devel] octets-to-string with UTF8 and Byte Order Marker
by Rob Blackwell 11 May '11

11 May '11

Hi, I have some byte arrays which are UTF8 and some which are UTF8 with byte order markers. I can convert these arrays to strings using > (babel:octets-to-string foo) and > (babel:octets-to-string foo :start 3) respectively, but I'm currently having to figure out whether there is a BOM, like this > (subseq foo 0 3) #(239 187 191) If I use (babel:octets-to-string foo) on a byte array with BOM markers, then my SBCL Lisp image dies. Is there a better way to ask Babel to discover the correct encoding by looking for Byte Order Marks? Ideally I'd like one function call that worked with any array and figured out which encoding was being used automatically and works whether or not a BOM is present? Sorry if I'm missing something obvious, I'm a Babel newbie .. Any guidance or code samples gratefully received. Thanks, Rob.

2 4

[babel-devel] patch for cp1252
by Nicolas Martyanoff 10 May '11

10 May '11

Hi, I added support for the cp1252 encoding: I hope you will find it useful. Regards, -- Nicolas Martyanoff http://codemore.org khaelin(a)gmail.com

2 1

[babel-devel] UTF-16/32 encodings with explicit endianness
by Dmitry Ignatiev 04 Nov '10

04 Nov '10

Hi there. I've added support for UTF-16LE, UTF-16BE, UTF-32LE and UTF-32BE. Here's the new version of enc-unicode.lisp

3 8

[babel-devel] Unicode encodings with explicit endianness
by Dmitry Ignatiev 28 Oct '10

28 Oct '10

Hello again, Luís Please, can you hurry up a bit with applying my unicode-related patch? I'm currently in process of developing a library, a binding to ms windows api: http://github.com/Lovesan/doors And the reason of me being worried about that patch is because my library is unable to correctly operate on windows unicode strings without UTF-16LE support in babel.

1 0

[babel-devel] question about #\Nul char and Unicode
by Vsevolod Dyomkin 07 Aug '10

07 Aug '10

Hi, I'm stuck with a problem: I'm using CL-ZMQ, that in turn uses CFFI, that in turn uses BABEL for such tasks as FOREIGN-STRING-TO-LISP conversion. There seams to be a problem with 0 (#\Nul) characters for such strings, which can be seen below: Illegal :UTF-8 character starting at position 328. [Condition of type BABEL-ENCODINGS:INVALID-UTF8-CONTINUATION-BYTE] Restarts: ... Backtrace: 0: ((LAMBDA (BABEL-ENCODINGS::SRC BABEL-ENCODINGS::START BABEL-ENCODINGS::END BABEL-ENCODINGS::DEST BABEL-ENCODINGS::D-START)) ..) 1: (CFFI:FOREIGN-STRING-TO-LISP #.(SB-SYS:INT-SAP #X0808E13C))[:EXTERNAL] ... The translated string in the current example is this: #(#\5 #\4 #\c #\6 #\7 #\5 #\5 #\b #\- #\9 #\6 #\2 #\8 #\- #\4 #\0 #\a #\4 #\- #\9 #\a #\2 #\d #\- #\c #\c #\8 #\2 #\a #\8 #\1 #\6 #\3 #\4 #\5 #\e #\ #\1 #\8 #\ #\/ #\h #\a #\n #\d #\l #\e #\r #\t #\e #\s #\t #\ #\2 #\6 #\0 #\Space #\{ #\" #\P #\A #\T #\H #\" #\Space #\" #\/ #\h #\a #\n #\d #\l #\e #\r #\t #\e #\s #\t #\" #\, #\" #\M #\E #\T #\H #\O #\D #\" #\Space #\" #\G #\E #\T #\" #\, #\" #\V #\E #\R #\S #\I #\O #\N #\" #\Space #\" #\H #\T #\T #\P #\/ #\1 #\. #\1 #\" #\, #\" #\U #\R #\I #\" #\Space #\" #\/ #\h #\a #\n #\d #\l #\e #\r #\t #\e #\s #\t #\" #\, #\" #\P #\A #\T #\T #\E #\R #\N #\" #\Space #\" #\/ #\h #\a #\n #\d #\l #\e #\r #\t #\e #\s #\t #\" #\, #\" #\A #\c #\c #\e #\p #\t #\" #\Space #\" #\* #\/ #\* #\" #\, #\" #\H #\o #\s #\t #\" #\Space #\" #\l #\o #\c #\a #\l #\h #\o #\s #\t #\Space #\6 #\7 #\6 #\7 #\" #\, #\" #\U #\s #\e #\r #\- #\A #\g #\e #\n #\t #\" #\Space #\" #\c #\u #\r #\l #\/ #\7 #\. #\2 #\0 #\. #\0 #\ #\( #\i #\4 #\8 #\6 #\- #\p #\c #\- #\l #\i #\n #\u #\x #\- #\g #\n #\u #\) #\ #\l #\i #\b #\c #\u #\r #\l #\/ #\7 #\. #\2 #\0 #\. #\0 #\ #\O #\p #\e #\n #\S #\S #\L #\/ #\0 #\. #\9 #\. #\8 #\n #\ #\z #\l #\i #\b #\/ #\1 #\. #\2 #\. #\3 #\. #\4 #\ #\l #\i #\b #\i #\d #\n #\/ #\1 #\. #\1 #\5 #\ #\l #\i #\b #\s #\s #\h #\2 #\/ #\1 #\. #\2 #\. #\4 #\" #\} #\, #\0 #\Space #\, #\n #\S #\S #\L #\/ #\0 #\. #\Nul #\Nul) Maybe, someone here can explain, why this 0-characters are not recognized as proper utf-8 ones? Thanks! Vsevolod

2 7

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

babel-devel