Here is a small patch to enable support for Unicode for CMUCL.
Seems to work. May need some more work on the *external-format-to-coding-system*, but iso-8859-1 and utf-8 appear to work ok, for the little bit of testing I've done.
Ray
--- swank-cmucl.lisp-orig 2008-06-01 18:06:07.000000000 -0400 +++ swank-cmucl.lisp 2008-06-01 18:01:44.000000000 -0400 @@ -99,10 +99,13 @@ (ext:close-socket fd)))
(defimplementation accept-connection (socket &key - external-format buffering timeout) - (declare (ignore timeout external-format)) + external-format buffering timeout) + (declare (ignore timeout)) (let ((buffering (or buffering :full))) - (make-socket-io-stream (ext:accept-tcp-connection socket) buffering))) + (make-socket-io-stream (ext:accept-tcp-connection socket) + buffering + :external-format + (or external-format :iso-8859-1))))
;;;;; Sockets
@@ -117,10 +120,23 @@ (let ((hostent (ext:lookup-host-entry hostname))) (car (ext:host-entry-addr-list hostent))))
-(defun make-socket-io-stream (fd buffering) +(defvar *external-format-to-coding-system* + '((:iso-8859-1 + "latin-1" "latin-1-unix" "iso-latin-1-unix" + "iso-8859-1" "iso-8859-1-unix") + (:utf-8 "utf-8" "utf-8-unix"))) + +(defimplementation find-external-format (coding-system) + (car (rassoc-if (lambda (x) (member coding-system x :test #'equal)) + *external-format-to-coding-system*))) + + +(defun make-socket-io-stream (fd buffering &key external-format) "Create a new input/output fd-stream for FD." (sys:make-fd-stream fd :input t :output t :element-type 'base-char - :buffering buffering)) + :buffering buffering + #+unicode :external-format + #+unicode external-format))
;;;;; Signal-driven I/O
* Raymond Toy [2008-06-02 00:12+0200] writes:
Here is a small patch to enable support for Unicode for CMUCL.
Seems to work. May need some more work on the *external-format-to-coding-system*, but iso-8859-1 and utf-8 appear to work ok, for the little bit of testing I've done.
Committed.
Helmut.
* Helmut Eller m28wxnn1si.fsf@common-lisp.net : Wrote on Mon, 02 Jun 2008 20:13:01 +0200:
| * Raymond Toy [2008-06-02 00:12+0200] writes: | |> Here is a small patch to enable support for Unicode for CMUCL. |> |> Seems to work. May need some more work on the |> *external-format-to-coding-system*, but iso-8859-1 and utf-8 appear to |> work ok, for the little bit of testing I've done. | | Committed.
The commit calls MAKE-FD-STREAM with an EXTERNAL-FORMAT keyword argument But MAKE-FD-STREAM in code/fd-stream and the structure do not support this yet?
(sys:make-fd-stream fd :input t :output t :element-type 'base-char - :buffering buffering)) + :buffering buffering + #+unicode :external-format + #+unicode external-format))
-- Madhu
Madhu wrote:
- Helmut Eller m28wxnn1si.fsf@common-lisp.net :
Wrote on Mon, 02 Jun 2008 20:13:01 +0200:
| * Raymond Toy [2008-06-02 00:12+0200] writes: | |> Here is a small patch to enable support for Unicode for CMUCL. |> |> Seems to work. May need some more work on the |> *external-format-to-coding-system*, but iso-8859-1 and utf-8 appear to |> work ok, for the little bit of testing I've done. | | Committed.
The commit calls MAKE-FD-STREAM with an EXTERNAL-FORMAT keyword argument But MAKE-FD-STREAM in code/fd-stream and the structure do not support this yet?
(sys:make-fd-stream fd :input t :output t :element-type 'base-char
:buffering buffering))
:buffering buffering#+unicode :external-format#+unicode external-format))
It's only supported if you have a unicode build, which hasn't been officially released yet.
This works for me. To be useful, you might want to do (setf *default-external-format* :utf-8) or something. You need to set up slime too to be able to see something useful.
Ray
* Raymond Toy 4894AA9A.5000804@gmail.com : Wrote on Sat, 02 Aug 2008 14:42:34 -0400: |> The commit calls MAKE-FD-STREAM with an EXTERNAL-FORMAT keyword argument |> But MAKE-FD-STREAM in code/fd-stream and the structure do not support |> this yet? | | It's only supported if you have a unicode build, which hasn't been | officially released yet.
I wanted to point out that the code for SYS:MAKE-FD-STREAM in "code/fd-stream.lisp" in the `unicode-utf16-branch' branch (CVS 2008-07-11) does not support an :EXTERNAL-FORMAT keyword argument. Perhaps there is a mismatch between your build and the CVS sources?
Reverting the committed patch hunk quoted below lets SLIME work with the unicode build (for usual iso-latin-1 interactions at least)
|> (sys:make-fd-stream fd :input t :output t :element-type 'base-char |> - :buffering buffering)) |> + :buffering buffering |> + #+unicode :external-format |> + #+unicode external-format))
| This works for me. To be useful, you might want to do (setf | *default-external-format* :utf-8) or something. You need to set up | slime too to be able to see something useful.
-- Madhu
Madhu wrote:
- Raymond Toy 4894AA9A.5000804@gmail.com :
Wrote on Sat, 02 Aug 2008 14:42:34 -0400: |> The commit calls MAKE-FD-STREAM with an EXTERNAL-FORMAT keyword argument |> But MAKE-FD-STREAM in code/fd-stream and the structure do not support |> this yet? | | It's only supported if you have a unicode build, which hasn't been | officially released yet.
I wanted to point out that the code for SYS:MAKE-FD-STREAM in "code/fd-stream.lisp" in the `unicode-utf16-branch' branch (CVS 2008-07-11) does not support an :EXTERNAL-FORMAT keyword argument. Perhaps there is a mismatch between your build and the CVS sources?
Try a cvs update. The version I have (07-14, rev 1.85.4.1.2.7) has :external-format.
Ray
* Raymond Toy 489640BA.6020200@gmail.com : Wrote on Sun, 03 Aug 2008 19:35:22 -0400:
| Madhu wrote: |> * Raymond Toy 4894AA9A.5000804@gmail.com : |> Wrote on Sat, 02 Aug 2008 14:42:34 -0400: |> |> The commit calls MAKE-FD-STREAM with an EXTERNAL-FORMAT keyword argument |> |> But MAKE-FD-STREAM in code/fd-stream and the structure do not support |> |> this yet? |> | |> | It's only supported if you have a unicode build, which hasn't been |> | officially released yet. |> |> I wanted to point out that the code for SYS:MAKE-FD-STREAM in |> "code/fd-stream.lisp" in the `unicode-utf16-branch' branch (CVS |> 2008-07-11) does not support an :EXTERNAL-FORMAT keyword argument. |> Perhaps there is a mismatch between your build and the CVS sources? | | Try a cvs update. The version I have (07-14, rev 1.85.4.1.2.7) has | :external-format.
This is checked in the `unicode-utf16-extfmt-branch' branch and not the `unicode-utf16-branch' which I was trying. I should probably follow up on the cmucl lists
-- Thanks, Madhu
Madhu wrote:
- Raymond Toy 489640BA.6020200@gmail.com :
Wrote on Sun, 03 Aug 2008 19:35:22 -0400:
| Madhu wrote: |> * Raymond Toy 4894AA9A.5000804@gmail.com : |> Wrote on Sat, 02 Aug 2008 14:42:34 -0400: |> |> The commit calls MAKE-FD-STREAM with an EXTERNAL-FORMAT keyword argument |> |> But MAKE-FD-STREAM in code/fd-stream and the structure do not support |> |> this yet? |> | |> | It's only supported if you have a unicode build, which hasn't been |> | officially released yet. |> |> I wanted to point out that the code for SYS:MAKE-FD-STREAM in |> "code/fd-stream.lisp" in the `unicode-utf16-branch' branch (CVS |> 2008-07-11) does not support an :EXTERNAL-FORMAT keyword argument. |> Perhaps there is a mismatch between your build and the CVS sources? | | Try a cvs update. The version I have (07-14, rev 1.85.4.1.2.7) has | :external-format.
This is checked in the `unicode-utf16-extfmt-branch' branch and not the `unicode-utf16-branch' which I was trying. I should probably follow up on the cmucl lists
Sorry for the delay. Yes, you need to be using the extfmt-branch to get support for external formats. The utf16 branch only basically just changes the internal representation and has minimal support for external format. The extftm-branch has much more complete support. (But still lacks lots of other support for unicode.)
Ray
-
Helmut Eller -
Madhu -
Raymond Toy