November 2022 - cmucl-cvs - mailman3.common-lisp.net

[Git][cmucl/cmucl] Pushed new branch issue-139-set-filename-encoding-to-utf8
by Raymond Toy (＠rtoy) 28 Nov '22

28 Nov '22

Raymond Toy pushed new branch issue-139-set-filename-encoding-to-utf8 at cmucl / cmucl -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/tree/issue-139-set-filename-en… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-set-terminal-to-utf8] 2 commits: Use stream::find-external-format to determine the formats
by Raymond Toy (＠rtoy) 26 Nov '22

26 Nov '22

Raymond Toy pushed to branch issue-139-set-terminal-to-utf8 at cmucl / cmucl Commits: 29bc9639 by Raymond Toy at 2022-11-25T21:39:24-08:00 Use stream::find-external-format to determine the formats The `:locale` format can be set to an alias, which itself is an alias. We want to use the final underlying format so use `stream::find-external-format` to do that and extract the name from that to use as the alias for `:locate`. - - - - - ed90e7e0 by Raymond Toy at 2022-11-26T06:54:32-08:00 Use stream::find-external-format to determine the formats The `:locale` format can be set to an alias, which itself is an alias. We want to use the final underlying format so use `stream::find-external-format` to do that and extract the name from that to use as the alias for `:locate`. - - - - - 1 changed file: - src/code/save.lisp Changes: ===================================== src/code/save.lisp ===================================== @@ -152,13 +152,15 @@ (setf (gethash :locale stream::*external-format-aliases*) *default-external-format*)) (t - (let ((codeset-format (intern codeset "KEYWORD"))) - ;; If we know the format, we can set the alias. - ;; Otherwise, print a warning and use :iso8859-1 as the + ;; If we know the format. This could be an alias to + ;; another format and so on, so use FIND-EXTERNAL-FORMAT + ;; to determine the final format and use that as the ;; alias. + (let* ((codeset-format (intern codeset "KEYWORD")) + (final-format (stream::find-external-format codeset-format))) (setf (gethash :locale stream::*external-format-aliases*) - (if (stream::find-external-format codeset-format nil) - codeset-format + (if final-format + (stream::ef-name final-format) (progn (warn "Unsupported external format; using :iso8859-1 instead: ~S" codeset-format) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/967f32851d4f265a2fc0b9… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/967f32851d4f265a2fc0b9… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-set-terminal-to-utf8] Change test to use the alias for :locale
by Raymond Toy (＠rtoy) 26 Nov '22

26 Nov '22

Raymond Toy pushed to branch issue-139-set-terminal-to-utf8 at cmucl / cmucl Commits: 967f3285 by Raymond Toy at 2022-11-25T19:18:25-08:00 Change test to use the alias for :locale Previously we tested that `sys:*stdin*` and friends had an external format of `:utf-8`. Instead, get the alias for `:locale` and verify that `sys:*stdin*` and friends have that as the external format. - - - - - 1 changed file: - tests/issues.lisp Changes: ===================================== tests/issues.lisp ===================================== @@ -701,13 +701,17 @@ (define-test issue.139-default-external-format (:tag :issues) (assert-eq :utf-8 stream:*default-external-format*) - (assert-eq :utf-8 (stream-external-format sys:*stdin*)) - (assert-eq :utf-8 (stream-external-format sys:*stdout*)) - (assert-eq :utf-8 (stream-external-format sys:*stderr*)) - ;; sys:*tty* can either be an fd-stream or a two-way-stream. - ;; stream-external-format doesn't work for a two-way-stream. - (unless (typep sys:*tty* 'two-way-stream) - (assert-eq :utf-8 (stream-external-format sys:*tty*)))) + ;; Find the alias for :locale, and verify it exists and verify that + ;; the system streams have that format. + (let ((locale-format (gethash :locale stream::*external-format-aliases*))) + (assert locale-format) + (assert-eq locale-format (stream-external-format sys:*stdin*)) + (assert-eq locale-format (stream-external-format sys:*stdout*)) + (assert-eq locale-format (stream-external-format sys:*stderr*)) + ;; sys:*tty* can either be an fd-stream or a two-way-stream. + ;; stream-external-format doesn't work for a two-way-stream. + (unless (typep sys:*tty* 'two-way-stream) + (assert-eq locale-format (stream-external-format sys:*tty*))))) (define-test issue.139-default-external-format-read-file (:tag :issues) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/967f32851d4f265a2fc0b96… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/967f32851d4f265a2fc0b96… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl] Pushed new branch issue-139-set-terminal-to-utf8
by Raymond Toy (＠rtoy) 26 Nov '22

26 Nov '22

Raymond Toy pushed new branch issue-139-set-terminal-to-utf8 at cmucl / cmucl -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/tree/issue-139-set-terminal-to… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-filename-encoding-utf8] 5 commits: Fix #140: External format for streams that are not file-streams
by Raymond Toy (＠rtoy) 25 Nov '22

25 Nov '22

Raymond Toy pushed to branch issue-139-filename-encoding-utf8 at cmucl / cmucl Commits: e7459829 by Raymond Toy at 2022-11-25T15:35:51+00:00 Fix #140: External format for streams that are not file-streams - - - - - 88843edc by Raymond Toy at 2022-11-25T15:35:52+00:00 Merge branch 'issue-140-stream-element-type-two-way-stream' into 'master' Fix #140: External format for streams that are not file-streams Closes #140 See merge request cmucl/cmucl!97 - - - - - 225940e4 by Raymond Toy at 2022-11-25T16:07:57+00:00 Address #139: Add :locale external format - - - - - bea34994 by Raymond Toy at 2022-11-25T16:07:57+00:00 Merge branch 'issue-139-add-alias-local-external-format' into 'master' Address #139: Add :locale external format See merge request cmucl/cmucl!102 - - - - - 73b09b08 by Raymond Toy at 2022-11-25T10:14:09-08:00 Merge branch 'master' into issue-139-filename-encoding-utf8 - - - - - 8 changed files: - src/code/save.lisp - src/code/stream.lisp - src/code/unix.lisp - src/general-info/release-21e.md - src/i18n/locale/cmucl-unix.pot - src/i18n/locale/cmucl.pot - src/lisp/os-common.c - tests/issues.lisp Changes: ===================================== src/code/save.lisp ===================================== @@ -142,6 +142,30 @@ (file c-call:c-string) (initial-function (alien:unsigned #.vm:word-bits))) +(defun set-up-locale-external-format () + "Add external format alias for :locale to the format specified by + the locale as set by setlocale(3C)." + (let ((codeset (unix::unix-get-locale-codeset))) + (cond ((zerop (length codeset)) + ;; Codeset was the empty string, so just set :locale to + ;; alias to the default external format. + (setf (gethash :locale stream::*external-format-aliases*) + *default-external-format*)) + (t + (let ((codeset-format (intern codeset "KEYWORD"))) + ;; If we know the format, we can set the alias. + ;; Otherwise, print a warning and use :iso8859-1 as the + ;; alias. + (setf (gethash :locale stream::*external-format-aliases*) + (if (stream::find-external-format codeset-format nil) + codeset-format + (progn + (warn "Unsupported external format; using :iso8859-1 instead: ~S" + codeset-format) + :iso8859-1))))))) + (values)) + + (defun save-lisp (core-file-name &key (purify t) (root-structures ()) @@ -252,8 +276,13 @@ ;; Set the runtime locale (unless (zerop (unix::unix-setlocale)) (warn "os_setlocale failed")) + ;; Load external format aliases now so we can aliases to + ;; specify the external format. + (stream::load-external-format-aliases) ;; Set the locale for lisp (intl::setlocale) + ;; Set up :locale format + (set-up-locale-external-format) (ext::process-command-strings process-command-line) (setf *editor-lisp-p* nil) (macrolet ((find-switch (name) ===================================== src/code/stream.lisp ===================================== @@ -290,13 +290,21 @@ (stream-dispatch stream ;; simple-stream (stream::%stream-external-format stream) - ;; lisp-stream - (typecase stream + ;; lisp-stream. For unsupported streams, signal a type error. + (etypecase stream #+unicode (fd-stream (fd-stream-external-format stream)) - (synonym-stream (stream-external-format - (symbol-value (synonym-stream-symbol stream)))) - (t :default)) + (broadcast-stream + ;; See http://www.lispworks.com/documentation/HyperSpec/Body/t_broadc.htm + (let ((components (broadcast-stream-streams stream))) + (if (null components) + :default + (stream-external-format (car (last components)))))) + (synonym-stream + ;; Not defined by CLHS. What should happen if + ;; (synonym-stream-symbol stream) is unbound? + (stream-external-format + (symbol-value (synonym-stream-symbol stream))))) ;; fundamental-stream :default)) ===================================== src/code/unix.lisp ===================================== @@ -2918,3 +2918,10 @@ 256))) (when (zerop result) (cast buf c-call:c-string))))) + +(defun unix-get-locale-codeset () + _N"Get the codeset from the locale" + (cast (alien-funcall + (extern-alien "os_get_locale_codeset" + (function (* char)))) + c-string)) ===================================== src/general-info/release-21e.md ===================================== @@ -59,7 +59,8 @@ public domain. * ~~#132~~ Ansi test `RENAME-FILE.1` no fails * ~~#134~~ Handle the case of `(expt complex complex-rational)` * ~~#136~~ `ensure-directories-exist` should return the given pathspec - * #139 `*default-external-format*` defaults to `:utf-8` + * #139 `*default-external-format*` defaults to `:utf-8`; add alias for `:locale` external format + * ~~#140~~ External format for streams that are not `file-stream`'s * ~~#141~~ Disallow locales that are pathnames to a localedef file * ~~#142~~ `(random 0)` signals incorrect error * ~~#147~~ `stream-line-column` method missing for `fundamental-character-output-stream` ===================================== src/i18n/locale/cmucl-unix.pot ===================================== @@ -1435,3 +1435,7 @@ msgid "" " calling this so that the correct locale is returned." msgstr "" +#: src/code/unix.lisp +msgid "Get the codeset from the locale" +msgstr "" + ===================================== src/i18n/locale/cmucl.pot ===================================== @@ -6714,6 +6714,12 @@ msgid "" "This is true if and only if the lisp was started with the -edit switch." msgstr "" +#: src/code/save.lisp +msgid "" +"Add external format alias for :locale to the format specified by\n" +" the locale as set by setlocale(3C)." +msgstr "" + #: src/code/save.lisp msgid "" "Saves a CMU Common Lisp core image in the file of the specified name. The\n" ===================================== src/lisp/os-common.c ===================================== @@ -7,6 +7,7 @@ #include <assert.h> #include <errno.h> +#include <langinfo.h> #include <locale.h> #include <math.h> #include <netdb.h> @@ -796,3 +797,9 @@ os_get_lc_messages(char *buf, int len) /* Return -1 if setlocale failed. */ return locale ? 0 : -1; } + +char * +os_get_locale_codeset() +{ + return nl_langinfo(CODESET); +} ===================================== tests/issues.lisp ===================================== @@ -766,6 +766,57 @@ (assert-equal (map 'list #'char-name string) (map 'list #'char-name (read-line s)))))) +(define-test issue.139-locale-external-format + (:tag :issues) + ;; Just verify that :locale format exists + (assert-true (stream::find-external-format :locale nil))) + +;;; Test stream-external-format for various types of streams. + +(define-test issue.140.two-way-stream + (:tag :issues) + (with-open-file (in (merge-pathnames "issues.lisp" cmucl-test-runner::*load-path*) + :direction :input + :external-format :utf-8) + (with-open-file (out "/tmp/output.tst" + :direction :output + :external-format :utf-8 + :if-exists :supersede) + (let ((two-way-stream (make-two-way-stream in out))) + (assert-error 'type-error + (stream-external-format two-way-stream)))))) + +;; Test synonym-stream returns the format of the underlying stream. +(define-test issue.140.synonym-stream + (:tag :issues) + (with-open-file (s (merge-pathnames "issues.lisp" cmucl-test-runner::*load-path*) + :direction :input + :external-format :iso8859-1) + (let ((syn (make-synonym-stream '*syn-stream*))) + (setf syn s) + (assert-equal :iso8859-1 (stream-external-format syn))))) + +(define-test issue.140.broadcast-stream + (:tag :issues) + ;; Create 3 output streams. The exact external formats aren't + ;; really important here as long as they're different for each file + ;; so we can tell if we got the right answer. + (with-open-file (s1 "/tmp/broad-1" + :direction :output + :if-exists :supersede + :external-format :latin1) + (with-open-file (s2 "/tmp/broad-2" + :direction :output + :if-exists :supersede + :external-format :utf-8) + (with-open-file (s3 "/tmp/broad-3" + :direction :output + :if-exists :supersede + :external-format :utf-16) + ;; The format must be the value from the last stream. + (assert-equal :utf-16 + (stream-external-format + (make-broadcast-stream s1 s2 s3))))))) (define-test issue.150 (:tag :issues) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/8149dbd2682f2e37a6103b… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/8149dbd2682f2e37a6103b… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][master] 2 commits: Address #139: Add :locale external format
by Raymond Toy (＠rtoy) 25 Nov '22

25 Nov '22

Raymond Toy pushed to branch master at cmucl / cmucl Commits: 225940e4 by Raymond Toy at 2022-11-25T16:07:57+00:00 Address #139: Add :locale external format - - - - - bea34994 by Raymond Toy at 2022-11-25T16:07:57+00:00 Merge branch 'issue-139-add-alias-local-external-format' into 'master' Address #139: Add :locale external format See merge request cmucl/cmucl!102 - - - - - 7 changed files: - src/code/save.lisp - src/code/unix.lisp - src/general-info/release-21e.md - src/i18n/locale/cmucl-unix.pot - src/i18n/locale/cmucl.pot - src/lisp/os-common.c - tests/issues.lisp Changes: ===================================== src/code/save.lisp ===================================== @@ -142,6 +142,30 @@ (file c-call:c-string) (initial-function (alien:unsigned #.vm:word-bits))) +(defun set-up-locale-external-format () + "Add external format alias for :locale to the format specified by + the locale as set by setlocale(3C)." + (let ((codeset (unix::unix-get-locale-codeset))) + (cond ((zerop (length codeset)) + ;; Codeset was the empty string, so just set :locale to + ;; alias to the default external format. + (setf (gethash :locale stream::*external-format-aliases*) + *default-external-format*)) + (t + (let ((codeset-format (intern codeset "KEYWORD"))) + ;; If we know the format, we can set the alias. + ;; Otherwise, print a warning and use :iso8859-1 as the + ;; alias. + (setf (gethash :locale stream::*external-format-aliases*) + (if (stream::find-external-format codeset-format nil) + codeset-format + (progn + (warn "Unsupported external format; using :iso8859-1 instead: ~S" + codeset-format) + :iso8859-1))))))) + (values)) + + (defun save-lisp (core-file-name &key (purify t) (root-structures ()) @@ -252,8 +276,13 @@ ;; Set the runtime locale (unless (zerop (unix::unix-setlocale)) (warn "os_setlocale failed")) + ;; Load external format aliases now so we can aliases to + ;; specify the external format. + (stream::load-external-format-aliases) ;; Set the locale for lisp (intl::setlocale) + ;; Set up :locale format + (set-up-locale-external-format) (ext::process-command-strings process-command-line) (setf *editor-lisp-p* nil) (macrolet ((find-switch (name) ===================================== src/code/unix.lisp ===================================== @@ -2915,3 +2915,10 @@ 256))) (when (zerop result) (cast buf c-call:c-string))))) + +(defun unix-get-locale-codeset () + _N"Get the codeset from the locale" + (cast (alien-funcall + (extern-alien "os_get_locale_codeset" + (function (* char)))) + c-string)) ===================================== src/general-info/release-21e.md ===================================== @@ -59,7 +59,7 @@ public domain. * ~~#132~~ Ansi test `RENAME-FILE.1` no fails * ~~#134~~ Handle the case of `(expt complex complex-rational)` * ~~#136~~ `ensure-directories-exist` should return the given pathspec - * #139 `*default-external-format*` defaults to `:utf-8` + * #139 `*default-external-format*` defaults to `:utf-8`; add alias for `:locale` external format * ~~#140~~ External format for streams that are not `file-stream`'s * ~~#141~~ Disallow locales that are pathnames to a localedef file * ~~#142~~ `(random 0)` signals incorrect error ===================================== src/i18n/locale/cmucl-unix.pot ===================================== @@ -1435,3 +1435,7 @@ msgid "" " calling this so that the correct locale is returned." msgstr "" +#: src/code/unix.lisp +msgid "Get the codeset from the locale" +msgstr "" + ===================================== src/i18n/locale/cmucl.pot ===================================== @@ -6714,6 +6714,12 @@ msgid "" "This is true if and only if the lisp was started with the -edit switch." msgstr "" +#: src/code/save.lisp +msgid "" +"Add external format alias for :locale to the format specified by\n" +" the locale as set by setlocale(3C)." +msgstr "" + #: src/code/save.lisp msgid "" "Saves a CMU Common Lisp core image in the file of the specified name. The\n" ===================================== src/lisp/os-common.c ===================================== @@ -7,6 +7,7 @@ #include <assert.h> #include <errno.h> +#include <langinfo.h> #include <locale.h> #include <math.h> #include <netdb.h> @@ -796,3 +797,9 @@ os_get_lc_messages(char *buf, int len) /* Return -1 if setlocale failed. */ return locale ? 0 : -1; } + +char * +os_get_locale_codeset() +{ + return nl_langinfo(CODESET); +} ===================================== tests/issues.lisp ===================================== @@ -745,6 +745,11 @@ (assert-equal (map 'list #'char-name string) (map 'list #'char-name (read-line s)))))) +(define-test issue.139-locale-external-format + (:tag :issues) + ;; Just verify that :locale format exists + (assert-true (stream::find-external-format :locale nil))) + ;;; Test stream-external-format for various types of streams. (define-test issue.140.two-way-stream @@ -792,7 +797,6 @@ (stream-external-format (make-broadcast-stream s1 s2 s3))))))) - (define-test issue.150 (:tag :issues) (let ((ext:*gc-verbose* nil) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/88843edcf0a2968c8da4b0… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/88843edcf0a2968c8da4b0… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-add-alias-local-external-format] 4 commits: Fix #140: External format for streams that are not file-streams
by Raymond Toy (＠rtoy) 25 Nov '22

25 Nov '22

Raymond Toy pushed to branch issue-139-add-alias-local-external-format at cmucl / cmucl Commits: e7459829 by Raymond Toy at 2022-11-25T15:35:51+00:00 Fix #140: External format for streams that are not file-streams - - - - - 88843edc by Raymond Toy at 2022-11-25T15:35:52+00:00 Merge branch 'issue-140-stream-element-type-two-way-stream' into 'master' Fix #140: External format for streams that are not file-streams Closes #140 See merge request cmucl/cmucl!97 - - - - - 85ff8a6a by Raymond Toy at 2022-11-25T07:39:38-08:00 Merge branch 'master' into issue-139-add-alias-local-external-format - - - - - 0ba8661b by Raymond Toy at 2022-11-25T07:43:51-08:00 Add a blank line that was removed when fixing merge conflict - - - - - 3 changed files: - src/code/stream.lisp - src/general-info/release-21e.md - tests/issues.lisp Changes: ===================================== src/code/stream.lisp ===================================== @@ -290,13 +290,21 @@ (stream-dispatch stream ;; simple-stream (stream::%stream-external-format stream) - ;; lisp-stream - (typecase stream + ;; lisp-stream. For unsupported streams, signal a type error. + (etypecase stream #+unicode (fd-stream (fd-stream-external-format stream)) - (synonym-stream (stream-external-format - (symbol-value (synonym-stream-symbol stream)))) - (t :default)) + (broadcast-stream + ;; See http://www.lispworks.com/documentation/HyperSpec/Body/t_broadc.htm + (let ((components (broadcast-stream-streams stream))) + (if (null components) + :default + (stream-external-format (car (last components)))))) + (synonym-stream + ;; Not defined by CLHS. What should happen if + ;; (synonym-stream-symbol stream) is unbound? + (stream-external-format + (symbol-value (synonym-stream-symbol stream))))) ;; fundamental-stream :default)) ===================================== src/general-info/release-21e.md ===================================== @@ -60,6 +60,7 @@ public domain. * ~~#134~~ Handle the case of `(expt complex complex-rational)` * ~~#136~~ `ensure-directories-exist` should return the given pathspec * #139 `*default-external-format*` defaults to `:utf-8`; add alias for `:locale` external format + * ~~#140~~ External format for streams that are not `file-stream`'s * ~~#141~~ Disallow locales that are pathnames to a localedef file * ~~#142~~ `(random 0)` signals incorrect error * ~~#147~~ `stream-line-column` method missing for `fundamental-character-output-stream` ===================================== tests/issues.lisp ===================================== @@ -750,6 +750,53 @@ ;; Just verify that :locale format exists (assert-true (stream::find-external-format :locale nil))) +;;; Test stream-external-format for various types of streams. + +(define-test issue.140.two-way-stream + (:tag :issues) + (with-open-file (in (merge-pathnames "issues.lisp" cmucl-test-runner::*load-path*) + :direction :input + :external-format :utf-8) + (with-open-file (out "/tmp/output.tst" + :direction :output + :external-format :utf-8 + :if-exists :supersede) + (let ((two-way-stream (make-two-way-stream in out))) + (assert-error 'type-error + (stream-external-format two-way-stream)))))) + +;; Test synonym-stream returns the format of the underlying stream. +(define-test issue.140.synonym-stream + (:tag :issues) + (with-open-file (s (merge-pathnames "issues.lisp" cmucl-test-runner::*load-path*) + :direction :input + :external-format :iso8859-1) + (let ((syn (make-synonym-stream '*syn-stream*))) + (setf syn s) + (assert-equal :iso8859-1 (stream-external-format syn))))) + +(define-test issue.140.broadcast-stream + (:tag :issues) + ;; Create 3 output streams. The exact external formats aren't + ;; really important here as long as they're different for each file + ;; so we can tell if we got the right answer. + (with-open-file (s1 "/tmp/broad-1" + :direction :output + :if-exists :supersede + :external-format :latin1) + (with-open-file (s2 "/tmp/broad-2" + :direction :output + :if-exists :supersede + :external-format :utf-8) + (with-open-file (s3 "/tmp/broad-3" + :direction :output + :if-exists :supersede + :external-format :utf-16) + ;; The format must be the value from the last stream. + (assert-equal :utf-16 + (stream-external-format + (make-broadcast-stream s1 s2 s3))))))) + (define-test issue.150 (:tag :issues) (let ((ext:*gc-verbose* nil) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/eb17d9369d6bfc901352f2… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/eb17d9369d6bfc901352f2… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][master] 2 commits: Fix #140: External format for streams that are not file-streams
by Raymond Toy (＠rtoy) 25 Nov '22

25 Nov '22

Raymond Toy pushed to branch master at cmucl / cmucl Commits: e7459829 by Raymond Toy at 2022-11-25T15:35:51+00:00 Fix #140: External format for streams that are not file-streams - - - - - 88843edc by Raymond Toy at 2022-11-25T15:35:52+00:00 Merge branch 'issue-140-stream-element-type-two-way-stream' into 'master' Fix #140: External format for streams that are not file-streams Closes #140 See merge request cmucl/cmucl!97 - - - - - 3 changed files: - src/code/stream.lisp - src/general-info/release-21e.md - tests/issues.lisp Changes: ===================================== src/code/stream.lisp ===================================== @@ -290,13 +290,21 @@ (stream-dispatch stream ;; simple-stream (stream::%stream-external-format stream) - ;; lisp-stream - (typecase stream + ;; lisp-stream. For unsupported streams, signal a type error. + (etypecase stream #+unicode (fd-stream (fd-stream-external-format stream)) - (synonym-stream (stream-external-format - (symbol-value (synonym-stream-symbol stream)))) - (t :default)) + (broadcast-stream + ;; See http://www.lispworks.com/documentation/HyperSpec/Body/t_broadc.htm + (let ((components (broadcast-stream-streams stream))) + (if (null components) + :default + (stream-external-format (car (last components)))))) + (synonym-stream + ;; Not defined by CLHS. What should happen if + ;; (synonym-stream-symbol stream) is unbound? + (stream-external-format + (symbol-value (synonym-stream-symbol stream))))) ;; fundamental-stream :default)) ===================================== src/general-info/release-21e.md ===================================== @@ -60,6 +60,7 @@ public domain. * ~~#134~~ Handle the case of `(expt complex complex-rational)` * ~~#136~~ `ensure-directories-exist` should return the given pathspec * #139 `*default-external-format*` defaults to `:utf-8` + * ~~#140~~ External format for streams that are not `file-stream`'s * ~~#141~~ Disallow locales that are pathnames to a localedef file * ~~#142~~ `(random 0)` signals incorrect error * ~~#147~~ `stream-line-column` method missing for `fundamental-character-output-stream` ===================================== tests/issues.lisp ===================================== @@ -745,6 +745,53 @@ (assert-equal (map 'list #'char-name string) (map 'list #'char-name (read-line s)))))) +;;; Test stream-external-format for various types of streams. + +(define-test issue.140.two-way-stream + (:tag :issues) + (with-open-file (in (merge-pathnames "issues.lisp" cmucl-test-runner::*load-path*) + :direction :input + :external-format :utf-8) + (with-open-file (out "/tmp/output.tst" + :direction :output + :external-format :utf-8 + :if-exists :supersede) + (let ((two-way-stream (make-two-way-stream in out))) + (assert-error 'type-error + (stream-external-format two-way-stream)))))) + +;; Test synonym-stream returns the format of the underlying stream. +(define-test issue.140.synonym-stream + (:tag :issues) + (with-open-file (s (merge-pathnames "issues.lisp" cmucl-test-runner::*load-path*) + :direction :input + :external-format :iso8859-1) + (let ((syn (make-synonym-stream '*syn-stream*))) + (setf syn s) + (assert-equal :iso8859-1 (stream-external-format syn))))) + +(define-test issue.140.broadcast-stream + (:tag :issues) + ;; Create 3 output streams. The exact external formats aren't + ;; really important here as long as they're different for each file + ;; so we can tell if we got the right answer. + (with-open-file (s1 "/tmp/broad-1" + :direction :output + :if-exists :supersede + :external-format :latin1) + (with-open-file (s2 "/tmp/broad-2" + :direction :output + :if-exists :supersede + :external-format :utf-8) + (with-open-file (s3 "/tmp/broad-3" + :direction :output + :if-exists :supersede + :external-format :utf-16) + ;; The format must be the value from the last stream. + (assert-equal :utf-16 + (stream-external-format + (make-broadcast-stream s1 s2 s3))))))) + (define-test issue.150 (:tag :issues) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/6764053dd9530292197b47… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/6764053dd9530292197b47… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-filename-encoding-utf8] 30 commits: Fix #132: Ansi test RENAME-FILE.1 fails
by Raymond Toy (＠rtoy) 23 Nov '22

23 Nov '22

Raymond Toy pushed to branch issue-139-filename-encoding-utf8 at cmucl / cmucl Commits: cdd7d328 by Raymond Toy at 2022-10-15T14:39:32+00:00 Fix #132: Ansi test RENAME-FILE.1 fails - - - - - e0e9f62d by Raymond Toy at 2022-10-15T14:39:35+00:00 Merge branch 'issue-132-ansi-test-rename-files' into 'master' Fix #132: Ansi test RENAME-FILE.1 fails Closes #132 See merge request cmucl/cmucl!90 - - - - - a05277c7 by Raymond Toy at 2022-10-15T20:53:20+00:00 Fix #134: Handle the case of (expt complex complex-rational) - - - - - 4dacd5ac by Raymond Toy at 2022-10-15T20:53:20+00:00 Merge branch 'issue-134-expt-bug' into 'master' Fix #134: Handle the case of (expt complex complex-rational) Closes #134 See merge request cmucl/cmucl!91 - - - - - 8719b21c by Raymond Toy at 2022-10-15T23:27:33+00:00 Fix #146: CI passes incorrectly - - - - - 9c0f63ff by Raymond Toy at 2022-10-15T23:27:34+00:00 Merge branch 'issue-146-ci-passes-incorrectly' into 'master' Fix #146: CI passes incorrectly Closes #146 See merge request cmucl/cmucl!100 - - - - - cde14045 by Raymond Toy at 2022-10-16T14:26:39+00:00 Fix #142: (random 0) signals incorrect error - - - - - 4c9cbf43 by Raymond Toy at 2022-10-16T14:26:41+00:00 Merge branch 'issue-142-random-0-wrong-error' into 'master' Fix #142: (random 0) signals incorrect error Closes #142 See merge request cmucl/cmucl!99 - - - - - b59185fc by Raymond Toy at 2022-10-16T14:27:39+00:00 Fix #136: ensure-directories-exist should return the given pathspec - - - - - 49ecc858 by Raymond Toy at 2022-10-16T14:27:39+00:00 Merge branch 'issue-136-ansi-test-ensure-directories-exist.8' into 'master' Fix #136: ensure-directories-exist should return the given pathspec Closes #136 See merge request cmucl/cmucl!92 - - - - - 08e5370a by Raymond Toy at 2022-10-16T07:33:23-07:00 Update release notes based on recent merges Forgot to update the release notes with recent merges that fixed a few issues. Hence update the notes now. Also testing see if we need to add a strikeout for closed issues, so didn't add strikeout for these. - - - - - 556b1a5b by Raymond Toy at 2022-10-16T07:35:57-07:00 Add strikeout for closed issues Nope, gitlab doesn't mark closed issues in anyway, unlike Trac that would automatically strikeout references to closed issues. We have to do it ourselves. - - - - - 95b4fc5c by Raymond Toy at 2022-10-16T13:05:09-07:00 Fix #146: CI passes incorrectly We forgot to update the script for macos to use separate `grep` commands like we did for linux. - - - - - 4a7207b6 by Raymond Toy at 2022-10-17T18:58:45+00:00 Fix #130: Implement file_author in C - - - - - ba5c5d2a by Raymond Toy at 2022-10-17T18:58:45+00:00 Merge branch 'issue-130-file-author-in-c' into 'master' Fix #130: Implement file_author in C Closes #130 See merge request cmucl/cmucl!88 - - - - - e8a0cc6c by Raymond Toy at 2022-10-30T15:03:27+00:00 Fix #147: Add method for stream-line-column - - - - - 0dad5a1a by Raymond Toy at 2022-10-30T15:03:28+00:00 Merge branch 'issue-147-stream-line-column-impl' into 'master' Fix #147: Add method for stream-line-column Closes #147 See merge request cmucl/cmucl!104 - - - - - 1300830b by Raymond Toy at 2022-10-31T17:12:48+00:00 Address #139: *default-external-format* is :utf-8 - - - - - 649a4f1e by Raymond Toy at 2022-10-31T17:12:49+00:00 Merge branch 'issue-139-default-external-format-utf8' into 'master' Address #139: *default-external-format* is :utf-8 See merge request cmucl/cmucl!103 - - - - - 88f6852f by Raymond Toy at 2022-11-01T12:04:55-07:00 Change :iso-8859-1 to :iso8859-1 in find-encoding While there's an alias for `:iso-8859-1`, it's safer to use `:iso8859-1` which is builtin. Using `:iso-8859-1` requires the alias database to be loaded, which isn't (currently) guaranteed when `find-encoding` is called. Thus use the builtin name instead. Besides, `:iso8859-1` is used in other places in "intl.lisp". (This is hard to test, but I noticed it when running ``` LANG=ko_KR.utf8 lisp ``` on the branch `issue-139-add-alias-local-external-format`.) - - - - - d5f1aa5e by Raymond Toy at 2022-11-01T20:35:49+00:00 Update release-21e.md with closed issues. - - - - - 402c0c01 by Raymond Toy at 2022-11-02T01:00:20+00:00 Fix #150: add aliases cp949 euckr - - - - - d825aa54 by Raymond Toy at 2022-11-02T01:00:20+00:00 Merge branch 'issue-150-add-aliases-cp949-euckr' into 'master' Fix #150: add aliases cp949 euckr Closes #150 See merge request cmucl/cmucl!106 - - - - - 33c760fa by Raymond Toy at 2022-11-03T04:47:09+00:00 Fix #149: Call setlocale(3C) on startup - - - - - 317a33f8 by Raymond Toy at 2022-11-03T04:47:10+00:00 Merge branch 'issue-149-add-setlocale' into 'master' Fix #149: Call setlocale(3C) on startup Closes #149 See merge request cmucl/cmucl!105 - - - - - 7bbb4843 by Raymond Toy at 2022-11-08T03:19:19+00:00 Fix #155: Wrap help strings neatly - - - - - 68f4ec70 by Raymond Toy at 2022-11-08T03:19:21+00:00 Merge branch 'issue-155-wrap-help-strings' into 'master' Fix #155: Wrap help strings neatly Closes #155 See merge request cmucl/cmucl!107 - - - - - 23f66902 by Raymond Toy at 2022-11-14T05:09:37+00:00 Fix #141: Use setlocale to handle localization settings - - - - - 6764053d by Raymond Toy at 2022-11-14T05:09:38+00:00 Merge branch 'issue-141-locale' into 'master' Fix #141: Use setlocale to handle localization settings Closes #141, #136, #142, #146, #134, and #132 See merge request cmucl/cmucl!101 - - - - - 8149dbd2 by Raymond Toy at 2022-11-23T12:35:55-08:00 Merge branch 'master' into issue-139-filename-encoding-utf8 - - - - - 0 changed files: Changes: View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/fba3f3a8be47647e528683… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/fba3f3a8be47647e528683… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-add-alias-local-external-format] 2 commits: Revert an unexpected space change that's not relevant
by Raymond Toy (＠rtoy) 22 Nov '22

22 Nov '22

Raymond Toy pushed to branch issue-139-add-alias-local-external-format at cmucl / cmucl Commits: ee9f2d02 by Raymond Toy at 2022-11-22T08:34:24-08:00 Revert an unexpected space change that's not relevant Not sure why tabs were converted to spaces, but that has been reverted so that the diff is smaller, as expected. - - - - - eb17d936 by Raymond Toy at 2022-11-22T13:44:34-08:00 Change os_get_locale_codeset to return the result of nl_langinfo Update unix::unix-get-locale-codeset appropriately - - - - - 4 changed files: - src/code/unix.lisp - src/i18n/locale/cmucl-unix.pot - src/lisp/os-common.c - tests/issues.lisp Changes: ===================================== src/code/unix.lisp ===================================== @@ -2918,10 +2918,7 @@ (defun unix-get-locale-codeset () _N"Get the codeset from the locale" - (with-alien ((codeset (array c-call:char 512))) - (alien-funcall + (cast (alien-funcall (extern-alien "os_get_locale_codeset" - (function void (* char) int)) - (cast codeset (* c-call:char)) - 512) - (cast codeset c-string))) + (function (* char)))) + c-string)) ===================================== src/i18n/locale/cmucl-unix.pot ===================================== @@ -1428,6 +1428,13 @@ msgstr "" msgid "Call setlocale(3c) with fixed args. Returns 0 on success." msgstr "" +#: src/code/unix.lisp +msgid "" +"Get LC_MESSAGES from the current locale. If we can't, return\n" +" NIL. A call to UNIX-SETLOCALE must have been done previously before\n" +" calling this so that the correct locale is returned." +msgstr "" + #: src/code/unix.lisp msgid "Get the codeset from the locale" msgstr "" ===================================== src/lisp/os-common.c ===================================== @@ -798,12 +798,8 @@ os_get_lc_messages(char *buf, int len) return locale ? 0 : -1; } -void -os_get_locale_codeset(char* codeset, int len) +char * +os_get_locale_codeset() { - char *code; - - code = nl_langinfo(CODESET); - - strncpy(codeset, code, len); + return nl_langinfo(CODESET); } ===================================== tests/issues.lisp ===================================== @@ -727,24 +727,24 @@ ;; using an explicit format of utf8 and verifying that we got the ;; right contents. (let ((string (concatenate 'string - ;; This is "hello" in Korean - '(#\Hangul_syllable_an - #\Hangul_Syllable_Nyeong - #\Hangul_Syllable_Ha - #\Hangul_Syllable_Se - #\Hangul_Syllable_Yo)))) + ;; This is "hello" in Korean + '(#\Hangul_syllable_an + #\Hangul_Syllable_Nyeong + #\Hangul_Syllable_Ha + #\Hangul_Syllable_Se + #\Hangul_Syllable_Yo)))) (with-open-file (s (merge-pathnames "out-utf8.txt" - *test-path*) - :direction :output - :if-exists :supersede) + *test-path*) + :direction :output + :if-exists :supersede) (write-line string s)) (with-open-file (s (merge-pathnames "out-utf8.txt" - *test-path*) - :direction :input - :external-format :utf-8) + *test-path*) + :direction :input + :external-format :utf-8) (assert-equal (map 'list #'char-name string) - (map 'list #'char-name (read-line s)))))) - + (map 'list #'char-name (read-line s)))))) + (define-test issue.139-locale-external-format (:tag :issues) ;; Just verify that :locale format exists View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/10f6311f91ae56ce58b57e… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/10f6311f91ae56ce58b57e… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0