- cmucl-cvs - mailman3.common-lisp.net

[Git][cmucl/cmucl][issue-158-darwin-pathnames] Clean up implementation
by Raymond Toy (＠rtoy) 21 Dec '22

21 Dec '22

Raymond Toy pushed to branch issue-158-darwin-pathnames at cmucl / cmucl Commits: 08496138 by Raymond Toy at 2022-12-21T11:27:36-08:00 Clean up implementation Remove the function `normalize-name` since it's in an flet now. Make `*enable-darwin-path-normalization*` only defined on darwin, which is the only OS where's it's needed. - - - - - 1 changed file: - src/code/pathname.lisp Changes: ===================================== src/code/pathname.lisp ===================================== @@ -252,6 +252,7 @@ ;;; This constructor is used to make an instance of the correct type ;;; from parsed arguments. +#+darwin (defvar *enable-darwin-path-normalization* nil "When non-NIL, pathnames are on Darwin are normalized when created. Otherwise, the pathnames are unchanged. @@ -259,22 +260,6 @@ This must be NIL during bootstrapping because Unicode is not yet available.") -(defun normalize-name (piece) - ;; Normalize Darwin pathnames by converting Hangul - ;; syllables to conjoining jamo, and converting the - ;; string to NFD form, but skipping over a range of - ;; characters. - (typecase piece - (string - (if *enable-darwin-path-normalization* - (decompose (unicode::decompose-hangul piece) - :compatibility nil - :darwinp t) - piece)) - (t - ;; What should we do about lisp::pattern objects? - piece))) - (defun %make-pathname-object (host device directory name type version) (if (typep host 'logical-host) (flet ((upcasify (thing) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/08496138d12cbf3f2b839d4… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/08496138d12cbf3f2b839d4… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-158-darwin-pathnames] Fix typo normalizing directories.
by Raymond Toy (＠rtoy) 21 Dec '22

21 Dec '22

Raymond Toy pushed to branch issue-158-darwin-pathnames at cmucl / cmucl Commits: 361d18f9 by Raymond Toy at 2022-12-21T11:01:14-08:00 Fix typo normalizing directories. We had written `(cdr directory)` but we really need just `directory` so that we keep the `:absolute` or `:relative` part of the directory. - - - - - 1 changed file: - src/code/pathname.lisp Changes: ===================================== src/code/pathname.lisp ===================================== @@ -310,10 +310,11 @@ :darwinp t) piece)) (t - ;; What should we do about lisp::pattern objects? + ;; What should we do about lisp::pattern objects + ;; that occur in the name component? piece)))) (%make-pathname host device - (mapcar #'normalize-name (cdr directory)) + (mapcar #'normalize-name directory) (normalize-name name) (normalize-name type) version)))) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/361d18f9ed7f243bbc6ab88… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/361d18f9ed7f243bbc6ab88… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-158-darwin-pathnames] Oops. Forgot to rename `*enable-normalization*` in lispinit.lisp
by Raymond Toy (＠rtoy) 21 Dec '22

21 Dec '22

Raymond Toy pushed to branch issue-158-darwin-pathnames at cmucl / cmucl Commits: 150ac481 by Raymond Toy at 2022-12-21T10:32:53-08:00 Oops. Forgot to rename `*enable-normalization*` in lispinit.lisp - - - - - 1 changed file: - src/code/lispinit.lisp Changes: ===================================== src/code/lispinit.lisp ===================================== @@ -309,7 +309,7 @@ (declaim (special *lisp-initialization-functions* *load-time-values* - *enable-normalization*)) + *enable-darwin-path-normalization*)) (eval-when (compile) (defmacro print-and-call (name) @@ -345,7 +345,7 @@ (setf *type-system-initialized* nil) (setf *break-on-signals* nil) (setf unix::*filename-encoding* nil) - (setf *enable-normalization* nil) + (setf *enable-darwin-path-normalization* nil) #+gengc (setf conditions::*handler-clusters* nil) (setq intl::*default-domain* "cmucl") (setq intl::*locale* "C") View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/150ac481132cefe8c73173d… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/150ac481132cefe8c73173d… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-158-darwin-pathnames] 2 commits: Actually enable Darwin path normalization on Darwin.
by Raymond Toy (＠rtoy) 21 Dec '22

21 Dec '22

Raymond Toy pushed to branch issue-158-darwin-pathnames at cmucl / cmucl Commits: be398f1a by Raymond Toy at 2022-12-21T10:17:05-08:00 Actually enable Darwin path normalization on Darwin. Previously, we had this enabled on Linux for testing. Now switch over to Darwin. Rename `*enable-normalization*` to `*enable-darwin-path-normalization*` to make it clearer. Add a docstring too. In save.lisp, only set it on darwin since it's not relevant to any other system. Fix up some compiler notes about unused variables in `decompose-hangul-syllable`. - - - - - 856f9a9f by Raymond Toy at 2022-12-21T10:21:08-08:00 Oops. Forgot to rename `*enable-normalization*` in save.lisp - - - - - 4 changed files: - src/code/pathname.lisp - src/code/save.lisp - src/code/unicode.lisp - src/code/unidata.lisp Changes: ===================================== src/code/pathname.lisp ===================================== @@ -252,7 +252,12 @@ ;;; This constructor is used to make an instance of the correct type ;;; from parsed arguments. -(defvar *enable-normalization* nil) +(defvar *enable-darwin-path-normalization* nil + "When non-NIL, pathnames are on Darwin are normalized when created. + Otherwise, the pathnames are unchanged. + + This must be NIL during bootstrapping because Unicode is not yet + available.") (defun normalize-name (piece) ;; Normalize Darwin pathnames by converting Hangul @@ -261,7 +266,7 @@ ;; characters. (typecase piece (string - (if *enable-normalization* + (if *enable-darwin-path-normalization* (decompose (unicode::decompose-hangul piece) :compatibility nil :darwinp t) @@ -289,16 +294,29 @@ (upcasify name) (upcasify type) (upcasify version))) - #-(not nil) + #-darwin (%make-pathname host device directory name type version) - #+(not nil) - (%make-pathname host device - (when directory - (list* (car directory) - (mapcar #'normalize-name (cdr directory)))) - (normalize-name name) - (normalize-name type) - version)))) + #+darwin + (flet ((normalize-name (piece) + ;; Normalize Darwin pathnames by converting Hangul + ;; syllables to conjoining jamo, and converting the + ;; string to NFD form, but skipping over a range of + ;; characters. + (typecase piece + (string + (if *enable-darwin-path-normalization* + (decompose (unicode::decompose-hangul piece) + :compatibility nil + :darwinp t) + piece)) + (t + ;; What should we do about lisp::pattern objects? + piece)))) + (%make-pathname host device + (mapcar #'normalize-name (cdr directory)) + (normalize-name name) + (normalize-name type) + version)))) ;;; *LOGICAL-HOSTS* --internal. ;;; ===================================== src/code/save.lisp ===================================== @@ -284,16 +284,17 @@ (set-up-locale-external-format) ;; Set terminal encodings to :locale (set-system-external-format :locale) - ;; Get some unicode stuff needed for decomposing strings. - ;; This is needed on Darwin to normalize pathname - ;; objects, which needs this information. If we don't, - ;; we'll load the information at runtime when creating - ;; the path to "unidata.dat", which then calls decompose - ;; again, and so on. + #+darwin (progn + ;; Get some unicode stuff needed for decomposing strings. + ;; This is needed on Darwin to normalize pathname + ;; objects, which needs this information. If we don't, + ;; we'll load the information at runtime when creating + ;; the path to "unidata.dat", which then calls decompose + ;; again, and so on. (lisp::load-decomp) (lisp::load-combining) - (setf *enable-normalization* t)) + (setf *enable-darwin-path-normalization* t)) (ext::process-command-strings process-command-line) (setf *editor-lisp-p* nil) (macrolet ((find-switch (name) ===================================== src/code/unicode.lisp ===================================== @@ -527,18 +527,19 @@ (l-base #x1100) (v-base #x1161) (t-base #x11a7) - (s-count 11172) - (l-count 19) (v-count 21) (t-count 28) (n-count (* v-count t-count))) ;; Step 1: Compute index of the syllable S (let ((s-index (- cp s-base))) - ;; Step 2: If s is in the range 0 <= s <= s-count, the compute the components + ;; Step 2: If s is in the range 0 <= s <= s-count, the compute + ;; the components. (let ((l (+ l-base (truncate s-index n-count))) (v (+ v-base (truncate (mod s-index n-count) t-count))) (tt (+ t-base (mod s-index t-count)))) - ;; Step 3: If tt = t-base, then there is no trailing character so replace s by the sequence <l,v>. Otherwise there is a trailing character, so replace s by the sequence <l,v,tt> + ;; Step 3: If tt = t-base, then there is no trailing character + ;; so replace s by the sequence <l,v>. Otherwise there is a + ;; trailing character, so replace s by the sequence <l,v,tt>. (princ (code-char l) stream) (princ (code-char v) stream) (unless (= tt t-base) ===================================== src/code/unidata.lisp ===================================== @@ -513,12 +513,11 @@ (read-vector lvec stm :endian-swap :network-order) (values split hvec mvec lvec)))) (declare (ignorable #'read16 #'read32 #'read-ntrie)) - (let (#+nil(lisp::*enable-normalization* nil)) - (with-open-file (,stm *unidata-path* :direction :input - :element-type '(unsigned-byte 8)) - (unless (unidata-locate ,stm ,locn) - (error (intl:gettext "No data in file."))) - ,@body)))))) + (with-open-file (,stm *unidata-path* :direction :input + :element-type '(unsigned-byte 8)) + (unless (unidata-locate ,stm ,locn) + (error (intl:gettext "No data in file."))) + ,@body))))) (defloader load-range (stm 0) (let* ((n (read32 stm)) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/d7a1099dd6c0ed5540a46f… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/d7a1099dd6c0ed5540a46f… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-set-filename-encoding-to-utf8] Address review comments
by Raymond Toy (＠rtoy) 21 Dec '22

21 Dec '22

Raymond Toy pushed to branch issue-139-set-filename-encoding-to-utf8 at cmucl / cmucl Commits: 1ee83644 by Raymond Toy at 2022-12-21T09:57:10-08:00 Address review comments Remove `-nositeinit` and undo some indentation as requested. - - - - - 3 changed files: - bin/run-tests.sh - src/code/extfmts.lisp - src/code/save.lisp Changes: ===================================== bin/run-tests.sh ===================================== @@ -49,7 +49,7 @@ trap cleanup EXIT if [ $# -eq 0 ]; then # No args so run all the tests - $LISP -noinit -nositeinit -load tests/run-tests.lisp -eval '(cmucl-test-runner:run-all-tests)' + $LISP -noinit -load tests/run-tests.lisp -eval '(cmucl-test-runner:run-all-tests)' else # Run selected files. Convert each file name to uppercase and append "-TESTS" result="" @@ -58,6 +58,6 @@ else new=`echo $f | tr '[a-z]' '[A-Z]'` result="$result "\"$new-TESTS\" done - $LISP -noinit -nositeinit -load tests/run-tests.lisp -eval "(progn (cmucl-test-runner:load-test-files) (cmucl-test-runner:run-test $result))" + $LISP -noinit -load tests/run-tests.lisp -eval "(progn (cmucl-test-runner:load-test-files) (cmucl-test-runner:run-test $result))" fi ===================================== src/code/extfmts.lisp ===================================== @@ -371,7 +371,7 @@ (defun load-external-format-aliases () ;; Set filename encoding to NIL to bypass any encoding; it's not - ;; needed to open the aliases file. + ;; needed to open the aliases file. NIL means the pathname string is passed as is where only the low 8 bits of the (let ((*package* (find-package "KEYWORD")) (unix::*filename-encoding* nil)) (with-open-file (stm "ext-formats:aliases" :if-does-not-exist nil ===================================== src/code/save.lisp ===================================== @@ -194,18 +194,18 @@ (stream:string-decode *unidata-path* file-locale)))) (defun save-lisp (core-file-name &key - (purify t) - (root-structures ()) - (environment-name "Auxiliary") - (init-function #'%top-level) - (load-init-file t) - (site-init "library:site-init") - (print-herald t) - (process-command-line t) - #+:executable - (executable nil) - (batch-mode nil) - (quiet nil)) + (purify t) + (root-structures ()) + (environment-name "Auxiliary") + (init-function #'%top-level) + (load-init-file t) + (site-init "library:site-init") + (print-herald t) + (process-command-line t) + #+:executable + (executable nil) + (batch-mode nil) + (quiet nil)) "Saves a CMU Common Lisp core image in the file of the specified name. The following keywords are defined: @@ -390,7 +390,7 @@ (let ((initial-function (get-lisp-obj-address #'restart-lisp)) (core-name (unix-namestring core-file-name nil))) (without-gcing - #+:executable + #+:executable (if executable (save-executable core-name initial-function) (save core-name initial-function #+sse2 1 #-sse2 0)) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/1ee836448664a215b534c3e… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/1ee836448664a215b534c3e… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-158-darwin-pathnames] Fix bootstrap issues in pathname normalization for Darwin
by Raymond Toy (＠rtoy) 21 Dec '22

21 Dec '22

Raymond Toy pushed to branch issue-158-darwin-pathnames at cmucl / cmucl Commits: d7a1099d by Raymond Toy at 2022-12-21T09:35:37-08:00 Fix bootstrap issues in pathname normalization for Darwin We create a pathname in `filesys-init` during loading of the kernel.core. Unicode isn't ready at this time, so we can't do pathname normalization yet. Add `*enable-normalization*` to control this. Defaults to `NIL` and gets set to `T` in the initial-function. This also requires that we load up the Unicode decomp and combining tables before setting `*enable-normalization*` to `T`. For testing, this is enabled on Linux where I do my development. We'll have to reorder this when this is working. - - - - - 4 changed files: - src/code/lispinit.lisp - src/code/pathname.lisp - src/code/save.lisp - src/code/unidata.lisp Changes: ===================================== src/code/lispinit.lisp ===================================== @@ -308,7 +308,8 @@ ;;; in Unwind-Protects will get executed. (declaim (special *lisp-initialization-functions* - *load-time-values*)) + *load-time-values* + *enable-normalization*)) (eval-when (compile) (defmacro print-and-call (name) @@ -344,6 +345,7 @@ (setf *type-system-initialized* nil) (setf *break-on-signals* nil) (setf unix::*filename-encoding* nil) + (setf *enable-normalization* nil) #+gengc (setf conditions::*handler-clusters* nil) (setq intl::*default-domain* "cmucl") (setq intl::*locale* "C") ===================================== src/code/pathname.lisp ===================================== @@ -252,6 +252,24 @@ ;;; This constructor is used to make an instance of the correct type ;;; from parsed arguments. +(defvar *enable-normalization* nil) + +(defun normalize-name (piece) + ;; Normalize Darwin pathnames by converting Hangul + ;; syllables to conjoining jamo, and converting the + ;; string to NFD form, but skipping over a range of + ;; characters. + (typecase piece + (string + (if *enable-normalization* + (decompose (unicode::decompose-hangul piece) + :compatibility nil + :darwinp t) + piece)) + (t + ;; What should we do about lisp::pattern objects? + piece))) + (defun %make-pathname-object (host device directory name type version) (if (typep host 'logical-host) (flet ((upcasify (thing) @@ -271,24 +289,16 @@ (upcasify name) (upcasify type) (upcasify version))) - #-darwin + #-(not nil) (%make-pathname host device directory name type version) - #+darwin - (flet ((normalize-name (string) - ;; Normalize Darwin pathnames by converting Hangul - ;; syllables to conjoining jamo, and converting the - ;; string to NFD form, but skipping over a range of - ;; characters. - (decompose (with-output-to-string (s) - (unicode::decompose-hangul string s)) - :compatibility nil - :darwinp t))) - (%make-pathname host device - (list (car directory) - (mapcar #'normalize-name (cdr directory))) - (normalize-name name) - (normalize-name type) - version)))) + #+(not nil) + (%make-pathname host device + (when directory + (list* (car directory) + (mapcar #'normalize-name (cdr directory)))) + (normalize-name name) + (normalize-name type) + version)))) ;;; *LOGICAL-HOSTS* --internal. ;;; ===================================== src/code/save.lisp ===================================== @@ -284,6 +284,16 @@ (set-up-locale-external-format) ;; Set terminal encodings to :locale (set-system-external-format :locale) + ;; Get some unicode stuff needed for decomposing strings. + ;; This is needed on Darwin to normalize pathname + ;; objects, which needs this information. If we don't, + ;; we'll load the information at runtime when creating + ;; the path to "unidata.dat", which then calls decompose + ;; again, and so on. + (progn + (lisp::load-decomp) + (lisp::load-combining) + (setf *enable-normalization* t)) (ext::process-command-strings process-command-line) (setf *editor-lisp-p* nil) (macrolet ((find-switch (name) ===================================== src/code/unidata.lisp ===================================== @@ -513,11 +513,12 @@ (read-vector lvec stm :endian-swap :network-order) (values split hvec mvec lvec)))) (declare (ignorable #'read16 #'read32 #'read-ntrie)) - (with-open-file (,stm *unidata-path* :direction :input - :element-type '(unsigned-byte 8)) - (unless (unidata-locate ,stm ,locn) - (error (intl:gettext "No data in file."))) - ,@body))))) + (let (#+nil(lisp::*enable-normalization* nil)) + (with-open-file (,stm *unidata-path* :direction :input + :element-type '(unsigned-byte 8)) + (unless (unidata-locate ,stm ,locn) + (error (intl:gettext "No data in file."))) + ,@body)))))) (defloader load-range (stm 0) (let* ((n (read32 stm)) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/d7a1099dd6c0ed5540a46f1… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/d7a1099dd6c0ed5540a46f1… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl] Pushed new branch issue-158-darwin-pathnames
by Raymond Toy (＠rtoy) 20 Dec '22

20 Dec '22

Raymond Toy pushed new branch issue-158-darwin-pathnames at cmucl / cmucl -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/tree/issue-158-darwin-pathnames You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-set-filename-encoding-to-utf8] Load utf16-be format before running test issue.25c
by Raymond Toy (＠rtoy) 12 Dec '22

12 Dec '22

Raymond Toy pushed to branch issue-139-set-filename-encoding-to-utf8 at cmucl / cmucl Commits: 2c91358a by Raymond Toy at 2022-12-12T11:10:10-08:00 Load utf16-be format before running test issue.25c This is a work around until issue #161 is fixed. We had new test that loads the utf16-be format before running the test `issue.25c`. - - - - - 1 changed file: - tests/issues.lisp Changes: ===================================== tests/issues.lisp ===================================== @@ -258,6 +258,13 @@ (assert-equal (map 'list #'char-code out-string) (map 'list #'char-code expected)))))) +(define-test issue.25c-setup + (:tag :issues) + ;; Get the external format before running the test issue.25c. See + ;; issue #161 + ;; (https://gitlab.common-lisp.net/cmucl/cmucl/-/issues/161) + (assert-true (stream::find-external-format :utf16-be))) + (define-test issue.25c (:tag :issues) ;; Modified test to verify that each octet read from run-program is @@ -267,9 +274,6 @@ #\greek_small_letter_beta))) (expected (stream:string-encode in-string :utf16-be)) (path #p"issue25c.txt")) - ;; Get the external format before opening the file. See issue - ;; #161 (https://gitlab.common-lisp.net/cmucl/cmucl/-/issues/161) - (stream::find-external-format :utf16-be) (with-open-file (s path :direction :output :if-exists :supersede :external-format :utf16-be) (write-string in-string s) (force-output s) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/2c91358a7b87522b3cc8bef… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/2c91358a7b87522b3cc8bef… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-set-filename-encoding-to-utf8] Workaround for test issue.25c
by Raymond Toy (＠rtoy) 11 Dec '22

11 Dec '22

Raymond Toy pushed to branch issue-139-set-filename-encoding-to-utf8 at cmucl / cmucl Commits: d12d1153 by Raymond Toy at 2022-12-11T09:22:51-08:00 Workaround for test issue.25c Load the `:utf16-be` format before opening the file with an external format of `:utf16-be`. Added comment on the underlying issue. - - - - - 1 changed file: - tests/issues.lisp Changes: ===================================== tests/issues.lisp ===================================== @@ -267,6 +267,9 @@ #\greek_small_letter_beta))) (expected (stream:string-encode in-string :utf16-be)) (path #p"issue25c.txt")) + ;; Get the external format before opening the file. See issue + ;; #161 (https://gitlab.common-lisp.net/cmucl/cmucl/-/issues/161) + (stream::find-external-format :utf16-be) (with-open-file (s path :direction :output :if-exists :supersede :external-format :utf16-be) (write-string in-string s) (force-output s) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/d12d1153d92bc42eb02e06c… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/commit/d12d1153d92bc42eb02e06c… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0

[Git][cmucl/cmucl][issue-139-set-filename-encoding-to-utf8] 4 commits: Convert runtime strings to utf-16
by Raymond Toy (＠rtoy) 11 Dec '22

11 Dec '22

Raymond Toy pushed to branch issue-139-set-filename-encoding-to-utf8 at cmucl / cmucl Commits: 689db03d by Raymond Toy at 2022-12-10T11:45:53-08:00 Convert runtime strings to utf-16 The C runtime initializes several variables from either the environment or the command line. These are, of course, encoded in the locale, so we need to convert these strings into utf-16 format. Add `decode-runtime-strings` to do just that. This problem showed up when installing cmucl in /tmp/αβ and trying to run lisp. We get an error when printing the herald for the core file that is being used because we call truename on the path. Since the path is encoded, we can't find the file. We need to convert the path to a utf-16 string that we can use. - - - - - 6720015c by Raymond Toy at 2022-12-10T11:55:03-08:00 Remove lisp:: package specifer We're in the lisp package, so we don't need the prefix. - - - - - 234d86a7 by Raymond Toy at 2022-12-10T16:03:53-08:00 Handle decode-runtime-strings and environment more carefully. `decode-runtime-strings` needs to be more careful in converting the strings from the C runtime. For the command line parameters and the environment list, we need to use the locale when converting to Lisp strings. But `*cmucl-lib*`, `*cmucl-core-path*`, and `*unidata-path*` needs to use the file encoding to convert the C result to Lisp. We also need to call `environment-init` after calling `decode-runtime-strings` because the strings could have changed. This important for the "library:" search-list which contains file names. Without this, the search-list is mangled so we can't find anything. Finally, call `intl::setlocale` after the `environment-init` has been called again because `intl::setlocale` needs the paths to the pot files so the "library:" search-list has to be valid pathnames. Previously, translations could be accessed early and error out because the paths were not correct. This was testing by installing in "/tmp/αβ" and running lisp. Before these changes, we errored out printing out the path to the core file because the path to the core file was not properly decoded. Now it works. Then we tried `(set-system-external-format :euc-kr)`. We can find the format implementation file correctly. We also tested with ``` LANG=ko_KR.EUC_KR bin/lisp -noinit ``` This correctly loads up the euc-kr file and sets the external format. There are no errors. (But of course the printed path for the core file is wrong because euc-kr can't handle the greek letters. - - - - - f43c6517 by Raymond Toy at 2022-12-10T16:26:21-08:00 Add some comments - - - - - 2 changed files: - src/code/extfmts.lisp - src/code/save.lisp Changes: ===================================== src/code/extfmts.lisp ===================================== @@ -493,7 +493,7 @@ ;; encoding to NIL because we don't need any special ;; encoding to open the format files. (let* ((*print-readably* nil) - (unix::*filename-encoding* nil) + ;;(unix::*filename-encoding* nil) (*package* (find-package "STREAM")) (lisp::*enable-package-locked-errors* nil) (s (open (format nil "ext-formats:~(~A~).lisp" name) ===================================== src/code/save.lisp ===================================== @@ -164,20 +164,48 @@ *default-external-format*)))) (values)) - +(defun decode-runtime-strings (locale file-locale) + ;; The C runtime can initialize the following strings from the + ;; command line or the environment. We need to decode these into + ;; the utf-16 strings that Lisp uses. + (setf lisp-command-line-list + (mapcar #'(lambda (s) + (stream:string-decode s locale)) + lisp-command-line-list)) + (setf lisp-environment-list + (mapcar #'(lambda (s) + (stream:string-decode s locale)) + lisp-environment-list)) + ;; This needs more work.. *cmucl-lib* could be set from the the envvar + ;; "CMUCLLIB" or from the "-lib" command-line option, and thus + ;; should use the LOCALE to decode the string. + (when *cmucl-lib* + (setf *cmucl-lib* + (stream:string-decode *cmucl-lib* file-locale))) + ;; This also needs more work since the core path could come from the + ;; "-core" command-line option and should thus use LOCALE to decode + ;; the string. It could also come from the "CMUCLCORE" envvar. + (setf *cmucl-core-path* + (stream:string-decode *cmucl-core-path* file-locale)) + ;; *unidata-path* defaults to a pathname object, but the user can + ;; specify a path, so we need to decode the string path if given. + (when (and *unidata-path* (stringp *unidata-path*)) + (setf *unidata-path* + (stream:string-decode *unidata-path* file-locale)))) + (defun save-lisp (core-file-name &key - (purify t) - (root-structures ()) - (environment-name "Auxiliary") - (init-function #'%top-level) - (load-init-file t) - (site-init "library:site-init") - (print-herald t) - (process-command-line t) - #+:executable - (executable nil) - (batch-mode nil) - (quiet nil)) + (purify t) + (root-structures ()) + (environment-name "Auxiliary") + (init-function #'%top-level) + (load-init-file t) + (site-init "library:site-init") + (print-herald t) + (process-command-line t) + #+:executable + (executable nil) + (batch-mode nil) + (quiet nil)) "Saves a CMU Common Lisp core image in the file of the specified name. The following keywords are defined: @@ -278,13 +306,18 @@ ;; Load external format aliases now so we can aliases to ;; specify the external format. (stream::load-external-format-aliases) - ;; Set the locale for lisp - (intl::setlocale) ;; Set up :locale format (set-up-locale-external-format) ;; Set terminal encodings to :locale and filename encoding to :utf-8. ;; (This needs more work on Darwin.) (set-system-external-format :locale :utf-8) + (decode-runtime-strings :locale :utf-8) + ;; Need to reinitialize the environment again because + ;; we've possibly changed the environment variables and + ;; pathnames. + (environment-init) + ;; Set the locale for lisp + (intl::setlocale) (ext::process-command-strings process-command-line) (setf *editor-lisp-p* nil) (macrolet ((find-switch (name) @@ -340,14 +373,14 @@ (unix:unix-exit (catch '%end-of-the-world (unwind-protect - (if *batch-mode* - (handler-case - (%restart-lisp) - (error (cond) - (format *error-output* (intl:gettext "Error in batch processing:~%~A~%") - cond) - (throw '%end-of-the-world 1))) - (%restart-lisp)) + (if *batch-mode* + (handler-case + (%restart-lisp) + (error (cond) + (format *error-output* (intl:gettext "Error in batch processing:~%~A~%") + cond) + (throw '%end-of-the-world 1))) + (%restart-lisp)) (finish-standard-output-streams)))))) ;; Record dump time and host @@ -357,7 +390,7 @@ (let ((initial-function (get-lisp-obj-address #'restart-lisp)) (core-name (unix-namestring core-file-name nil))) (without-gcing - #+:executable + #+:executable (if executable (save-executable core-name initial-function) (save core-name initial-function #+sse2 1 #-sse2 0)) View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/354f94f5be60e66e139a09… -- View it on GitLab: https://gitlab.common-lisp.net/cmucl/cmucl/-/compare/354f94f5be60e66e139a09… You're receiving this email because of your account on gitlab.common-lisp.net.

1 0