April 2015 - cmucl-cvs - mailman3.common-lisp.net

[git] CMU Common Lisp branch master updated. snapshot-2013-08-8-gde69bad
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via de69bade53c2229f0ca1d2e9ddcc3e44135dc60f (commit) from ddfb0372fb591209c4aba8ea46f94e469e0686ca (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit de69bade53c2229f0ca1d2e9ddcc3e44135dc60f Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Sat Sep 7 13:18:09 2013 -0700 Update from logs. diff --git a/src/general-info/release-20e.txt b/src/general-info/release-20e.txt index 9d91bcc..8abf083 100644 --- a/src/general-info/release-20e.txt +++ b/src/general-info/release-20e.txt @@ -48,12 +48,17 @@ New in this release: * Added UNICODE package to hold Unicode-related functions. Currently contains Unicode extended versions of STRING-UPCASE, STRING-DOWNCASE, and STRING-CAPITALIZE. + * Optimizations in the x86 backend to speed up some operations and + reduce register restrictions. * ANSI compliance fixes: * Attempts to modify the standard readtable or the standard pprint dispatch table will now signal a continuable error. * Remove the Unicode extensions from string-upcase and friends. This functionality is moved to the new UNICODE package. + * DISASSEMBLE is conforming now; there are no options to + DISASSEMBLE. However, you can use DISASSEM:DISASSEMBLE if more + options are needed. * Bugfixes: * REPLACE and friends on strings were limited to strings less than @@ -74,6 +79,11 @@ New in this release: is represented internally as a UNION type, confusing the type derivation code. * Clean up and optimize x86 move VOPS and length/list VOP. + * DISASSEM:DISASSEMBLE takes new keyword args, base, case, radix + to specify values for *print-base*, *print-case*, and + *print-radix* for disassembly. The defaults are 16, :downcase, + and *print-radix*, respectively. This is a change from previous + versions which used a base of 10 and a case of :upcase. * Trac Tickets: ----------------------------------------------------------------------- Summary of changes: src/general-info/release-20e.txt | 10 ++++++++++ 1 file changed, 10 insertions(+) hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-08-9-g375ce21
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via 375ce218ab639fbc0b433942dbb7398a0bcd0177 (commit) from e585e8d6acbc146e978d1c2c5171987b6ba2fddf (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 375ce218ab639fbc0b433942dbb7398a0bcd0177 Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Thu Aug 14 21:06:20 2014 -0700 Fix ticket:106 by returning the correctly rounded value. * src/lisp/e_exp.c: * Add special case to return the correctly rounded value of exp(1). * tests/trac.lisp: * Add test that the correct value is returned. * Add test that exp(x) is still monotonic around x = 1. * src/general-info/release-20f.txt: * Update. diff --git a/src/general-info/release-20f.txt b/src/general-info/release-20f.txt index 265b0b5..92df6fa 100644 --- a/src/general-info/release-20f.txt +++ b/src/general-info/release-20f.txt @@ -97,6 +97,8 @@ New in this release: * In some situations KERNEL:DOUBLE-FLOAT-BITS on x86 would cause a segfault. This has been fixed. * For linux, motifd is no longer a 64-bit app. + * (exp 1d0) now returns the correctly rounded value of + e. Previously, it was off by one bit. * Trac Tickets: * Ticket #90 fixed. @@ -111,6 +113,7 @@ New in this release: * Ticket #84 fixed on x86. * Ticket #105 fixed. * Ticket #101 fixed. + * Ticket #106 fixed. * Other changes: diff --git a/src/lisp/e_exp.c b/src/lisp/e_exp.c index 8233bc3..8b7ff6d 100644 --- a/src/lisp/e_exp.c +++ b/src/lisp/e_exp.c @@ -111,6 +111,17 @@ P5 = 4.13813679705723846039e-08; /* 0x3E663769, 0x72BEA4D0 */ unsigned hx; union { int i[2]; double d; } ux; + /* + * CMUCL addition: Return correctly rounded value for + * exp(1). There are tests to verify that exp(x) is still + * monotonic around exp(1), so this change doens't break + * anything. + */ + if (x == 1) { + /* Return the correctly rounded value for x = 1 */ + return 2.718281828459045; + } + ux.d = x; hx = ux.i[HIWORD]; /* high word of x */ xsb = (hx>>31)&1; /* sign bit of x */ diff --git a/tests/trac.lisp b/tests/trac.lisp index 8253bc2..2bf420d 100644 --- a/tests/trac.lisp +++ b/tests/trac.lisp @@ -425,4 +425,14 @@ No dispatch function defined for #\\W.") (read-string-fn "#\wtf"))) - \ No newline at end of file +(define-test trac.106 + (:tag :trac) + ;; Verify the value is correct + (assert-equal 2.718281828459045d0 + (exp 1d0)) + ;; Verify that exp is still monotonic around 1 + (assert-true (<= (exp (1- double-float-negative-epsilon)) + (exp 1d0) + (exp (1+ double-float-epsilon))))) + + ----------------------------------------------------------------------- Summary of changes: src/general-info/release-20f.txt | 3 +++ src/lisp/e_exp.c | 11 +++++++++++ tests/trac.lisp | 12 +++++++++++- 3 files changed, 25 insertions(+), 1 deletion(-) hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-08-32-g54dd52c
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via 54dd52c6348b74051dd63be6abdd1952be7137af (commit) from 9b6ad20132565202c72dec2dd2d8c6ee287c1601 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 54dd52c6348b74051dd63be6abdd1952be7137af Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Mon Aug 25 19:52:38 2014 -0700 Update documentation to include float-signaling-nan-p. diff --git a/src/docs/cmu-user/extensions.tex b/src/docs/cmu-user/extensions.tex index e0177d8..839ce5d 100644 --- a/src/docs/cmu-user/extensions.tex +++ b/src/docs/cmu-user/extensions.tex @@ -62,12 +62,14 @@ values. \end{defun} \begin{defun}{extensions:}{float-nan-p}{\args{\var{x}}} + \defunx[extensions:]{float-signaling-nan-p}{\args{\var{x}}} \defunx[extensions:]{float-trapping-nan-p}{\args{\var{x}}} \code{float-nan-p} returns true if \var{x} is an IEEE NaN (Not A - Number) object. \code{float-trapping-nan-p} returns true only if + Number) object. \code{float-signaling-nan-p} returns true only if \var{x} is a trapping NaN. With either function, \var{x} must be a - float. + float. \code{float-trapping-nan-p} is the former name of + \code{float-signaling-nan-p} and is deprecated. \end{defun} \subsubsection{Negative Zero} ----------------------------------------------------------------------- Summary of changes: src/docs/cmu-user/extensions.tex | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-02-3-g1554c68
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via 1554c686c73c29228bf5b77425837fe27f3aa201 (commit) from 9be8ccbffdd602b08e0ba12aa59bc0d137dc569d (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 1554c686c73c29228bf5b77425837fe27f3aa201 Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Fri Feb 7 08:15:27 2014 -0800 Add news about lisp-unit. diff --git a/src/general-info/release-20f.txt b/src/general-info/release-20f.txt index 8df0287..628236b 100644 --- a/src/general-info/release-20f.txt +++ b/src/general-info/release-20f.txt @@ -45,6 +45,8 @@ New in this release: * Run-time checks for zero pages on newly allocated pages has been disabled now that the lazy zeroing of the heap pages has been the default for several months now. + * Add lisp-unit as a contrib. Use (require :lisp-unit) to load + it. Precompiled fasls are not included. * ANSI compliance fixes: * The values on the branch cuts for the inverse trig and ----------------------------------------------------------------------- Summary of changes: src/general-info/release-20f.txt | 2 ++ 1 file changed, 2 insertions(+) hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp annotated tag movable-linkage-space-alloc-fixed created. movable-linkage-space-alloc-fixed
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The annotated tag, movable-linkage-space-alloc-fixed has been created at 161af01f18804789172ed93e0f50a5f03575dc74 (tag) tagging fc13642018cdc018403d8f9e891b8c6d8b1b5012 (commit) replaces snapshot-2012-09 tagged by Raymond Toy on Wed Sep 12 23:10:08 2012 -0700 - Log ----------------------------------------------------------------- Changes to allocation complete so that we can do an indirect call. Raymond Toy (11): Finish splitting out x86-validate into separate OS versions. Fix ticket:62: Needed an IN-PACKAGE. Output lisp strings in utf8 format in ldb print. New MOVE-FROM-UNSIGNED, more like sparc and MOVE-FROM-SIGNED. Change the optional args for ALLOCATION and WITH-FIXED-ALLOCATION to ALLOCATION and WITH-ALLOCATION take a new temp arg for a register to Allow the temp arg to ALLOCATION to be NIL; replace all uses of the macros.lisp: Move common allocation stuff into SLOW-ALLOCATION-PATH. Always do indirect call. Remove the temp arg from ALLOCATION and WITH-FIXED-ALLOCATION. Remove ----------------------------------------------------------------------- hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-01-9-gc9a3b19
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via c9a3b19c19a5f2a5a09500c621694cc061cb6d5a (commit) from 189dc5b6a717ad0a93354e55d87bc0ab41e6745d (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit c9a3b19c19a5f2a5a09500c621694cc061cb6d5a Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Sat Jan 11 17:34:21 2014 -0800 Remove the zero checking of the heap. It's been several months without any issue after adding the lazy zeroing of heap pages, so it's probably safe to remove the zero checking of the heap pages. It can still be enabled by setting the appropriate variables. diff --git a/src/lisp/gencgc.c b/src/lisp/gencgc.c index a9c3e96..cdc95e0 100644 --- a/src/lisp/gencgc.c +++ b/src/lisp/gencgc.c @@ -367,7 +367,7 @@ boolean gencgc_zero_check_during_free_heap = FALSE; * enough. */ -#define DO_GENCGC_ZERO_CHECK (gencgc_zero_check || (gencgc_unmap_zero == MODE_LAZY) || (gencgc_unmap_zero == MODE_MADVISE)) +#define DO_GENCGC_ZERO_CHECK (gencgc_zero_check) /* * Only to the zero check during free_heap if both @@ -375,7 +375,7 @@ boolean gencgc_zero_check_during_free_heap = FALSE; * MODE_MAP or MODE_MEMSET because in all other modes, unallocated * pages are known not to contain zeroes. */ -#define DO_GENCGC_ZERO_CHECK_DURING_FREE_HEAP (gencgc_zero_check_during_free_heap && ((gencgc_unmap_zero == MODE_MAP) || (gencgc_unmap_zero == MODE_MEMSET))) +#define DO_GENCGC_ZERO_CHECK_DURING_FREE_HEAP (gencgc_zero_check_during_free_heap) /* * The minimum size for a large object. ----------------------------------------------------------------------- Summary of changes: src/lisp/gencgc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-06-93-gb424b78
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via b424b78d3e33bf8438a364920c5a2987aa0cacb8 (commit) from 75d14d2ed7dc9085858d9119013b67fbdfe77390 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit b424b78d3e33bf8438a364920c5a2987aa0cacb8 Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Mon Aug 4 18:59:06 2014 -0700 Remove -mtune=generic from Config.x86_common. Netbsd apparently doesn't like this, so we're moving that option to the linux and darwin config files. Reported by Robert Swindells. diff --git a/src/lisp/Config.x86_common b/src/lisp/Config.x86_common index ad2b1a0..1873f7d 100644 --- a/src/lisp/Config.x86_common +++ b/src/lisp/Config.x86_common @@ -54,7 +54,7 @@ CPP_INCLUDE_OPTIONS := -I. -I$(PATH1) -I- endif CPPFLAGS := $(CPP_DEFINE_OPTIONS) $(CPP_INCLUDE_OPTIONS) -CFLAGS += -Wstrict-prototypes -Wall -O2 -g -mtune=generic +CFLAGS += -Wstrict-prototypes -Wall -O2 -g ASFLAGS = -g ASSEM_SRC = x86-assem.S diff --git a/src/lisp/Config.x86_darwin b/src/lisp/Config.x86_darwin index 39c09bd..7de9ef2 100644 --- a/src/lisp/Config.x86_darwin +++ b/src/lisp/Config.x86_darwin @@ -6,7 +6,7 @@ include Config.x86_common MIN_VER = -mmacosx-version-min=10.5 CPPFLAGS += -DDARWIN $(MIN_VER) -m32 -CFLAGS += -g3 +CFLAGS += -g3 -mtune=generic ASFLAGS += -g3 $(MIN_VER) diff --git a/src/lisp/Config.x86_linux b/src/lisp/Config.x86_linux index cba5ec6..f43ac53 100644 --- a/src/lisp/Config.x86_linux +++ b/src/lisp/Config.x86_linux @@ -1,7 +1,8 @@ # -*- Mode: makefile -*- include Config.x86_common -CPPFLAGS += -m32 -rdynamic -D__NO_CTYPE -D_GNU_SOURCE -march=pentium4 -mfpmath=sse +CPPFLAGS += -D__NO_CTYPE -D_GNU_SOURCE +CFLAGS += -m32 -rdynamic -march=pentium4 -mfpmath=sse -mtune=generic UNDEFSYMPATTERN = -Xlinker -u -Xlinker & ASSEM_SRC += linux-stubs.S ----------------------------------------------------------------------- Summary of changes: src/lisp/Config.x86_common | 2 +- src/lisp/Config.x86_darwin | 2 +- src/lisp/Config.x86_linux | 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-) hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-08-34-g1345f3b
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via 1345f3b146c9840447fa7a943daf85f111011a7f (commit) from df548f6e69bc4fa0aec3825d10074135c9e4ac03 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 1345f3b146c9840447fa7a943daf85f111011a7f Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Tue Aug 26 20:29:56 2014 -0700 Use the two-prod algorithm from crlibm documentation. This handles overflows better and (c::two-prod 1.7976931214684583d308 (1+ (scale-float 1d0 -28))) doesn't signal an overflow like the old algorithm. diff --git a/src/compiler/float-tran.lisp b/src/compiler/float-tran.lisp index 7da6b95..ccb6e06 100644 --- a/src/compiler/float-tran.lisp +++ b/src/compiler/float-tran.lisp @@ -2289,22 +2289,9 @@ (kernel:%make-double-double-float hi lo)))) (declaim (maybe-inline split)) -;; This algorithm is the version given by Yozo Hida. It has problems -;; with overflow because we multiply by 1+2^27. -;; -;; But be very careful about replacing this with a new algorithm. The -;; values computed here are very important to get the rounding right. -;; If you change this, the rounding may be different, which will -;; affect other parts of the algorithm. -;; -;; I (rtoy) tried a different algorithm that split the number in two -;; as described, but without overflow. However, that caused -;; -9.4294948327242751340284975915175w0/1w14 to return a value that -;; wasn't really close to -9.4294948327242751340284975915175w-14. -;; -;; This also means we can't print numbers like 1w308 with the current -;; printing algorithm, or even divide 1w308 by 10. -#+nil +;; See Listing 2.6: Mul12 in "CR-LIBM: A library of correctly rounded +;; elementary functions in double-precision". Also known as Dekker's +;; algorithm. (defun split (a) "Split the double-float number a into a-hi and a-lo such that a = a-hi + a-lo and a-hi contains the upper 26 significant bits of a and @@ -2315,68 +2302,61 @@ (a-lo (- a a-hi))) (values a-hi a-lo))) -;; +split-limit+ is the largest number for which Yozo's algorithm -;; still works. Basically we want a*(1+2^27) <= -;; most-positive-double-float < 2^1024. Therefore, a < 2^1024/(1+2^27) -;; If we calculate that, we get a = 1.3393857490036326d300. A quick -;; test shows that this would cause overflow, but previous float would -;; not. This is the value we want. -(defconstant +split-limit+ - (scale-float (/ (float (1+ (expt 2 27)) 1d0)) 1024)) +;; Values used for scaling in two-prod. These are used to determine +;; if SPLIT might overflow so the value (and result) can be scaled to +;; prevent overflow. +(defconstant +two970+ + (scale-float 1d0 970)) -(defun split (a) - "Split the double-float number a into a-hi and a-lo such that a = - a-hi + a-lo and a-hi contains the upper 26 significant bits of a and - a-lo contains the lower 26 bits." - (declare (double-float a) - (optimize (speed 3) - (inhibit-warnings 3))) - ;; This splits the number a into 2 parts of 27 and 26 bits each, but - ;; the parts are, I think, supposed to be properly rounded in an - ;; IEEE fashion. - ;; - ;; For numbers that are very large, we use a different algorithm. - ;; For smaller numbers, we can use the original algorithm of Yozo - ;; Hida. - (if (>= (abs a) +split-limit+) - ;; I've tested this algorithm against Yozo's method for 1 - ;; billion randomly generated double-floats between 2^(-995) and - ;; 2^996, and identical results are obtained. For numbers that - ;; are very small, this algorithm produces different numbers - ;; because of underflow. For very large numbers, we, of course - ;; produce different results because Yozo's method causes - ;; overflow. - (let* ((tmp (* a (+ 1 (scale-float 1d0 -27)))) - (as (* a (scale-float 1d0 -27))) - (a-hi (* (- tmp (- tmp as)) (expt 2 27))) - (a-lo (- a a-hi))) - (values a-hi a-lo)) - ;; Yozo's algorithm. - (let* ((tmp (* a (+ 1 (expt 2 27)))) - (a-hi (- tmp (- tmp a))) - (a-lo (- a a-hi))) - (values a-hi a-lo)))) +(defconstant +two53+ + (scale-float 1d0 53)) +(defconstant +two-53+ + (scale-float 1d0 -53)) (declaim (inline two-prod)) + +;; This is essentially the algorithm given by Listing 2.7 Mul12Cond +;; given in "CR-LIBM: A library of correctly rounded elementary +;; functions in double-precision". #-ppc (defun two-prod (a b) _N"Compute fl(a*b) and err(a*b)" - (declare (double-float a b)) - (let ((p (* a b))) - (multiple-value-bind (a-hi a-lo) - (split a) - ;;(format t "a-hi, a-lo = ~S ~S~%" a-hi a-lo) - (multiple-value-bind (b-hi b-lo) - (split b) - ;;(format t "b-hi, b-lo = ~S ~S~%" b-hi b-lo) - (let ((e (+ (+ (- (* a-hi b-hi) p) - (* a-hi b-lo) - (* a-lo b-hi)) - (* a-lo b-lo)))) - (locally - (declare (optimize (inhibit-warnings 3))) - (values p e))))))) + (declare (double-float a b) + (optimize (speed 3))) + ;; If the numbers are too big, scale them done so SPLIT doesn't overflow. + (multiple-value-bind (aa bb) + (values (if (> a +two970+) + (* a +two-53+) + a) + (if (> b +two970+) + (* b +two-53+) + b)) + (let ((p (* aa bb))) + (declare (double-float p) + (inline split)) + (multiple-value-bind (aa-hi aa-lo) + (split aa) + ;;(format t "aa-hi, aa-lo = ~S ~S~%" aa-hi aa-lo) + (multiple-value-bind (bb-hi bb-lo) + (split bb) + ;;(format t "bb-hi, bb-lo = ~S ~S~%" bb-hi bb-lo) + (let ((e (+ (+ (- (* aa-hi bb-hi) p) + (* aa-hi bb-lo) + (* aa-lo bb-hi)) + (* aa-lo bb-lo)))) + (declare (double-float e)) + (locally + (declare (optimize (inhibit-warnings 3))) + ;; If the numbers was scaled down, we need to scale the + ;; result back up. + (when (> a +two970+) + (setf p (* p +two53+) + e (* e +two53+))) + (when (> b +two970+) + (setf p (* p +two53+) + e (* e +two53+))) + (values p e)))))))) #+ppc (defun two-prod (a b) @@ -2402,6 +2382,27 @@ (values q (+ (+ (- (* a-hi a-hi) q) (* 2 a-hi a-lo)) (* a-lo a-lo))))))) +(defun two-sqr (a) + _N"Compute fl(a*a) and err(a*b). This is a more efficient + implementation of two-prod" + (declare (double-float a)) + (let ((aa (if (> a +two970+) + (* a +two-53+) + a))) + (let ((q (* aa aa))) + (declare (double-float q) + (inline split)) + (multiple-value-bind (a-hi a-lo) + (split aa) + (locally + (declare (optimize (inhibit-warnings 3))) + (let ((e (+ (+ (- (* a-hi a-hi) q) + (* 2 a-hi a-lo)) + (* a-lo a-lo)))) + (if (> a +two970+) + (values (* q +two53+) + (* e +two53+)) + (values q e)))))))) #+ppc (defun two-sqr (a) ----------------------------------------------------------------------- Summary of changes: src/compiler/float-tran.lisp | 143 +++++++++++++++++++++--------------------- 1 file changed, 72 insertions(+), 71 deletions(-) hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-03-4-g8e693e5
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via 8e693e5bbc23fbfad188d2d6fd8b76be7fcef031 (commit) from 179e05df049f705f95eea224d1248ba5e979ce18 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 8e693e5bbc23fbfad188d2d6fd8b76be7fcef031 Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Mon Mar 10 20:21:09 2014 -0700 Add unicode tests, based on the tests in src/i18n/tests. diff --git a/tests/unicode.lisp b/tests/unicode.lisp new file mode 100644 index 0000000..f863e31 --- /dev/null +++ b/tests/unicode.lisp @@ -0,0 +1,194 @@ +(defpackage :unicode-tests + (:use :cl :lisp-unit)) + +(in-package "UNICODE-TESTS") + +(defvar *normalization-test* "target:i18n/tests/NormalizationTest.txt") + +(defun parse-norm-line (line) + (declare (string line)) + (cond ((or (char= (aref line 0) #\@) + (char= (aref line 0) #\#)) + ;; Delete lines starting with # or @ + nil) + (t + ;; Split the line into columns. There are 5 columns. Column 6 is a comment. + (let* ((p1 (position #\; line)) + (p2 (position #\; line :start (1+ p1))) + (p3 (position #\; line :start (1+ p2))) + (p4 (position #\; line :start (1+ p3))) + (p5 (position #\; line :start (1+ p4)))) + (labels + ((convert (codes) + (let ((result nil)) + (map nil #'(lambda (x) + (multiple-value-bind (hi lo) + (lisp::surrogates x) + (push hi result) + (when lo (push lo result)))) + codes) + (coerce (nreverse result) 'simple-string))) + (read-values (string start end) + (with-input-from-string (s string :start start :end end) + (let ((*read-base* 16)) + (convert + (loop for x = (read s nil nil) while x collect x)))))) + (values (read-values line 0 p1) + (read-values line (1+ p1) p2) + (read-values line (1+ p2) p3) + (read-values line (1+ p3) p4) + (read-values line (1+ p4) p5))))))) + +(define-test unicode.nfd + "Test Unicode NFD normalization" + (:tag :unicode) + (with-open-file (testfile *normalization-test* :direction :input :external-format :iso8859-1) + (loop for line = (read-line testfile nil nil) while line do + (multiple-value-bind (c1 c2 c3 c4 c5) + (parse-norm-line line) + (when c1 + (let ((test1 (lisp::string-to-nfd c1)) + (test2 (lisp::string-to-nfd c2)) + (test3 (lisp::string-to-nfd c3)) + (test4 (lisp::string-to-nfd c4)) + (test5 (lisp::string-to-nfd c5))) + (assert-equalp (list c3 c3 c3 c5 c5) + (list test1 test2 test3 test4 test5)) + )))))) + +(define-test unicode.nfkd + "Test Unicode NFKD normalization" + (:tag :unicode) + (with-open-file (testfile *normalization-test* :direction :input :external-format :utf-8) + (loop for line = (read-line testfile nil nil) while line do + ;;(format t "line = ~S~%" line) + (multiple-value-bind (c1 c2 c3 c4 c5) + (parse-norm-line line) + (when c1 + (let ((test1 (lisp::string-to-nfkd c1)) + (test2 (lisp::string-to-nfkd c2)) + (test3 (lisp::string-to-nfkd c3)) + (test4 (lisp::string-to-nfkd c4)) + (test5 (lisp::string-to-nfkd c5))) + (assert-equalp (list c5 c5 c5 c5 c5) + (list test1 test2 test3 test4 test5)))))))) + +(define-test unicode.nfc + "Test Unicode NFC normalization" + (:tag :unicode) + (with-open-file (testfile *normalization-test* :direction :input :external-format :utf-8) + (loop for line = (read-line testfile nil nil) while line do + (multiple-value-bind (c1 c2 c3 c4 c5) + (parse-norm-line line) + (when c1 + (let ((test1 (lisp::string-to-nfc c1)) + (test2 (lisp::string-to-nfc c2)) + (test3 (lisp::string-to-nfc c3)) + (test4 (lisp::string-to-nfc c4)) + (test5 (lisp::string-to-nfc c5))) + (assert-equalp (list c2 c2 c2 c4 c4) + (list test1 test2 test3 test4 test5)))))))) + +(define-test unicode-nfkc + "Test Unicode NFKC normalization" + (:tag :unicode) + (with-open-file (testfile *normalization-test* :direction :input :external-format :utf-8) + (loop for line = (read-line testfile nil nil) while line do + (multiple-value-bind (c1 c2 c3 c4 c5) + (parse-norm-line line) + (when c1 + (let ((test1 (lisp::string-to-nfkc c1)) + (test2 (lisp::string-to-nfkc c2)) + (test3 (lisp::string-to-nfkc c3)) + (test4 (lisp::string-to-nfkc c4)) + (test5 (lisp::string-to-nfkc c5))) + (assert-equalp (list c4 c4 c4 c4 c4) + (list test1 test2 test3 test4 test5)))))))) + +(defvar *word-break-test* "target:i18n/tests/WordBreakTest.txt") + +(defun parse-word-break-line (line) + (let* ((eos (or (position #\# line) + (length line)))) + ;; See WordBreakTest.txt for the format of the file. Basically + ;; each test is on one line and consists of a marker and a + ;; codepoint. The marker indicates whether a break is allowed (a + ;; division sign for a break, a multiplication sign for no break). + ;; The test ends with a marker to indicate if a division is + ;; allowed at the end of the test. After this marker is a comment + ;; for the test. + (when (plusp eos) + ;; For each non-empty line, read the codepints into a string and + ;; the markers into a second array. The second array contains + ;; indices into the string where a break is allowed. + (let ((*read-base* 16) + (count 0) + (breaks (make-array 10 :fill-pointer 0)) + (string (make-array 10 :fill-pointer 0 :element-type 'character))) + (with-input-from-string (s line :end eos) + (flet ((handle-break (c) + (when (eql #\division_sign (char (string c) 0)) + (vector-push-extend count breaks)))) + (handle-break (read s)) + (incf count) + (loop + (let ((c (read s nil nil))) + (unless c + (return)) + ;; Handle codepoints outside the BMP carefully. + (if (> c #xffff) + (let ((s (lisp::codepoints-string (list c)))) + ;; Need to increment the count because of our + ;; UTF-16 encoding of strings. + (incf count) + (vector-push-extend (aref s 0) string) + (vector-push-extend (aref s 1) string)) + (vector-push-extend (code-char c) string)) + (let ((c (read s))) + (handle-break c)) + (incf count))))) + (values string breaks))))) + +(defun do-test (string breaks) + (let ((posn -1) + (count 0) + (string (coerce string 'simple-string))) + (loop + (let ((end (unicode:string-next-word-break string posn))) + (when (= end posn) + (return)) + (unless (= end (aref breaks count)) + (format t "Test failed!~%") + (format t " String = ~S~%" (map 'list #'char-name string)) + (format t " Breaks = ~S~%" breaks) + (format t " Expected ~S but got ~S for break #~D~%" (aref breaks count) end count) + (finish-output) + (return-from do-test nil)) + (setf posn end) + (incf count))) + t)) + +(define-test unicode.word-break + "Test Unicode word break algorithm" + (:tag :unicode) + (with-open-file (s *word-break-test* :direction :input :external-format :utf) + (loop for line = (read-line s nil nil) + while line + do + (progn + (multiple-value-bind (s b) + (parse-word-break-line line) + (when s + (flet ((do-test (string) + (let ((posn -1) + (string (coerce string 'simple-string)) + (computed-breaks (make-array 10 :fill-pointer 0))) + (loop + (let ((end (unicode:string-next-word-break string posn))) + (when (= end posn) + (return)) + (vector-push-extend end computed-breaks) + (setf posn end))) + computed-breaks))) + (assert-equalp b + (do-test s))))))))) \ No newline at end of file ----------------------------------------------------------------------- Summary of changes: tests/unicode.lisp | 194 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 194 insertions(+) create mode 100644 tests/unicode.lisp hooks/post-receive -- CMU Common Lisp

1 0

[git] CMU Common Lisp branch master updated. snapshot-2014-06-96-g4b8c616
by rtoy＠common-lisp.net 08 Apr '15

08 Apr '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "CMU Common Lisp". The branch, master has been updated via 4b8c616db3f31214542a2793fa7365e464990f74 (commit) from c355485f5a73b92f3bf9a5fdc726bec9ffc15857 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 4b8c616db3f31214542a2793fa7365e464990f74 Author: Raymond Toy <toy.raymond(a)gmail.com> Date: Mon Aug 4 20:39:05 2014 -0700 Document that the argument/result timespec starts at 0, not 1. diff --git a/src/compiler/meta-vmdef.lisp b/src/compiler/meta-vmdef.lisp index 0b202de..bc679f0 100644 --- a/src/compiler/meta-vmdef.lisp +++ b/src/compiler/meta-vmdef.lisp @@ -1669,9 +1669,9 @@ :To Time-Spec Specify the beginning or end of the operand's lifetime. :From can only be used with results, and :To only with arguments. The default - for the N'th argument/result is (:ARGUMENT N)/(:RESULT N). These - options are necessary primarily when operands are read or written out - of order. + for the N'th argument/result is (:ARGUMENT N)/(:RESULT N). (The + first argument/result is number 0.) These options are necessary + primarily when operands are read or written out of order. :Conditional This is used in place of :RESULTS with conditional branch VOPs. There ----------------------------------------------------------------------- Summary of changes: src/compiler/meta-vmdef.lisp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) hooks/post-receive -- CMU Common Lisp

1 0