April 2015 - armedbear-devel - mailman3.common-lisp.net

[armedbear-devel] Re: cl-test-grid: Testing ABCL-1.2.0-RC
by Mark Evenson 08 Apr '15

08 Apr '15

Agreed: fixing STD-FIND-METHOD-COMBINATION seems to be necessary. @anton: thanks for the tests @rudi: dunno if this is you, but could you take a look to see if this is easy? If you fix on trunk, I will take care of the release. At this point, I would release abcl-1.2.1 and announce, as abcl-1.2.0 has already been "softly launched" without announcement. -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad idea? A: Top-posting. Q: What is the most annoying thing in email? On Jun 9, 2013, at 1:15, Anton Vodonosov <avodonosov(a)yandex.ru> wrote: > 08.06.2013, 02:35, "Anton Vodonosov" <avodonosov(a)yandex.ru>: >> 08.06.2013, 02:13, "Erik Huelsmann" <ehuels(a)gmail.com>: >> >>> Hi Anton, >>> Could you test ABCL-1.2.0-RC with the latest Quicklisp snapshot and publish reports? I'd love to see how we're doing in the CL ecosystem before we release! >>> >>> Thanks! >>> >>> Bye, >>> >>> Erik. >> >> Hello, Erik. >> >> Just today I started the tests, on the 1.2.0 source tarball from abcl.org and on the current svn head. >> They are running on your VM. The results will be a day or two later. >> >> Best regards, >> - Anton > > Hello again. > > The results for ABCL 1.2.0 RC (the soruce tarball form abcl.org) are ready. > Comparision with ABCL 1.1.1: > http://common-lisp.net/project/cl-test-grid/abcl/abcl-diff10.html > > As you see there are some regressions - results that were green and have become red. > Most of the seem to be caused by the same error: Wrong number of arguments for #<STD-FIND-METHOD-COMBINATION {17D2F0E}>. > > Best regards, > - Anton

5 7

[armedbear-devel] Fix MAKE-FUNCTION-POINTER for abcl-1.2.1 with jna-4.0.0.
by Mark Evenson 08 Apr '15

08 Apr '15

A [fix for CFFI with function pointers when using jna-4.0.0][1] and [the corresponding naked diff][2]. Can’t quite seem to figure out how to make a “pull request” out of this. *sigh* [1]: https://github.com/easye/cffi/commit/0f63798ab20e274d3062964a3286e34855747d… [2]: http://slack.net/~evenson/abcl/cffi/abcl-cffi-encoding-20131124a.diff -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

1 0

[armedbear-devel] FIND-SYSTEM-JAR ignores versioned jars
by Olof-Joachim Frahm 08 Apr '15

08 Apr '15

Hi, I've noticed (while building with Maven), that the versioned system and contrib jars aren't found properly because of how the loader constructs the pathnames. Attached are three patches to clean up abcl-contrib.lisp a bit and then fix this by accepting every matching pathname directly. This still doesn't check for the correct version string, but maybe that's not a problem. Best regards, Olof

2 3

[armedbear-devel] COMPILE-FILE fails with stack overflow when INLINE meets recursion
by Xiaofeng Yang 08 Apr '15

08 Apr '15

Hi, all. It seems that cl-prime-maker compiles failed with ABCL because of the following bug. This bug can also be found in ABCL 1.1.1 (I don't know whether this bug can be found in 1.1.0 or earlier), not only 1.2.x and 1.3.x. $ cat bug-for-abcl.lisp (in-package :cl-user) (declaim (inline foo)) (defun foo () (foo)) (defun bar () (foo)) $ ~/abcl/abcl Armed Bear Common Lisp 1.3.0 Java 1.8.0 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.364 seconds. Startup completed in 1.739 seconds. Type ":help" for a list of available commands. CL-USER(1): (compile-file "bug-for-abcl.lisp") ; Compiling /tmp/bug-for-abcl.lisp ... ; (IN-PACKAGE :CL-USER) ; (DECLAIM (INLINE FOO)) ; (DEFUN FOO ...) ; (DEFUN BAR ...) #<THREAD "interpreter" {776DDE2A}>: Debugger invoked on condition of type STORAGE-CONDITION Stack overflow. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2): After the stack overflow, if I returned to the top level, remove the line "(declaim (inline foo))" and COMPILE-FILE again, it also failed. Unless I restart ABCL and perform COMPILE-FILE with the file without the line "(declaim (inline foo))", it can be compiled now. Best regards, Xiaofeng Yang

2 1

[armedbear-devel] :back not supported?
by Faré 08 Apr '15

08 Apr '15

See https://bugs.launchpad.net/asdf/+bug/1260011 http://www.lispworks.com/documentation/HyperSpec/Body/19_bbdc.htm [2] CL-USER(5): (make-pathname :directory '(:relative :back)) #<THREAD "interpreter" {2B3B7C28}>: Debugger invoked on condition of type FILE-ERROR Unsupported directory component BACK. Restarts: 0: ABORT Return to debug level 2. 1: ABORT Return to debug level 1. 2: TOP-LEVEL Return to top level. [3] CL-USER(6): (lisp-implementation-version) "1.2.1" "OpenJDK_64-Bit_Server_VM-Sun_Microsystems_Inc.-1.6.0_27-b27" "amd64-Linux-3.8.0-34-generic" [3] CL-USER(7): I suppose I can try to paper over it in ASDF. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Anarchism is founded on the observation that since few men are wise enough to rule themselves, even fewer are wise enough to rule others. — Edward Abbey

2 1

[armedbear-devel] latest cl-test-grid testing
by Anton Vodonosov 08 Apr '15

08 Apr '15

Hello. I have recently preformed cl-test-grid tests runs for several ABCL versions, and reviewed the results now. Two things I want to report. 1. Some degradation of error handling is introduced somewhere between abcl-1.2.0-fasl42 and abcl-1.3.0-dev-svn-14580-fasl42. It may be noticed when loading ASDF systems :cells and :test-gtk (the latter is from the cells-gtk3 project). ASDF:load-op fails for these systems with stack overflow. Old ABCL allows me to catch the stack overflow error and record asdf:load failure, while new ABCL sometimes crashes - the java process terminates: http://common-lisp.net/project/cl-test-grid/abcl/abcl-diff16.html To reproduce the situations on the cl-test-grid.cloud.efficito.com machine, lets ql:quickload the :cells system and save the ABCL process output: Old ABCL: java -jar ~/lisps/abcl-1.2.0/dist/abcl.jar --noinit --nosystem --batch --load ~/cl-test-grid/work-dir/agent/quicklisp/setup.lisp --eval '(handler-case (ql:quickload :cells) (serious-condition (e) (format t "!!! Serious condition occurred: ~A !!!~%" (type-of e))))' > log 2>&1 && less log Old ABCL always behave the same: prints the stacktrace, and then prints the message from our handler-case - "!!! Serious condition occurred .." New ABCL: java -jar ~/lisps/abcl/dist/abcl.jar --noinit --nosystem --batch --load ~/cl-test-grid/work-dir/agent/quicklisp/setup.lisp --eval '(handler-case (ql:quickload :cells) (serious-condition (e) (format t "!!! Serious condition occured: ~A !!!~%" (type-of e))))' > log 2>&1 && less log New ABCL sometimes only prints the "!!! Serious condition occurred ..." message from our handler-case, without the stack trace. But sometimes it doesn't invoke our handler-case at all, and instead prints a stack trace as shown below, and then java process exits: Exception in thread "interpreter" java.lang.ClassCastException: org.armedbear.lisp.Symbol cannot be cast to org.armedbear.lis p.LispThread$StackMarker at org.armedbear.lisp.LispThread.getStackTop(LispThread.java:721) at org.armedbear.lisp.LispThread.pushStackFrame(LispThread.java:736) at org.armedbear.lisp.Lisp.pushJavaStackFrames(Lisp.java:374) at org.armedbear.lisp.Lisp.stackError(Lisp.java:388) at org.armedbear.lisp.asdf_134.execute(asdf.lisp:1436) at org.armedbear.lisp.Symbol.execute(Symbol.java:814) at org.armedbear.lisp.LispThread.execute(LispThread.java:832) at org.armedbear.lisp.asdf_402.execute(asdf.lisp:4458) at org.armedbear.lisp.Symbol.execute(Symbol.java:803) at org.armedbear.lisp.LispThread.execute(LispThread.java:814) at org.armedbear.lisp.asdf_675.execute(asdf.lisp:6127) The full stacktrace is in the file attached. As you can see, the problem is caused by ClassCastException when handling the first (stack overflow) error. Why the ClassCastException doesn't happen always - I don't know. Maybe it depends on the internal state of the ABCL process. 2. Erik asked whether cl-test-grid testing becomes faster after the patch for "Lazy allocation of LispStackFrames". There is no noticeable speedup, full test run on both new and old ABCL takes around 18 hours. For your information: in cl-test-grid every ASDF system and every testsuite is peformed in a separate process, in total around 1500 processes are started. It might be the case that in some other use case, lisp code execution has speedup, but in cl-test-grid it is not noticeable. Best regards, - Anton

1 0

[armedbear-devel] "Hardcoding" methods that funcall a symbol
by Theam Yong Chew 08 Apr '15

08 Apr '15

Hi all, This is old history, but I remember seeing some really old email threads about hardcoding of Lisp function (as symbols) calls into Java methods. Attached please find a patch for trying this out. I refactored the original method body generation code slightly, then added java::runtime-class-add-symbol-method-body which involves low level bytecode generation, so I'd appreciate additional checks/reviews (I believe the repetitive load/store sequences won't have any effects with the JIT & optimisers, but may be fractionally easier to decompile...). I also updated java:jnew-runtime-class's docstring, but I'm not sure if the wording "function designator" is accurate enough, since (funcall '(lambda () 1)) => 1 Would '(lambda () 1) be considered a FUNCTION DESIGNATOR? In any case, java::runtime-class-add-methods already works with it just the same. This code has not been rigorously tested as it doesn't work in abcl-1.2.1 (tested/works for older versions) due to a regression, since fixed in r14570 (Restore Packages.findPackage() API...Fixes #324). See appendix, where I include a start for some tests. And in case it isn't obvious, the hardcoding of symbol funcalls (test3) into a class method is the only new functionality here. Yong (defun test3 (this) "test3") (defun test4 (this x y z) (+ x y z)) (defparameter *test-class* (java:jnew-runtime-class "runtime_class" :methods `(("test1" "java.lang.String" () ,(lambda (this) "test1")) ("test2" "java.lang.String" () (lambda (this) "test2")) ("test3" "java.lang.String" () test3) ("test4" "java.lang.Object" ;; (:int :int :int) ;; unsupported for now ("java.lang.Object" "java.lang.Object" "java.lang.Object") ,(function test4)) ("test5" "java.lang.String" ("java.lang.String") (lambda (this x) (declare (ignore x)) "test5")) ("undefined" :void () undefined)))) (let ((obj (java:jnew *test-class*))) (list (java:jcall "test1" obj) (java:jcall "test2" obj) (java:jcall "test3" obj) (java:jcall "test4" obj 1 2 3) (java:jcall "test5" obj "foo"))) ==> ("test1" "test2" "test3" 6 "test5") ;; This should thrown an error... (java:jcall "undefined" (java:jnew *test-class*)) ;; As should this, but java.lang.NoSuchMethodException: No applicable method is not very obvious... (java:jcall "test5" (java:jnew *test-class*)) ;; This shouldn't, but does, see stack trace below, (quite relatedly, at least for me using Slime, (java:jcall "undefined" (java:jnew *test-class*)) above actually seemed to throw 2 errors) (ignore-errors (java:jcall "undefined" (java:jnew *test-class*))) ==> ;;error Java exception 'org.armedbear.lisp.Go'. [Condition of type JAVA:JAVA-EXCEPTION] Restarts: 0: [RETRY] Retry SLIME interactive evaluation request. 1: [*ABORT] Return to SLIME's top level. 2: [ABORT] Abort thread. Backtrace: 0: (#<FUNCTION {106FAF1E}> #<JAVA-EXCEPTION {658A7B6C}> #<FUNCTION {106FAF1E}>) Locals: "??" = #<FUNCTION {106FAF1E}> "??"#1 = #<JAVA:JAVA-EXCEPTION org.armedbear.lisp.Go {658A7B6C}> "??"#2 = #<FUNCTION {106FAF1E}> 1: (APPLY #<FUNCTION {106FAF1E}> (#<JAVA-EXCEPTION {658A7B6C}> #<FUNCTION {106FAF1E}>)) Locals: "??" = APPLY "??"#1 = #<FUNCTION {106FAF1E}> "??"#2 = (#<JAVA:JAVA-EXCEPTION org.armedbear.lisp.Go {658A7B6C}> ..) 2: (SYSTEM::RUN-HOOK SYSTEM::*INVOKE-DEBUGGER-HOOK* #<JAVA-EXCEPTION {658A7B6C}> #<FUNCTION {106FAF1E}>) Locals: "??" = SYSTEM::RUN-HOOK "??"#1 = SYSTEM::*INVOKE-DEBUGGER-HOOK* "??"#2 = #<JAVA:JAVA-EXCEPTION org.armedbear.lisp.Go {658A7B6C}> "??"#3 = #<FUNCTION {106FAF1E}> 3: (INVOKE-DEBUGGER #<JAVA-EXCEPTION {658A7B6C}>) Locals: "??" = INVOKE-DEBUGGER "??"#1 = #<JAVA:JAVA-EXCEPTION org.armedbear.lisp.Go {658A7B6C}> 4: org.armedbear.lisp.Lisp.error(Lisp.java:382) Locals: "??" = "??" 5: org.armedbear.lisp.Java.jcall(Java.java:909) Locals: "??" = "??" 6: org.armedbear.lisp.Java$pf_jcall.execute(Java.java:756) Locals: "??" = :CLASS "??"#1 = "org.armedbear.lisp.Java$pf_jcall" "??"#2 = :METHOD "??"#3 = "execute" "??"#4 = :FILE "??"#5 = "Java.java" "??"#6 = :LINE "??"#7 = 756 7: org.armedbear.lisp.Primitive.execute(Primitive.java:123) Locals: "??" = :CLASS "??"#1 = "org.armedbear.lisp.Primitive" "??"#2 = :METHOD "??"#3 = "execute" "??"#4 = :FILE "??"#5 = "Primitive.java" "??"#6 = :LINE "??"#7 = 123 8: (JAVA:JCALL "undefined" #<runtime_class runtime_class@1451e300 {C0E1F6C}>) Locals: "??" = JAVA:JCALL "??"#1 = "undefined" "??"#2 = #<runtime_class runtime_class@1451e300 {C0E1F6C}> 9: (#<FUNCTION (LAMBDA ()) {7330F1A2}>) Locals: "??" = #<FUNCTION (LAMBDA ()) {7330F1A2}> 10: (JAVA:JRUN-EXCEPTION-PROTECTED #<FUNCTION (LAMBDA ()) {7330F1A2}>) 11: (SYSTEM::%EVAL (IGNORE-ERRORS (JAVA:JCALL "undefined" (JAVA:JNEW *TEST-CLASS*)))) 12: (EVAL (IGNORE-ERRORS (JAVA:JCALL "undefined" (JAVA:JNEW *TEST-CLASS*)))) 13: (#<FUNCTION {1247FDC3}>)

2 3

[armedbear-devel] Backquote bug
by Faré 08 Apr '15

08 Apr '15

Subtle bug in ABCL. What should this form read as? `#5(1 ,@`(2 3)) ECL, LispWorks and fare-quasiquote agree on #(1 2 3 2 3) allegro, ccl, clisp, sbcl return the arguably conformant #(1 2 3 2 3 2 3 2 3) abcl, cmucl, gcl, xcl all return the arguably completely buggy #(1 2 3) —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org It may be bad manners to talk with your mouth full, but it isn't too good either if you speak when your head is empty.

4 6

[armedbear-devel] Gray streams bugs
by Theam Yong Chew 08 Apr '15

08 Apr '15

Hi all, I inadvertently discovered some bugs in the gray-streams implementation due to a stale flexi-streams that did not get upgraded properly. There were quite a few things that needed tidying - the type checking using EQ is wrong. Also, after trying to hunt around CLHS, I think BYTE is not a CL type (though I may be mistaken). This could be a reflection of some Corman Lisp specific feature in the past (Yes! These bugs seem to have been around ABCL for that long!) Some test examples (avoiding flexi-streams or other dependencies) ;; ----- setup (require 'gray-streams) (defclass test-gray-binary-input-stream (gray-streams:fundamental-binary-input-stream) ()) (defclass test-gray-character-input-stream (gray-streams:fundamental-character-input-stream) ()) (let ((bytes (list 65 66 67 68 69)) (pos -1)) (defmethod gray-streams::stream-read-byte ((stream test-gray-binary-input-stream)) (elt bytes (mod (incf pos) 5)))) (let ((chars (list #\A #\B #\C #\D #\E)) (pos -1)) (defmethod gray-streams::stream-read-char ((stream test-gray-character-input-stream)) (elt chars (mod (incf pos) 5)))) ;; ----- (let ((s (make-instance 'test-gray-binary-input-stream))) (loop repeat 10 collect (read-byte s))) => (65 66 67 68 69 65 66 67 68 69) (let ((arr (make-array 10 :initial-element nil))) (list (read-sequence arr (make-instance 'test-gray-binary-input-stream) :start 2 :end 7) arr)) Expected (7 #(NIL NIL 65 66 67 68 69 NIL NIL NIL)), but got an error instead, ... no applicable method... I guess implementors of gray streams would normally define both read-byte & read-sequence. But ABCL's default/fallback method does try to support binary streams, it just isn't being dispatched on the right class. Also, read-sequence should return the "final" array index, not the number of bytes/characters read. Similarly, write-sequence needs to do its work, then return the sequence itself. Attached is a patch with what I hope are the required fixes, please review. Two additional notes, 1. Even after loading my proposed fixes, I'd would still get this, which is ok, but the error message is not obvious enough. This is actually a method not found error, not a type error. There appears to be some type guessing going on, so perhaps the message should indicate failure to dispatch on STREAM or similar. (gray-streams::stream-element-type (make-instance 'test-gray-binary-input-stream)) The value #<TEST-GRAY-BINARY-INPUT-STREAM {500BBBF9}> is not of type STREAM. [Condition of type TYPE-ERROR] So we just need something like this, (defmethod gray-streams::stream-element-type ((stream test-gray-binary-input-stream)) '(unsigned-byte 8)) 2. I also don't understand this snippet in the original code, converting unsigned-bytes/integers (basically the wrong type) to a character? (#+nil ccl:int-char code-char (elt sequence n)) Of course, after patching the above, I discovered that re-compiling flexi-streams got rid of my particular set of problems :-) Yong

3 6

[armedbear-devel] abcl-1.3.0 released
by Mark Evenson 08 Apr '15

08 Apr '15

http://abcl.org/releases/1.3.0 abcl-1.3.0 is a feature release. 15-MAR-2014 ## Features * Make LispStackFrame.UNAVAILABLE_ARG a singleton object, and lazily create the little used portions of the Lisp stack. Aggressively cache and control the use of memory by the underlying Lisp stack frame representation by introducing the private LispThread.StackFrame and LispThread.StackSegments classes. Contributed by Dmitry Nadezhin. LispStackFrame object are allocated on every LispThread.execute(...) . However, they are seldom [accessed] ([... verify via] inspect[tion of the] stack trace). This patch delays allocation of LispStackFrame? objects until they are requested. Raw information about stack frames is stored in stack. Stack is an Object[] array (more precisely a list of [...]4 [Mib] Object[] arrays). ME: We are going to need a way to try to less agressively grab 4Mib chunks in low memory situations. Memory profiling of ABCL shows that the classes with largest allocation count are org.armedbear.lisp.LispStackFrame and org.armedbear.lisp.LispStackFrame.UnavailableArgument. Contributed by Dmitry Nadezhin. [r14572]: http://abcl.org/trac/changeset/14572 [r14579]: http://abcl.org/trac/changeset/14579 * ASDF 3.0.1.94 shipped with the implementation * per function call stack and memory exception handler in CL:COMPILE Inline calls to jrun-exception-protected (used by handler-bind to catch out of memory conditions). This commit saves generation roughly 50 cls files. [r14552]: http://abcl.org/trac/changeset/14552 * SYS:SHA256 audited The functionality if the SYS:SHA256 algorithim has been audited for use on inputs of single for files with recently shipping ORCL Java 7 implementations (through jdk-1.7.0_51). [r14582]: http://abcl.org/trac/changeset/14582 * Connect to NetBeans controlled JDWP via SLIME The Netbeans IDE configuration now includes a way to connect to the running-under-jdb ABCL via SLIME. One needs a version of SLIME able to be loaded from its 'swank.asd' definition. * Install 'abcl.jar' and 'abcl-contrib.jar' locally as Maven artifacts The Ant `abcl.mvn.install` target now installs build artifacts into the local Maven repository (Olof-Joachim Frahm) [r14579]: http://abcl.org/trac/changeset/14606 ## Compatibility * CL:DIRECTORY The implementation specific :RESOLVE-SYMLINKS argument to the ANSI DIRECTORY function has been changed to nil. This implements behavior closer to SBCL and guarantees that a DIRECTORY operation will not signal a file error. [r14619]: http://abcl.org/trac/changeset/14619 [ticket-340]: http://abcl.org/trac/ticket/340 ## Fixes * Fix CL:SLEEP for intervals less than a millisecond. For intervals less than or equal to a nanosecond, including an interval of zero, the current thread merely yields execution to other threads. [r14632]: http://abcl.org/trac/changeset/14632 ## Tested ### "Java_HotSpot(TM)_64-Bit_Server_VM-Oracle_Corporation-1.7.0_51-b13" "x86_64-Mac_OS_X-10.9.1" ### "Java_HotSpot(TM)_64-Bit_Server_VM-Oracle_Corporation-1.8.0-b129" "x86_64-Mac_OS_X-10.9.2" ## Contrib #### abcl-asdf * Now working with both Maven 3.0.x and 3.1.x. Thanks to Anton for the help! [ticket-328]: http://abcl.org/trac/ticket/328 * cache Maven dependency resolution to avoid repeated lookups. Instead of calling ABCL-ASDF:RESOLVE in both the ASDF COMPILE-OP and LOAD-OP, we now cache the result of invocation in COMPILE-OP and add this value in the LOAD-OP phase. Contributed by Cyrus Harmon. [r14631]: http://abcl.org/trac/changeset/14631 #### jna Now references jna-4.0.0. Some incompatibility with CFFI ([in progress with fixing upstream][cffi-easye]). [cffi-easye]: http://github.com/easye/cffi/

5 16