November 2020 - armedbear-devel - mailman3.common-lisp.net

Setting different class loader
by Steven Nunez 06 Feb '23

06 Feb '23

Greetings all, I have what I think is a problem with the ABCL class loader. I am working with a 'big data' library, Spark, but run into an issue on line 2 of the programming guide example. I am able to load the JARs from Maven with the ASDF system definition: (asdf:defsystem #:spark :description "Wrapper for Spark 3.0" :serial t :defsystem-depends-on (abcl-asdf) :depends-on (#:jss #:javaparser) :components ((:mvn "org.apache.spark/spark-core_2.12" :version "3.0.0") (:file "package") (:file "spark"))) and can create a SparkConf object: (defvar *spark-conf* #1"new SparkConf() .setAppName("abcl-app") .setMaster("local")" ) But when I try to create a 'context' (defvar *sc* (new 'JavaSparkContext *spark-conf*)) I get an error in the initialisation: Java exception 'java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.package$'. There isn't much on this except from the Jenkins guys, who have attempted to put Spark and Spark applications into a CI system. They seem to think that it's related to a call to get a properties file in the package class and on a StackOverflow discussion suggested that "you should make sure that you set the classloader that Spark was loaded through using the Thread.currentThread().setContextClassLoader(myCustomLoader) call". I've verified with (java:dump-classpath) that the JAR is on the ABCL classpath, and the JAR file does contain the spark-version-info.properties file. I've also tried getting the file myself with: (defvar rs #1"Thread.currentThread() .getContextClassLoader() .getResourceAsStream("spark-version-info.properties")" ) which returns nil, so their theory may be correct. Messing around with class loaders is a bit beyond my 20 year old Java knowledge so I thought I'd ask here if anyone has any ideas on how I can load Spark in way to use the default Java class loader. Alternatively it occurs to me to ask why the ABCL class loader isn't able to find the properties file if the JAR is on the classpath and then to correct whatever that problem is. Cheers, Steve

3 6

CLOS MOP bug in 1.8.0
by Pascal Costanza 27 Nov '20

27 Nov '20

Hi, I have encountered a bug in 1.8.0 when doing my regular Closer to MOP checks. This must be a new bug, because 1.7.1 was fine. Here is the case that triggers the problem: (use-package :mop) (defclass test-direct-slot-definition (standard-direct-slot-definition) ()) (defclass test-effective-slot-definition (standard-effective-slot-definition) ()) (defclass test-class (standard-class) ()) (defmethod validate-superclass ((class test-class) (superclass standard-class)) t) (defmethod direct-slot-definition-class ((class test-class) &rest initargs) (declare (ignore initargs)) (find-class 'test-direct-slot-definition)) (defmethod effective-slot-definition-class ((class test-class) &rest initargs) (declare (ignore initargs)) (find-class 'test-effective-slot-definition)) (defclass test-object () ((some-slot :accessor some-slot :initarg :some-slot :initform 'some-slot :type symbol :allocation :class :documentation "a slot")) (:metaclass test-class)) (unless (class-finalized-p (find-class 'test-object)) (finalize-inheritance (find-class 'test-object))) Armed Bear Common Lisp 1.8.0 Java 15.0.1 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM Low-level initialization completed in 0.134 seconds. Startup completed in 0.65 seconds. Loading /Users/costanza/.abclrc completed in 6.061 seconds. Type ":help" for a list of available commands. CL-USER(1): (load "test.lisp") Error loading /Users/costanza/Temp/test.lisp at line 36 (offset 943) #<THREAD "interpreter" {1970D65C}>: Debugger invoked on condition of type UNBOUND-SLOT The slot LOCATION is unbound in the object #<TEST-EFFECTIVE-SLOT-DEFINITION {556F331F}>. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2): :bt 0: (SYSTEM:BACKTRACE) 1: (INVOKE-DEBUGGER #<UNBOUND-SLOT {67D22A29}>) 2: (ERROR UNBOUND-SLOT :INSTANCE #<TEST-EFFECTIVE-SLOT-DEFINITION {556F331F}> :NAME SYSTEM::LOCATION) 3: (#<FUNCTION #<FUNCTION {22446549}> {22446549}> (#<STANDARD-CLASS TEST-EFFECTIVE-SLOT-DEFINITION {6670307A}> #<TEST-EFFECTIVE-SLOT-DEFINITION {556F331F}> SYSTEM::LOCATION)) 4: (SLOT-VALUE #<TEST-EFFECTIVE-SLOT-DEFINITION {556F331F}> SYSTEM::LOCATION) 5: (SLOT-DEFINITION-LOCATION #<TEST-EFFECTIVE-SLOT-DEFINITION {556F331F}>) 6: (MOP::STD-FINALIZE-INHERITANCE #<TEST-CLASS TEST-OBJECT {67510DAF}>) 7: (#<FUNCTION #<FUNCTION {2C5408FE}> {2C5408FE}> (#<TEST-CLASS TEST-OBJECT {67510DAF}>)) I believe the bug is triggered by find-class or class-finalized-p, but not sure. Pascal -- Pascal Costanza

2 1

Feature request: command-line option to disable debugger
by Robert Munyer 27 Nov '20

27 Nov '20

Many Lisp implementations (including CCL, CLISP and SBCL) have a command-line option that causes unhandled errors to terminate the Lisp process instead of invoking the debugger. This option is useful for programs that don't have irreplaceable valuable state in the heap, and that have users whose reaction upon seeing a debug REPL would be "How do I get out of this!?". Also, a few days ago someone on IRC mentioned having the problem that "if there is an error in the expression" the server thread "gets locked in the ABCL repl". I think his or her use-case might also benefit from a debugger-disabling switch. Happy Thanksgiving!

2 1

Bug report: COMPILE-FILE not doing minimal compilation in DEFCONSTANT/DEFPARAMETER/DEFVAR
by Robert Munyer 15 Nov '20

15 Nov '20

COMPILE-FILE is supposed to expand all macros in such a way that they will not be expanded again when the compiled file is loaded. (3.2.2.2) The file below has 4 output lines that start with "*** Expanding macro". None of those lines should be printed when the compiled file is loaded. ------- begin test-min-compilation.lisp ------- (macrolet ((m1 () (format t "~&*** Expanding macro 1.~%") '(format t "~&Output of macro 1.~%"))) (m1)) (defconstant +dc+ (macrolet ((m2 () (format t "~&*** Expanding macro 2 inside DEFCONSTANT.~%") '(format t "~&Output of macro 2.~%"))) (m2))) (defparameter *dp* (macrolet ((m3 () (format t "~&*** Expanding macro 3 inside DEFPARAMETER.~%") '(format t "~&Output of macro 3.~%"))) (m3))) (defvar *dv* (macrolet ((m4 () (format t "~&*** Expanding macro 4 inside DEFVAR.~%") '(format t "~&Output of macro 4.~%"))) (m4))) ------- end test-min-compilation.lisp ------- CCL, CLISP, ECL and SBCL print none of the 4, ABCL prints 3 of the 4. This one might be relatively easy, because ABCL is already handling #1 correctly, and just needs to handle #2/3/4 in the same way as #1. The rest of this message is the transcript. $ java -jar abcl-344b4f66.jar Armed Bear Common Lisp 1.8.1-dev Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.183 seconds. Startup completed in 0.837 seconds. Type ":help" for a list of available commands. CL-USER(1): (compile-file "test-min-compilation.lisp") ; Compiling /tmp/foo/test-min-compilation.lisp ... ; (M1) *** Expanding macro 1. ; (DEFCONSTANT +DC+ ...) *** Expanding macro 2 inside DEFCONSTANT. Output of macro 2. ; (DEFPARAMETER *DP* ...) ; (DEFVAR *DV* ...) ; Wrote /tmp/foo/test-min-compilation.abcl (0.163 seconds) #P"/tmp/foo/test-min-compilation.abcl" NIL NIL CL-USER(2): (load "test-min-compilation.abcl") Output of macro 1. *** Expanding macro 2 inside DEFCONSTANT. Output of macro 2. *** Expanding macro 3 inside DEFPARAMETER. Output of macro 3. *** Expanding macro 4 inside DEFVAR. Output of macro 4. T CL-USER(3):

2 1

Bug report: Loading a FASL into a package which doesn't import SETQ
by Robert Munyer 15 Nov '20

15 Nov '20

The code below works in CCL, CLISP and SBCL, but not in ABCL. $ echo '(cl:format cl:t "~&I am in package ~s.~%" cl:*package*)' > foo.lisp $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.236 seconds. Startup completed in 0.931 seconds. Type ":help" for a list of available commands. CL-USER(1): (defpackage "BAR" (:use)) #<PACKAGE BAR> CL-USER(2): (let ((*package* (find-package "BAR"))) (load (compile-file "foo"))) ; Compiling /tmp/foo/abcl-bin-1.7.1/foo.lisp ... ; (COMMON-LISP:FORMAT COMMON-LISP:T ...) ; Wrote /tmp/foo/abcl-bin-1.7.1/foo.abcl (0.036 seconds) Error loading /tmp/foo/abcl-bin-1.7.1/foo.abcl at line 4 (offset 125) #<THREAD "interpreter" {1B31C40C}>: Debugger invoked on condition of type UNDEFINED-FUNCTION The function BAR::SETQ is undefined. Restarts: 0: CONTINUE Try again. 1: USE-VALUE Specify a function to call instead. 2: RETURN-VALUE Return one or more values from the call to SETQ. 3: TOP-LEVEL Return to top level. [1] BAR(3):

4 16

ABCL Docker Images
by Eric Timmons 12 Nov '20

12 Nov '20

Hello fans of the Armed Bear, As you've probably noticed, Docker and similar technologies are extremely popular these days. Additionally, there is unfortunately no Common Lisp representation in Docker's Official Images program (the Docker standard library, if you will). I'd like to change that and ABCL seems like a good candidate to start with because it's both relatively simple to Dockerize and Mark is supportive of getting CL representation in that program. I've been maintaining some Common Lisp related Docker images in my personal namespace (daewok) for several years now. I recently moved the repos to a gitlab.common-lisp.net group (mirrored to Github) and have cleaned up/gotten ABCL's repos into a state where I believe they adhere to all of the Official Image program's guidelines and are ready for inclusion. I wanted to reach out to 1) get any feedback from ABCL devs and users to make sure the images are useful and 2) get consent from the ABCL devs to submit to the program. First, the code: The Dockerfiles themselves are located at https://gitlab.common-lisp.net/cl-docker-images/abcl The manifest for inclusion in the Official Images program is at https://gitlab.common-lisp.net/cl-docker-images/official-images/-/blob/abcl… The documentation for inclusion on Docker Hub is at https://gitlab.common-lisp.net/cl-docker-images/docs/-/tree/abcl/abcl All of these are also mirrored to the cl-docker-images org on Github. The images are built for the cross product of {buster, windowsservercore-1809, windowsservercore-ltsc2016} x {jdk8, jdk11, jdk15} x {arm64, amd64} (except for all combos containing jdk8 and arm64). The current versions of the images are located at https://hub.docker.com/r/daewok/abcl if you would like to check them out. I am planning that only the latest version of ABCL will be "supported". The initial request would also backfill 1.7.1. Once a version becomes unsupported it would not be removed from Docker hub, it just receives no more base image updates. Next, some info about Docker Official Images: Documentation on the program is located at https://docs.docker.com/docker-hub/official_images/ and https://github.com/docker-library/official-images In summary, every image in the program is reviewed by a team for best practices and security, built by that team, and is available for pulling without specifying a namespace (e.g., `docker pull abcl:latest`). This helps give users confidence in the code they're running, as opposed to pulling from random personal namespaces. Even the Official Images team admits that "Official Images" is not a great name as it can imply that the upstream of the software contained in the image maintains the images when that is frequently not the case. As I'm not an ABCL dev myself, the image would be considered community maintained and the description on Docker Hub would clearly point to the cl-docker-images/abcl repo as where to file issues with the images. But it is very conceivable that random users may show up on this list asking for help. I am subscribed to this list and will do my best to redirect/answer any of those types of emails that come in. Thoughts, questions, concerns? -Eric

3 4

Bug report: COMPILE-FILE, COMPILE-FILE-PATHNAME disagree on output dir
by Robert Munyer 11 Nov '20

11 Nov '20

COMPILE-FILE-PATHNAME is supposed to return "the pathname that COMPILE-FILE would write into, if given the same arguments" but in the example below, COMPILE-FILE-PATHNAME returns "src/obj/foo.abcl" while the file that is actually written into is "obj/foo.abcl". Looking at the spec, my opinion is that COMPILE-FILE is doing the right thing, and COMPILE-FILE-PATHNAME should be changed to match it. I suppose some might disagree with me about which behavior is correct, but no one can disagree that the two functions' behavior should match. $ mkdir src obj $ echo '(format t "~&Hello, world!~%")' > src/foo.lisp $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.179 seconds. Startup completed in 0.828 seconds. Type ":help" for a list of available commands. CL-USER(1): (defvar *infile* (make-pathname :directory '(:relative "src") :name "foo" :type "lisp")) *INFILE* CL-USER(2): (defvar *outfile* (make-pathname :directory '(:relative "obj") :type "abcl" :defaults *infile*)) *OUTFILE* CL-USER(3): (compile-file *infile* :output-file *outfile*) ; Compiling /tmp/foo/src/foo.lisp ... ; (FORMAT T ...) ; Wrote /tmp/foo/obj/foo.abcl (0.014 seconds) #P"/tmp/foo/obj/foo.abcl" NIL NIL CL-USER(4): (compile-file-pathname *infile* :output-file *outfile*) #P"/tmp/foo/src/obj/foo.abcl" CL-USER(5):

2 5

ABCL 1.8.0
by Mark Evenson 01 Nov '20

01 Nov '20

ABCL 1.8.0 Under the gathering storms of the Fall 2020, we are pleased to release ABCL 1.8.0 as the Ninth major revision of the implementation. This Ninth Edition of the implementation now supports building and running on the recently released openjdk15 platform. This release is intended as the last major release to support the openjdk6, openjdk7, and openjdk8 platforms, for with abcl-2.0.0 we intend to move the minimum platform to openjdk11 or better in order to efficiently [implement atomic memory compare and swap operations][github/issues/92]. With this release, the implementation of the EXT:JAR-PATHNAME and EXT:URL-PATHNAME subtypes of cl:PATHNAME has been fixed to the point that arbitrary references to ZIP archives within archives now work for read-only stream operations (CL:PROBE-FILE, CL:TRUENAME, CL:OPEN, CL:LOAD, CL:FILE-WRITE-DATE, CL:DIRECTORY, and CL:MERGE-PATHNAMES). The previous versions of the implementation relied on the ability for java.net.URL to open streams of an archive within an archive, behavior that was silently dropped after Java 5, and consequently hasn't worked on common platforms supported by the Bear in a long time. The overhaul of the implementation restores the feasibility of accessing fasls from within jar files. Interested parties may examine the ASDF-JAR contrib for a recipe for packaging and accessing such artifacts. Please consult the ["Beyond ANSI: Pathnames" Section 4.2 of the User Manual][manual] for further details for how namestrings and components of PATHNAME objects have been revised. A more comprehensive list of [CHANGES][] is available with the source. [1.8.0]: https://abcl.org/releases/1.8.0/ [asdf-jar]: https://abcl.org/svn/trunk/abcl/contrib/asdf-jar/README.markdown [github/issues/92]: https://github.com/armedbear/abcl/issues/92 [manual]: https://abcl.org/releases/1.8.0/abcl-1.8.0.pdf [CHANGES]: https://abcl.org/svn/tags/1.8.0/CHANGES -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

5 9