October 2020 - armedbear-devel - mailman3.common-lisp.net

Setting different class loader
by Steven Nunez 06 Feb '23

06 Feb '23

Greetings all, I have what I think is a problem with the ABCL class loader. I am working with a 'big data' library, Spark, but run into an issue on line 2 of the programming guide example. I am able to load the JARs from Maven with the ASDF system definition: (asdf:defsystem #:spark :description "Wrapper for Spark 3.0" :serial t :defsystem-depends-on (abcl-asdf) :depends-on (#:jss #:javaparser) :components ((:mvn "org.apache.spark/spark-core_2.12" :version "3.0.0") (:file "package") (:file "spark"))) and can create a SparkConf object: (defvar *spark-conf* #1"new SparkConf() .setAppName("abcl-app") .setMaster("local")" ) But when I try to create a 'context' (defvar *sc* (new 'JavaSparkContext *spark-conf*)) I get an error in the initialisation: Java exception 'java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.package$'. There isn't much on this except from the Jenkins guys, who have attempted to put Spark and Spark applications into a CI system. They seem to think that it's related to a call to get a properties file in the package class and on a StackOverflow discussion suggested that "you should make sure that you set the classloader that Spark was loaded through using the Thread.currentThread().setContextClassLoader(myCustomLoader) call". I've verified with (java:dump-classpath) that the JAR is on the ABCL classpath, and the JAR file does contain the spark-version-info.properties file. I've also tried getting the file myself with: (defvar rs #1"Thread.currentThread() .getContextClassLoader() .getResourceAsStream("spark-version-info.properties")" ) which returns nil, so their theory may be correct. Messing around with class loaders is a bit beyond my 20 year old Java knowledge so I thought I'd ask here if anyone has any ideas on how I can load Spark in way to use the default Java class loader. Alternatively it occurs to me to ask why the ABCL class loader isn't able to find the properties file if the JAR is on the classpath and then to correct whatever that problem is. Cheers, Steve

3 6

Bug report: Loading a FASL into a package which doesn't import SETQ
by Robert Munyer 16 Nov '20

16 Nov '20

The code below works in CCL, CLISP and SBCL, but not in ABCL. $ echo '(cl:format cl:t "~&I am in package ~s.~%" cl:*package*)' > foo.lisp $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.236 seconds. Startup completed in 0.931 seconds. Type ":help" for a list of available commands. CL-USER(1): (defpackage "BAR" (:use)) #<PACKAGE BAR> CL-USER(2): (let ((*package* (find-package "BAR"))) (load (compile-file "foo"))) ; Compiling /tmp/foo/abcl-bin-1.7.1/foo.lisp ... ; (COMMON-LISP:FORMAT COMMON-LISP:T ...) ; Wrote /tmp/foo/abcl-bin-1.7.1/foo.abcl (0.036 seconds) Error loading /tmp/foo/abcl-bin-1.7.1/foo.abcl at line 4 (offset 125) #<THREAD "interpreter" {1B31C40C}>: Debugger invoked on condition of type UNDEFINED-FUNCTION The function BAR::SETQ is undefined. Restarts: 0: CONTINUE Try again. 1: USE-VALUE Specify a function to call instead. 2: RETURN-VALUE Return one or more values from the call to SETQ. 3: TOP-LEVEL Return to top level. [1] BAR(3):

4 16

Bug report: COMPILE-FILE, COMPILE-FILE-PATHNAME disagree on output dir
by Robert Munyer 11 Nov '20

11 Nov '20

COMPILE-FILE-PATHNAME is supposed to return "the pathname that COMPILE-FILE would write into, if given the same arguments" but in the example below, COMPILE-FILE-PATHNAME returns "src/obj/foo.abcl" while the file that is actually written into is "obj/foo.abcl". Looking at the spec, my opinion is that COMPILE-FILE is doing the right thing, and COMPILE-FILE-PATHNAME should be changed to match it. I suppose some might disagree with me about which behavior is correct, but no one can disagree that the two functions' behavior should match. $ mkdir src obj $ echo '(format t "~&Hello, world!~%")' > src/foo.lisp $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.179 seconds. Startup completed in 0.828 seconds. Type ":help" for a list of available commands. CL-USER(1): (defvar *infile* (make-pathname :directory '(:relative "src") :name "foo" :type "lisp")) *INFILE* CL-USER(2): (defvar *outfile* (make-pathname :directory '(:relative "obj") :type "abcl" :defaults *infile*)) *OUTFILE* CL-USER(3): (compile-file *infile* :output-file *outfile*) ; Compiling /tmp/foo/src/foo.lisp ... ; (FORMAT T ...) ; Wrote /tmp/foo/obj/foo.abcl (0.014 seconds) #P"/tmp/foo/obj/foo.abcl" NIL NIL CL-USER(4): (compile-file-pathname *infile* :output-file *outfile*) #P"/tmp/foo/src/obj/foo.abcl" CL-USER(5):

2 5

ABCL 1.8.0
by Mark Evenson 01 Nov '20

01 Nov '20

ABCL 1.8.0 Under the gathering storms of the Fall 2020, we are pleased to release ABCL 1.8.0 as the Ninth major revision of the implementation. This Ninth Edition of the implementation now supports building and running on the recently released openjdk15 platform. This release is intended as the last major release to support the openjdk6, openjdk7, and openjdk8 platforms, for with abcl-2.0.0 we intend to move the minimum platform to openjdk11 or better in order to efficiently [implement atomic memory compare and swap operations][github/issues/92]. With this release, the implementation of the EXT:JAR-PATHNAME and EXT:URL-PATHNAME subtypes of cl:PATHNAME has been fixed to the point that arbitrary references to ZIP archives within archives now work for read-only stream operations (CL:PROBE-FILE, CL:TRUENAME, CL:OPEN, CL:LOAD, CL:FILE-WRITE-DATE, CL:DIRECTORY, and CL:MERGE-PATHNAMES). The previous versions of the implementation relied on the ability for java.net.URL to open streams of an archive within an archive, behavior that was silently dropped after Java 5, and consequently hasn't worked on common platforms supported by the Bear in a long time. The overhaul of the implementation restores the feasibility of accessing fasls from within jar files. Interested parties may examine the ASDF-JAR contrib for a recipe for packaging and accessing such artifacts. Please consult the ["Beyond ANSI: Pathnames" Section 4.2 of the User Manual][manual] for further details for how namestrings and components of PATHNAME objects have been revised. A more comprehensive list of [CHANGES][] is available with the source. [1.8.0]: https://abcl.org/releases/1.8.0/ [asdf-jar]: https://abcl.org/svn/trunk/abcl/contrib/asdf-jar/README.markdown [github/issues/92]: https://github.com/armedbear/abcl/issues/92 [manual]: https://abcl.org/releases/1.8.0/abcl-1.8.0.pdf [CHANGES]: https://abcl.org/svn/tags/1.8.0/CHANGES -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

5 9

Bug report: Wrong file type returned by COMPILE-FILE-PATHNAME
by Robert Munyer 27 Oct '20

27 Oct '20

The code below returns "lx64fsl" in CCL, "fas" in CLISP, and "fasl" in SBCL, but returns "lisp" in ABCL. I think it should return "abcl" in ABCL. $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.586 seconds. Startup completed in 1.336 seconds. Type ":help" for a list of available commands. CL-USER(1): (pathname-type (compile-file-pathname (make-pathname :name "foo" :type "lisp") :output-file (make-pathname :name "bar"))) "lisp" CL-USER(2):

2 1

Possible that abcl-2.0.0 will not support openjdk8
by Mark Evenson 06 Oct '20

06 Oct '20

A question for all Bear fans: would you be severly impacted if abcl-2.0.0 dropped support for openjdk8, just supporting openjdk11 and up? In order to realistically get ABCL to implement Compare and Swap at all efficiently, I think we need to require that the JVM memory access model conforms to the fixes identified by Doug Lea et. al. as codified in the [VarHandle interface first introduced in openjdk9][varhandle]. abcl-1.8.0 would be the last release supporting openjdk8. Comments with this policy? Please yell early if you absolutely could not migrate an existing use of ABCL to openjdk11. [cas]: https://github.com/armedbear/abcl/issues/92 [varhandle]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/invo… -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

2 1

Bug report: Regression, immediately invoked function expressions
by Robert Munyer 01 Oct '20

01 Oct '20

The function below works in CCL, CLISP, SBCL and ABCL 1.4.0, but not in ABCL 1.7.1. $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.188 seconds. Startup completed in 0.869 seconds. Type ":help" for a list of available commands. CL-USER(1): (defun add3 (foo) ((lambda (bar) ((lambda (baz) (1+ baz)) (1+ bar))) (1+ foo))) #<THREAD "interpreter" {32B46F86}>: Debugger invoked on condition of type TYPE-ERROR The value (LAMBDA (BAZ) (1+ BAZ)) is not of type SYMBOL. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2):

3 5

Bug report: IllegalStateException, when reading after rewinding
by Robert Munyer 01 Oct '20

01 Oct '20

The code below works in CCL, CLISP and SBCL, but not in ABCL. $ echo hello > hello.text $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.181 seconds. Startup completed in 0.945 seconds. Type ":help" for a list of available commands. CL-USER(1): (with-open-file (in "hello.text") (list (read in) (read in nil) (file-position in :start) (read in))) java.lang.IllegalStateException: Current state = CODING_END, new state = CODING at java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:992) at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:572) at org.armedbear.lisp.util.RandomAccessCharacterFile.read(RandomAccessCharacterFile.java:392) [ many stack frames elided ] at org.armedbear.lisp.Main$1.run(Main.java:48) at java.lang.Thread.run(Thread.java:748) #<THREAD "interpreter" {2CD58E2}>: Debugger invoked on condition of type ERROR Caught java.lang.IllegalStateException: Current state = CODING_END, new state = CODING. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2):

2 2