- armedbear-devel - mailman3.common-lisp.net

Bug report: COMPILE-FILE not doing minimal compilation in DEFCONSTANT/DEFPARAMETER/DEFVAR
by Robert Munyer 16 Nov '20

16 Nov '20

COMPILE-FILE is supposed to expand all macros in such a way that they will not be expanded again when the compiled file is loaded. (3.2.2.2) The file below has 4 output lines that start with "*** Expanding macro". None of those lines should be printed when the compiled file is loaded. ------- begin test-min-compilation.lisp ------- (macrolet ((m1 () (format t "~&*** Expanding macro 1.~%") '(format t "~&Output of macro 1.~%"))) (m1)) (defconstant +dc+ (macrolet ((m2 () (format t "~&*** Expanding macro 2 inside DEFCONSTANT.~%") '(format t "~&Output of macro 2.~%"))) (m2))) (defparameter *dp* (macrolet ((m3 () (format t "~&*** Expanding macro 3 inside DEFPARAMETER.~%") '(format t "~&Output of macro 3.~%"))) (m3))) (defvar *dv* (macrolet ((m4 () (format t "~&*** Expanding macro 4 inside DEFVAR.~%") '(format t "~&Output of macro 4.~%"))) (m4))) ------- end test-min-compilation.lisp ------- CCL, CLISP, ECL and SBCL print none of the 4, ABCL prints 3 of the 4. This one might be relatively easy, because ABCL is already handling #1 correctly, and just needs to handle #2/3/4 in the same way as #1. The rest of this message is the transcript. $ java -jar abcl-344b4f66.jar Armed Bear Common Lisp 1.8.1-dev Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.183 seconds. Startup completed in 0.837 seconds. Type ":help" for a list of available commands. CL-USER(1): (compile-file "test-min-compilation.lisp") ; Compiling /tmp/foo/test-min-compilation.lisp ... ; (M1) *** Expanding macro 1. ; (DEFCONSTANT +DC+ ...) *** Expanding macro 2 inside DEFCONSTANT. Output of macro 2. ; (DEFPARAMETER *DP* ...) ; (DEFVAR *DV* ...) ; Wrote /tmp/foo/test-min-compilation.abcl (0.163 seconds) #P"/tmp/foo/test-min-compilation.abcl" NIL NIL CL-USER(2): (load "test-min-compilation.abcl") Output of macro 1. *** Expanding macro 2 inside DEFCONSTANT. Output of macro 2. *** Expanding macro 3 inside DEFPARAMETER. Output of macro 3. *** Expanding macro 4 inside DEFVAR. Output of macro 4. T CL-USER(3):

2 1

Bug report: Loading a FASL into a package which doesn't import SETQ
by Robert Munyer 16 Nov '20

16 Nov '20

The code below works in CCL, CLISP and SBCL, but not in ABCL. $ echo '(cl:format cl:t "~&I am in package ~s.~%" cl:*package*)' > foo.lisp $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.236 seconds. Startup completed in 0.931 seconds. Type ":help" for a list of available commands. CL-USER(1): (defpackage "BAR" (:use)) #<PACKAGE BAR> CL-USER(2): (let ((*package* (find-package "BAR"))) (load (compile-file "foo"))) ; Compiling /tmp/foo/abcl-bin-1.7.1/foo.lisp ... ; (COMMON-LISP:FORMAT COMMON-LISP:T ...) ; Wrote /tmp/foo/abcl-bin-1.7.1/foo.abcl (0.036 seconds) Error loading /tmp/foo/abcl-bin-1.7.1/foo.abcl at line 4 (offset 125) #<THREAD "interpreter" {1B31C40C}>: Debugger invoked on condition of type UNDEFINED-FUNCTION The function BAR::SETQ is undefined. Restarts: 0: CONTINUE Try again. 1: USE-VALUE Specify a function to call instead. 2: RETURN-VALUE Return one or more values from the call to SETQ. 3: TOP-LEVEL Return to top level. [1] BAR(3):

4 16

ABCL Docker Images
by Eric Timmons 12 Nov '20

12 Nov '20

Hello fans of the Armed Bear, As you've probably noticed, Docker and similar technologies are extremely popular these days. Additionally, there is unfortunately no Common Lisp representation in Docker's Official Images program (the Docker standard library, if you will). I'd like to change that and ABCL seems like a good candidate to start with because it's both relatively simple to Dockerize and Mark is supportive of getting CL representation in that program. I've been maintaining some Common Lisp related Docker images in my personal namespace (daewok) for several years now. I recently moved the repos to a gitlab.common-lisp.net group (mirrored to Github) and have cleaned up/gotten ABCL's repos into a state where I believe they adhere to all of the Official Image program's guidelines and are ready for inclusion. I wanted to reach out to 1) get any feedback from ABCL devs and users to make sure the images are useful and 2) get consent from the ABCL devs to submit to the program. First, the code: The Dockerfiles themselves are located at https://gitlab.common-lisp.net/cl-docker-images/abcl The manifest for inclusion in the Official Images program is at https://gitlab.common-lisp.net/cl-docker-images/official-images/-/blob/abcl… The documentation for inclusion on Docker Hub is at https://gitlab.common-lisp.net/cl-docker-images/docs/-/tree/abcl/abcl All of these are also mirrored to the cl-docker-images org on Github. The images are built for the cross product of {buster, windowsservercore-1809, windowsservercore-ltsc2016} x {jdk8, jdk11, jdk15} x {arm64, amd64} (except for all combos containing jdk8 and arm64). The current versions of the images are located at https://hub.docker.com/r/daewok/abcl if you would like to check them out. I am planning that only the latest version of ABCL will be "supported". The initial request would also backfill 1.7.1. Once a version becomes unsupported it would not be removed from Docker hub, it just receives no more base image updates. Next, some info about Docker Official Images: Documentation on the program is located at https://docs.docker.com/docker-hub/official_images/ and https://github.com/docker-library/official-images In summary, every image in the program is reviewed by a team for best practices and security, built by that team, and is available for pulling without specifying a namespace (e.g., `docker pull abcl:latest`). This helps give users confidence in the code they're running, as opposed to pulling from random personal namespaces. Even the Official Images team admits that "Official Images" is not a great name as it can imply that the upstream of the software contained in the image maintains the images when that is frequently not the case. As I'm not an ABCL dev myself, the image would be considered community maintained and the description on Docker Hub would clearly point to the cl-docker-images/abcl repo as where to file issues with the images. But it is very conceivable that random users may show up on this list asking for help. I am subscribed to this list and will do my best to redirect/answer any of those types of emails that come in. Thoughts, questions, concerns? -Eric

3 4

Bug report: COMPILE-FILE, COMPILE-FILE-PATHNAME disagree on output dir
by Robert Munyer 11 Nov '20

11 Nov '20

COMPILE-FILE-PATHNAME is supposed to return "the pathname that COMPILE-FILE would write into, if given the same arguments" but in the example below, COMPILE-FILE-PATHNAME returns "src/obj/foo.abcl" while the file that is actually written into is "obj/foo.abcl". Looking at the spec, my opinion is that COMPILE-FILE is doing the right thing, and COMPILE-FILE-PATHNAME should be changed to match it. I suppose some might disagree with me about which behavior is correct, but no one can disagree that the two functions' behavior should match. $ mkdir src obj $ echo '(format t "~&Hello, world!~%")' > src/foo.lisp $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.179 seconds. Startup completed in 0.828 seconds. Type ":help" for a list of available commands. CL-USER(1): (defvar *infile* (make-pathname :directory '(:relative "src") :name "foo" :type "lisp")) *INFILE* CL-USER(2): (defvar *outfile* (make-pathname :directory '(:relative "obj") :type "abcl" :defaults *infile*)) *OUTFILE* CL-USER(3): (compile-file *infile* :output-file *outfile*) ; Compiling /tmp/foo/src/foo.lisp ... ; (FORMAT T ...) ; Wrote /tmp/foo/obj/foo.abcl (0.014 seconds) #P"/tmp/foo/obj/foo.abcl" NIL NIL CL-USER(4): (compile-file-pathname *infile* :output-file *outfile*) #P"/tmp/foo/src/obj/foo.abcl" CL-USER(5):

2 5

ABCL 1.8.0
by Mark Evenson 01 Nov '20

01 Nov '20

ABCL 1.8.0 Under the gathering storms of the Fall 2020, we are pleased to release ABCL 1.8.0 as the Ninth major revision of the implementation. This Ninth Edition of the implementation now supports building and running on the recently released openjdk15 platform. This release is intended as the last major release to support the openjdk6, openjdk7, and openjdk8 platforms, for with abcl-2.0.0 we intend to move the minimum platform to openjdk11 or better in order to efficiently [implement atomic memory compare and swap operations][github/issues/92]. With this release, the implementation of the EXT:JAR-PATHNAME and EXT:URL-PATHNAME subtypes of cl:PATHNAME has been fixed to the point that arbitrary references to ZIP archives within archives now work for read-only stream operations (CL:PROBE-FILE, CL:TRUENAME, CL:OPEN, CL:LOAD, CL:FILE-WRITE-DATE, CL:DIRECTORY, and CL:MERGE-PATHNAMES). The previous versions of the implementation relied on the ability for java.net.URL to open streams of an archive within an archive, behavior that was silently dropped after Java 5, and consequently hasn't worked on common platforms supported by the Bear in a long time. The overhaul of the implementation restores the feasibility of accessing fasls from within jar files. Interested parties may examine the ASDF-JAR contrib for a recipe for packaging and accessing such artifacts. Please consult the ["Beyond ANSI: Pathnames" Section 4.2 of the User Manual][manual] for further details for how namestrings and components of PATHNAME objects have been revised. A more comprehensive list of [CHANGES][] is available with the source. [1.8.0]: https://abcl.org/releases/1.8.0/ [asdf-jar]: https://abcl.org/svn/trunk/abcl/contrib/asdf-jar/README.markdown [github/issues/92]: https://github.com/armedbear/abcl/issues/92 [manual]: https://abcl.org/releases/1.8.0/abcl-1.8.0.pdf [CHANGES]: https://abcl.org/svn/tags/1.8.0/CHANGES -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

5 9

Bug report: Wrong file type returned by COMPILE-FILE-PATHNAME
by Robert Munyer 27 Oct '20

27 Oct '20

The code below returns "lx64fsl" in CCL, "fas" in CLISP, and "fasl" in SBCL, but returns "lisp" in ABCL. I think it should return "abcl" in ABCL. $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.586 seconds. Startup completed in 1.336 seconds. Type ":help" for a list of available commands. CL-USER(1): (pathname-type (compile-file-pathname (make-pathname :name "foo" :type "lisp") :output-file (make-pathname :name "bar"))) "lisp" CL-USER(2):

2 1

Possible that abcl-2.0.0 will not support openjdk8
by Mark Evenson 06 Oct '20

06 Oct '20

A question for all Bear fans: would you be severly impacted if abcl-2.0.0 dropped support for openjdk8, just supporting openjdk11 and up? In order to realistically get ABCL to implement Compare and Swap at all efficiently, I think we need to require that the JVM memory access model conforms to the fixes identified by Doug Lea et. al. as codified in the [VarHandle interface first introduced in openjdk9][varhandle]. abcl-1.8.0 would be the last release supporting openjdk8. Comments with this policy? Please yell early if you absolutely could not migrate an existing use of ABCL to openjdk11. [cas]: https://github.com/armedbear/abcl/issues/92 [varhandle]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/invo… -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

2 1

Bug report: Regression, immediately invoked function expressions
by Robert Munyer 01 Oct '20

01 Oct '20

The function below works in CCL, CLISP, SBCL and ABCL 1.4.0, but not in ABCL 1.7.1. $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.188 seconds. Startup completed in 0.869 seconds. Type ":help" for a list of available commands. CL-USER(1): (defun add3 (foo) ((lambda (bar) ((lambda (baz) (1+ baz)) (1+ bar))) (1+ foo))) #<THREAD "interpreter" {32B46F86}>: Debugger invoked on condition of type TYPE-ERROR The value (LAMBDA (BAZ) (1+ BAZ)) is not of type SYMBOL. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2):

3 5

Bug report: IllegalStateException, when reading after rewinding
by Robert Munyer 01 Oct '20

01 Oct '20

The code below works in CCL, CLISP and SBCL, but not in ABCL. $ echo hello > hello.text $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.181 seconds. Startup completed in 0.945 seconds. Type ":help" for a list of available commands. CL-USER(1): (with-open-file (in "hello.text") (list (read in) (read in nil) (file-position in :start) (read in))) java.lang.IllegalStateException: Current state = CODING_END, new state = CODING at java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:992) at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:572) at org.armedbear.lisp.util.RandomAccessCharacterFile.read(RandomAccessCharacterFile.java:392) [ many stack frames elided ] at org.armedbear.lisp.Main$1.run(Main.java:48) at java.lang.Thread.run(Thread.java:748) #<THREAD "interpreter" {2CD58E2}>: Debugger invoked on condition of type ERROR Caught java.lang.IllegalStateException: Current state = CODING_END, new state = CODING. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2):

2 2

The road to ABCL 2.0.0
by Mark Evenson 19 Sep '20

19 Sep '20

Greetings Armed Bear fans, Since, I've been able to get our favorite Bear to run on [openjdk15][6], which has now replaced openjdk14 in our CI test regime, an update is in order about the release schedule of [ABCL 2.0][3] for those of you following along at home, and let’s face it: only the homeless aren’t at home these days. ** Either abcl-1.8.0, or abcl-2.0.0 or both will be released by 10.10.2020 ** [abcl-1.8.0][1] will be the last release support openjdk6 and openjdk7 runtimes. [abcl-2.0.0][2] will require openjdk8, and will run on openjdk11, openjdk13, openjdk14, and openjdk15. We will fix compatibility with previous versions of the implementation by incompletely and inefficiently re-implement the use of CL:PATHNAME-DEVICE to "paper over” the use of nested jar references. The ability to access nested jar entries via CL:PATHNAME references has been broken since the openjdk support for nested references to jar files dropped for their creation. Nobody seemed to notice this problem, accept for customers of the [ASDF-JAR][4] contrib which is currently unable to load classes Since either openjdk5 or openjdk6, references like #p"jar:jar:file:/var/tmp/cl-ppcre-2.1.1.jar!/cl-ppcre/packages.abcl!/__loader__._” while syntactically valid cannot actually address the contents on zip files withing zip files. Since we would like such nested jar entries for use by strategies like ABCL All-in-Once (AiO), with [abcl-1.8.0][2] will allow an arbitrary PATHNAME-DEVICE component designating a sequence of nested archive entries. With openjdk8, we can use abstractions to make the caching efficient, but the current strategy (will) just reference everything by a weak reference to allow the implementation to garbage collect if it finds itself under memory pressure. In an era of the available of cheap gibibyte heaps, potentially retaining references to the bytes of artifacts accessed via CL:LOAD will be a viable strategy for at least the core implementation which is only ~ 11 mebibyte. ABCL-2.0.0 will use all of the ABCL-1.8.0 code in additional to an implementation of multi-threaded sensitive atomic operations supporting ubiquitous contemporary ANSI CL compare and swap for memory. The presence of a new compiler for ABCL-2.0.0 is no longer a release driver as it currently stands, but I would love some help if it shows up. Once I get things working well under openjdk8, I will start to work on the new compiler infrastructure. Comments, patches, criticisms welcome! yours, Mark [1]: <https://github.com/armedbear/abcl/blob/05e4b396658d7ba4bd074a8e4f627deec3b7…> [2]: <https://abcl.org/trac/milestone/1.8.0> [3]: <https://abcl.org/trac/milestone/2.0.0> [4]: <https://gitlab.common-lisp.net/abcl/abcl/-/blob/master/contrib/abcl-asdf/RE…> [5]: <https://gitlab.common-lisp.net/abcl/abcl/-/commit/0ba7dea278a474efe0a8e7550…> [6]: <https://travis-ci.org/github/armedbear/abcl/jobs/728128870> -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

1 0