- armedbear-devel - mailman3.common-lisp.net

Bug report: Wrong file type returned by COMPILE-FILE-PATHNAME
by Robert Munyer 27 Oct '20

27 Oct '20

The code below returns "lx64fsl" in CCL, "fas" in CLISP, and "fasl" in SBCL, but returns "lisp" in ABCL. I think it should return "abcl" in ABCL. $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_272 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.586 seconds. Startup completed in 1.336 seconds. Type ":help" for a list of available commands. CL-USER(1): (pathname-type (compile-file-pathname (make-pathname :name "foo" :type "lisp") :output-file (make-pathname :name "bar"))) "lisp" CL-USER(2):

2 1

Possible that abcl-2.0.0 will not support openjdk8
by Mark Evenson 06 Oct '20

06 Oct '20

A question for all Bear fans: would you be severly impacted if abcl-2.0.0 dropped support for openjdk8, just supporting openjdk11 and up? In order to realistically get ABCL to implement Compare and Swap at all efficiently, I think we need to require that the JVM memory access model conforms to the fixes identified by Doug Lea et. al. as codified in the [VarHandle interface first introduced in openjdk9][varhandle]. abcl-1.8.0 would be the last release supporting openjdk8. Comments with this policy? Please yell early if you absolutely could not migrate an existing use of ABCL to openjdk11. [cas]: https://github.com/armedbear/abcl/issues/92 [varhandle]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/invo… -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

2 1

Bug report: Regression, immediately invoked function expressions
by Robert Munyer 01 Oct '20

01 Oct '20

The function below works in CCL, CLISP, SBCL and ABCL 1.4.0, but not in ABCL 1.7.1. $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.188 seconds. Startup completed in 0.869 seconds. Type ":help" for a list of available commands. CL-USER(1): (defun add3 (foo) ((lambda (bar) ((lambda (baz) (1+ baz)) (1+ bar))) (1+ foo))) #<THREAD "interpreter" {32B46F86}>: Debugger invoked on condition of type TYPE-ERROR The value (LAMBDA (BAZ) (1+ BAZ)) is not of type SYMBOL. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2):

3 5

Bug report: IllegalStateException, when reading after rewinding
by Robert Munyer 01 Oct '20

01 Oct '20

The code below works in CCL, CLISP and SBCL, but not in ABCL. $ echo hello > hello.text $ java -jar abcl-1.7.1.jar Armed Bear Common Lisp 1.7.1 Java 1.8.0_265 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.181 seconds. Startup completed in 0.945 seconds. Type ":help" for a list of available commands. CL-USER(1): (with-open-file (in "hello.text") (list (read in) (read in nil) (file-position in :start) (read in))) java.lang.IllegalStateException: Current state = CODING_END, new state = CODING at java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:992) at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:572) at org.armedbear.lisp.util.RandomAccessCharacterFile.read(RandomAccessCharacterFile.java:392) [ many stack frames elided ] at org.armedbear.lisp.Main$1.run(Main.java:48) at java.lang.Thread.run(Thread.java:748) #<THREAD "interpreter" {2CD58E2}>: Debugger invoked on condition of type ERROR Caught java.lang.IllegalStateException: Current state = CODING_END, new state = CODING. Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(2):

2 2

The road to ABCL 2.0.0
by Mark Evenson 19 Sep '20

19 Sep '20

Greetings Armed Bear fans, Since, I've been able to get our favorite Bear to run on [openjdk15][6], which has now replaced openjdk14 in our CI test regime, an update is in order about the release schedule of [ABCL 2.0][3] for those of you following along at home, and let’s face it: only the homeless aren’t at home these days. ** Either abcl-1.8.0, or abcl-2.0.0 or both will be released by 10.10.2020 ** [abcl-1.8.0][1] will be the last release support openjdk6 and openjdk7 runtimes. [abcl-2.0.0][2] will require openjdk8, and will run on openjdk11, openjdk13, openjdk14, and openjdk15. We will fix compatibility with previous versions of the implementation by incompletely and inefficiently re-implement the use of CL:PATHNAME-DEVICE to "paper over” the use of nested jar references. The ability to access nested jar entries via CL:PATHNAME references has been broken since the openjdk support for nested references to jar files dropped for their creation. Nobody seemed to notice this problem, accept for customers of the [ASDF-JAR][4] contrib which is currently unable to load classes Since either openjdk5 or openjdk6, references like #p"jar:jar:file:/var/tmp/cl-ppcre-2.1.1.jar!/cl-ppcre/packages.abcl!/__loader__._” while syntactically valid cannot actually address the contents on zip files withing zip files. Since we would like such nested jar entries for use by strategies like ABCL All-in-Once (AiO), with [abcl-1.8.0][2] will allow an arbitrary PATHNAME-DEVICE component designating a sequence of nested archive entries. With openjdk8, we can use abstractions to make the caching efficient, but the current strategy (will) just reference everything by a weak reference to allow the implementation to garbage collect if it finds itself under memory pressure. In an era of the available of cheap gibibyte heaps, potentially retaining references to the bytes of artifacts accessed via CL:LOAD will be a viable strategy for at least the core implementation which is only ~ 11 mebibyte. ABCL-2.0.0 will use all of the ABCL-1.8.0 code in additional to an implementation of multi-threaded sensitive atomic operations supporting ubiquitous contemporary ANSI CL compare and swap for memory. The presence of a new compiler for ABCL-2.0.0 is no longer a release driver as it currently stands, but I would love some help if it shows up. Once I get things working well under openjdk8, I will start to work on the new compiler infrastructure. Comments, patches, criticisms welcome! yours, Mark [1]: <https://github.com/armedbear/abcl/blob/05e4b396658d7ba4bd074a8e4f627deec3b7…> [2]: <https://abcl.org/trac/milestone/1.8.0> [3]: <https://abcl.org/trac/milestone/2.0.0> [4]: <https://gitlab.common-lisp.net/abcl/abcl/-/blob/master/contrib/abcl-asdf/RE…> [5]: <https://gitlab.common-lisp.net/abcl/abcl/-/commit/0ba7dea278a474efe0a8e7550…> [6]: <https://travis-ci.org/github/armedbear/abcl/jobs/728128870> -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

1 0

Challenge with ABCL on a commercial project: Licensing
by Steven Nunez 07 Sep '20

07 Sep '20

Gents, I nearly had ABCL into an upcoming project, until I checked the license. I should have looked at this before I embarked down this path, but for some reason assumed that since ABCL is a relatively young implementation it was using Apache/MIT/BSD or something similar. Here's the opensource clause from the contract: 1.1 OPEN SOURCE SOFTWARE. To the extent that the Company develops orprovides any software to <Customer> under this Agreement, without <Customer’s> priorwritten consent, that <Customer> may grant or withhold in its sole discretion, nosuch deliverable shall incorporate, link to or call upon any Open SourceSoftware. “Open Source Software” means any software that is generally availableto the public in source code form under (i) licenses substantially similar tothose approved by the Open Source Initiative and listed athttp://www.opensource.org/licenses, or (ii) any copyleft license (including aReciprocal License), or (iii) any other license that contains terms thatrequire, whether or not conditionally, disclosure of any source code. This clause, or one very much like it, has been in every contract for the last 10 years or so, or at least since the BusyBox debacle. Most of the time companies have a list of pre-approved licenses, with MIT/Apache/BSD almost always being on it. Note the specific exclusion "any copyleft license" and any license in any form that might cause disclosure of source code. Reading the ABCL FAQ and web page, it seems the intention is to avoid compelling a company from releasing their source code: The use of Armed Bear Common Lisp (ABCL) is covered by the GNU General Public License with Classpath exception, meaning that you can use ABCL in your application without the requirement to open the sources to your application. However there's a few problem with this: - It has the words "GNU General Public License", which rings loud alarm bells with corporate lawyers (and is disqualified by clauses ii/iii anyway) - Whilst the intention is to allow ABCL to be mixed with proprietary code, the IP equivalent of ambulance chasing lawyers may still see it as an opportunity for profit. It's worth reading the BusyBox saga if you're not familiar with it. - It's never been tested in court The last point is worth elaborating on. I once cornered a friendly corporate lawyer at one of our company drinks functions and showed him the LLGPL and asked what he thought. His response put it all in perspective. He said: "It looks fine to me, but it's never been tested in court. Would you take an open source project and deploy it to production and bet the company's livelihood on it, without testing it first?" I guess I can't blame them really. Would you want stick your neck out for something with potentially catastrophic consequences, when there's plenty of alternatives? Lisp has enough of an uphill climb just on the technology front, this is the nail in the coffin. Questions: - Would the current maintainers of ABCL consider relicensing it under a MIT, BSD or Apache license? - If so, do we have a list of all the contributors? - If so, how many are there and are they contactable?

6 9

Test
by Steven Nunez 08 Aug '20

08 Aug '20

Testing. I sent out a message last night but haven't seen it yet; neither in my mailbox nor the archives.

1 0

Am I at a dead-end?
by Steven Nunez 07 Aug '20

07 Aug '20

I've been looking at a few other ways to accomplish wrapping Spark, and none look good. I could generate Java functions, which seem to be inherently serialisable, but that defeats the point of using ABCL. I also considered an approach using Java to call into ABCL, but then I've got to ensure that the ABCL functions are distributed to each node, defeating the point of using Spark. @Alessio Stalla, your most recent github commit suggests that things might have moved along to the point that function and closure serialisation is working. Are things stable enough to give it a try? Not this week, but perhaps on Monday? I'm happy to debug anything that I turn up. There's a new project on the horizon and I'd like to use the Java compatibility angle to try and get Lisp into it. If I'm lucky, I may not even have to mention lisp, depending on how/if the contract terms work out.

2 3

Regarding Steps Towards ABCL STEP Functionality
by loveleague 03 Aug '20

03 Aug '20

Hello, While reading through the release manual, I came across the realization -- from section 1.1.2 -- that ABCL still needs an implementation of the STEP function. I would like to look into the details of the hurdles one might need to overcome to bring STEP functionality to ABCL. I imagine there may be some additional challenges from dealing with the JVM, but I'm not well versed as of yet on the topic of debuggers or JVM internals. If it would be possible to share some details on where one would get started in trying to approach this, that would be appreciated. - Luke

4 3

run-program issues with OpenJDK 11
by Eric Timmons 30 Jul '20

30 Jul '20

It appears that java.lang.UNIXProcess was removed in OpenJDK 9 or thereabouts, assuming I'm interpreting https://github.com/openjdk/jdk/commit/aa6b19f38ee4dc69cac664d1211bff0e69b31… correctly (/definitely/ not a Java developer here...). This seems to make run-program unusable when using prebuilt jars on recent JREs. Using the prebuilt 1.7.0 release and OpenJDK 11, I get the errors shown in prebuilt-openjdk11.txt (attached). If compiling using OpenJDK 11, run-program seems usable, but any attempt to get the PID of a process gives the error in from-source-openjdk11.txt (attached). -Eric VM settings: Max. Heap Size (Estimated): 7.79G Using VM: OpenJDK 64-Bit Server VM Armed Bear Common Lisp 1.7.0 Java 11.0.7 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.198 seconds. Startup completed in 1.107 seconds. Type ":help" for a list of available commands. CL-USER(1): (sys:run-program "ls" nil) Error loading jar:file:/home/abcl/work/abcl/dist/abcl.jar!/org/armedbear/lisp/run-program.abcl at line 278 (offset 19142) #<THREAD "interpreter" {5C1AE0C4}>: Debugger invoked on condition of type ERROR Class not found: java.lang.UNIXProcess Restarts: 0: TOP-LEVEL Return to top level. [1] SYS(2): 0 java.lang.ExceptionInInitializerError at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at java.base/java.lang.Class.newInstance(Class.java:584) at org.armedbear.lisp.FaslClassLoader.loadFunction(FaslClassLoader.java:130) at org.armedbear.lisp.FaslClassLoader$pf_get_fasl_function.execute(FaslClassLoader.java:165) at org.armedbear.lisp.LispThread.execute(LispThread.java:832) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:582) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:577) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.progn(Lisp.java:709) at org.armedbear.lisp.SpecialOperators$sf_progn.execute(SpecialOperators.java:273) at org.armedbear.lisp.Lisp.eval(Lisp.java:530) at org.armedbear.lisp.Load.faslLoadStream(Load.java:667) at org.armedbear.lisp.Load$init_fasl.execute(Load.java:457) at org.armedbear.lisp.LispThread.execute(LispThread.java:832) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:582) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Load.loadStream(Load.java:629) at org.armedbear.lisp.Load.loadFileFromStream(Load.java:597) at org.armedbear.lisp.Load.loadFileFromStream(Load.java:477) at org.armedbear.lisp.Load.loadSystemFile(Load.java:375) at org.armedbear.lisp.Load.loadSystemFile(Load.java:255) at org.armedbear.lisp.Autoload.effectiveLoad(Autoload.java:110) at org.armedbear.lisp.Autoload.load(Autoload.java:147) at org.armedbear.lisp.Lisp.eval(Lisp.java:537) at org.armedbear.lisp.Primitives$pf__eval.execute(Primitives.java:345) at org.armedbear.lisp.LispThread.execute(LispThread.java:814) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:575) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.progn(Lisp.java:709) at org.armedbear.lisp.Primitives$sf_block.execute(Primitives.java:3743) at org.armedbear.lisp.Lisp.eval(Lisp.java:530) at org.armedbear.lisp.Lisp.progn(Lisp.java:709) at org.armedbear.lisp.Closure.execute(Closure.java:220) at org.armedbear.lisp.Closure.execute(Closure.java:148) at org.armedbear.lisp.LispThread.execute(LispThread.java:814) at org.armedbear.lisp.Lisp$1.execute(Lisp.java:285) at org.armedbear.lisp.Symbol.execute(Symbol.java:803) at org.armedbear.lisp.LispThread.execute(LispThread.java:814) at org.armedbear.lisp.top_level_47.execute(top-level.lisp:407) at org.armedbear.lisp.CompiledClosure.execute(CompiledClosure.java:89) at org.armedbear.lisp.Symbol.execute(Symbol.java:793) at org.armedbear.lisp.LispThread.execute(LispThread.java:798) at org.armedbear.lisp.top_level_48.execute(top-level.lisp:415) at org.armedbear.lisp.LispThread.execute(LispThread.java:798) at org.armedbear.lisp.Interpreter.run(Interpreter.java:361) at org.armedbear.lisp.Main$1.run(Main.java:48) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.armedbear.lisp.Go Error loading jar:file:/home/abcl/work/abcl/dist/abcl.jar!/org/armedbear/lisp/run-program.abcl at line 278 (offset 19142) #<THREAD "interpreter" {5C1AE0C4}>: Debugger invoked on condition of type ERROR Compiled function can't be loaded: org.armedbear.lisp.run_program_60 from org.armedbear.lisp.Pathname@49bd2fd0 Restarts: 0: TOP-LEVEL Return to top level. [1] SYS(3): 0 CL-USER(4): (exit) VM settings: Max. Heap Size (Estimated): 7.79G Using VM: OpenJDK 64-Bit Server VM Armed Bear Common Lisp 1.7.1-dev Java 11.0.7 Oracle Corporation OpenJDK 64-Bit Server VM Low-level initialization completed in 0.205 seconds. Startup completed in 1.224 seconds. Type ":help" for a list of available commands. CL-USER(1): (sys:run-program "ls" nil :wait nil) #S(SYSTEM:PROCESS :JPROCESS #<java.lang.ProcessImpl Process[pid=125, exitValue=0] {1E464845}> :%INPUT #S(SYSTEM::SYSTEM-STREAM) :%OUTPUT #S(SYSTEM::SYSTEM-STREAM) :%ERROR #S(SYSTEM::SYSTEM-STREAM)) CL-USER(2): (sys:process-pid *) #<THREAD "interpreter" {7278E1D6}>: Debugger invoked on condition of type ERROR Class not found: java.lang.UNIXProcess Restarts: 0: TOP-LEVEL Return to top level. [1] CL-USER(3): 0 CL-USER(4): (exit)

3 8