Hi,
Compiling the loading the code below (a simplified extract of SBCL's codebase, found attempting to crossbuild SBCL with ABCL) results in a VerifyError, possibly related to the implementation of PROGN. Using 1.6.1-dev-svn-15217.
java.lang.VerifyError: (class: org/armedbear/lisp/foo_1, method: execute signature: (Lorg/armedbear/lisp/LispObject;)Lorg/armedbear/lisp/LispObject;) Expecting to find object/array on stack at java.base/java.lang.Class.getDeclaredConstructors0(Native Method) at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137) at java.base/java.lang.Class.getConstructor0(Class.java:3342) at java.base/java.lang.Class.newInstance(Class.java:556) at org.armedbear.lisp.FaslClassLoader.loadFunction(FaslClassLoader.java:130) at org.armedbear.lisp.FaslClassLoader$pf_get_fasl_function.execute(FaslClassLoader.java:165) at org.armedbear.lisp.LispThread.execute(LispThread.java:832) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:582) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:577) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.progn(Lisp.java:709) at org.armedbear.lisp.SpecialOperators$sf_progn.execute(SpecialOperators.java:273) at org.armedbear.lisp.Lisp.eval(Lisp.java:530) at org.armedbear.lisp.Load.faslLoadStream(Load.java:667) at org.armedbear.lisp.Load$init_fasl.execute(Load.java:457)
--- (in-package :cl-user)
(progn (defvar*sxhash-crosscheck* nil) (defun sxhash (x) (let ((answer (if (string= x "NIL") (ash 1343225879 (- 1))))) (push (cons x answer)*sxhash-crosscheck*) answer)))
-- Eric Marsden https://risk-engineering.org/
On Dec 14, 2019, at 10:43, Eric Marsden eric.marsden@free.fr wrote:
Hi,
Compiling the loading the code below (a simplified extract of SBCL's codebase, found attempting to crossbuild SBCL with ABCL) results in a VerifyError, possibly related to the implementation of PROGN. Using 1.6.1-dev-svn-15217.
Thanks for the report. I agree that it looks like a problem with ABCL.
Way back in 2009 while working towards releasing ABCL-1.0.0 as the first ANSI conformant release of the Bear, fixing ABCL to host a build of SBCL was quite helpful in shaking out errors in ABCL. And it also was helpful in finding problems with SBCL as sometimes the fault lay in their stars.
In attending [the SBCL workshop last week][sbcl20], I intended to see what the Bear’s current status was in hosting a contemporary SBCL build.
[sbcl20]: http://www.sbcl.org/sbcl20/
(Un)fortunately, I spent most of my time at SBCL20 learning about what others were doing in Common Lisp, especially taking advantage of the proximity of Zach (Quicklisp), Luis (SLIME), and Christophe (SBCL) to frame larger questions of the open CL implementation that ABCL finds itself within as we go into 2020.
Consequently, I never got beyond the new (to me) SBCL `make.sh` facade failing to create a configuration.
I’m stuck at trying to host SBCL with ABCL via
bash -x make.sh --xc-host="$HOME/work/abcl/abcl —batch”
ends in the lack of SBCL features being generated:
gmake: Entering directory '/usr/home/mevenson/work/sbcl/src/runtime' GNUmakefile:42: genesis/Makefile.features: No such file or directory gmake: *** No rule to make target 'genesis/Makefile.features'. Stop. gmake: Leaving directory '/usr/home/mevenson/work/sbcl/src/runtime’
I’ve started to poke around in the build system by manually invoking each step, but haven’t found out why an ABCL-hosted compile is not generating the default SBCL features. Mebbe the interwebs can help a Bear our here, or I’ll eventually figure it out for myself.
On Sun, Dec 15, 2019 at 10:49 AM Mark Evenson evenson@panix.com wrote:
Mebbe the interwebs can help a Bear our here, or I’ll eventually figure it out for myself.
Well, Eric already did help, identifying the offending form.
On Dec 15, 2019, at 12:52, Stas Boukarev stassats@gmail.com wrote:
On Sun, Dec 15, 2019 at 10:49 AM Mark Evenson evenson@panix.com wrote:
Mebbe the interwebs can help a Bear our here, or I’ll eventually figure it out for myself.
Well, Eric already did help, identifying the offending form.
Agreed.
I was pointing out my inability to figure out how to undertake an ABCL-hosted SBCL build, but again this is entirely within my ability when I find the time.
BTW, this error also prevents hyperluminal-mem library from loading: https://common-lisp.net/project/cl-test-grid/abcl/abcl-diff30.html
16.12.2019, 13:03, "Mark Evenson" evenson@panix.com:
On Dec 15, 2019, at 12:52, Stas Boukarev stassats@gmail.com wrote:
On Sun, Dec 15, 2019 at 10:49 AM Mark Evenson evenson@panix.com wrote:
Mebbe the interwebs can help a Bear our here, or I’ll eventually figure it out for myself.
Well, Eric already did help, identifying the offending form.
Agreed.
I was pointing out my inability to figure out how to undertake an ABCL-hosted SBCL build, but again this is entirely within my ability when I find the time.
-- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."
Thank you for sending in this form -- I apologize - I am certain that this was introduced by my patch earlier this year to fix a couple of stack inconsistency bugs[0] -- a result of my modification to the p2-ash function.
An minimal form that reproduces this bug: (let ((fn (compile nil '(lambda () (ash 1343225879 (- 1)))))) (funcall fn))
In this particular case, (ash 1343225879 (- 1)) should compile to a constant, but doesn't, and the particular Java verification error stems from trying to load and store the -1 constant using aload/astore* instead of iload*. I have a local fix for this, and am also generating test cases with different types specified to exercise all possible code paths in p2-ash (such as): (the fixnum (ash (the fixnum x) 2))
Sorry for the trouble! And again, thanks for sending this in, especially something so small and reproducible.
I hope to be able to contribute more to ABCL this year if it makes sense, but at the very least can help fix this problem I introduced, I should be able to send out a patch tonight or at least very soon,
-Mark
[0] https://mailman.common-lisp.net/pipermail/armedbear-devel/2019-May/003977.ht...
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, December 14, 2019 9:43 AM, Eric Marsden eric.marsden@free.fr wrote:
Hi,
Compiling the loading the code below (a simplified extract of SBCL's codebase, found attempting to crossbuild SBCL with ABCL) results in a VerifyError, possibly related to the implementation of PROGN. Using 1.6.1-dev-svn-15217.
java.lang.VerifyError: (class: org/armedbear/lisp/foo_1, method: execute signature: (Lorg/armedbear/lisp/LispObject;)Lorg/armedbear/lisp/LispObject;) Expecting to find object/array on stack at java.base/java.lang.Class.getDeclaredConstructors0(Native Method) at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137) at java.base/java.lang.Class.getConstructor0(Class.java:3342) at java.base/java.lang.Class.newInstance(Class.java:556) at org.armedbear.lisp.FaslClassLoader.loadFunction(FaslClassLoader.java:130) at org.armedbear.lisp.FaslClassLoader$pf_get_fasl_function.execute(FaslClassLoader.java:165) at org.armedbear.lisp.LispThread.execute(LispThread.java:832) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:582) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:577) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.progn(Lisp.java:709) at org.armedbear.lisp.SpecialOperators$sf_progn.execute(SpecialOperators.java:273) at org.armedbear.lisp.Lisp.eval(Lisp.java:530) at org.armedbear.lisp.Load.faslLoadStream(Load.java:667) at org.armedbear.lisp.Load$init_fasl.execute(Load.java:457)
(in-package :cl-user)
(progn (defvarsxhash-crosscheck nil) (defun sxhash (x) (let ((answer (if (string= x "NIL") (ash 1343225879 (- 1))))) (push (cons x answer)sxhash-crosscheck) answer)))
-- Eric Marsden https://risk-engineering.org/
Patch attached that should fix this issue. Root cause: when I added the with-operand-accumulation forms for opstack safety (to fix the stack inconsistency bug reported in #69 [0]), I had incorrect types specified in some of the p2-ash code paths. The form (ash 1343225879 (- 1)) also was not being compiled as a constant, and took a code path to actually calculate the shift based on the -1 constant. In the patch attached I:
- Added a constant folding case if arg2 was detected as a constant (but wasn't a literal integer, such as (- 1) ) - Fixed all with-operand-accumulation forms to have the correct types - Added test cases that exercise all code paths of p2-ash
Patch is applied againt: https://github.com/armedbear/abcl Latest master (3a54c91f3a0cbb71ae8d161acadecf118a65404c).
Please let me know if there is any more troubles,
-Mark
[0] https://github.com/armedbear/abcl/issues/69
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Sunday, December 15, 2019 6:12 PM, somewhat-functional-programmer somewhat-functional-programmer@protonmail.com wrote:
Thank you for sending in this form -- I apologize - I am certain that this was introduced by my patch earlier this year to fix a couple of stack inconsistency bugs[0] -- a result of my modification to the p2-ash function.
An minimal form that reproduces this bug: (let ((fn (compile nil '(lambda () (ash 1343225879 (- 1)))))) (funcall fn))
In this particular case, (ash 1343225879 (- 1)) should compile to a constant, but doesn't, and the particular Java verification error stems from trying to load and store the -1 constant using aload/astore* instead of iload*. I have a local fix for this, and am also generating test cases with different types specified to exercise all possible code paths in p2-ash (such as): (the fixnum (ash (the fixnum x) 2))
Sorry for the trouble! And again, thanks for sending this in, especially something so small and reproducible.
I hope to be able to contribute more to ABCL this year if it makes sense, but at the very least can help fix this problem I introduced, I should be able to send out a patch tonight or at least very soon,
-Mark
[0] https://mailman.common-lisp.net/pipermail/armedbear-devel/2019-May/003977.ht...
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Saturday, December 14, 2019 9:43 AM, Eric Marsden eric.marsden@free.fr wrote:
Hi, Compiling the loading the code below (a simplified extract of SBCL's codebase, found attempting to crossbuild SBCL with ABCL) results in a VerifyError, possibly related to the implementation of PROGN. Using 1.6.1-dev-svn-15217. java.lang.VerifyError: (class: org/armedbear/lisp/foo_1, method: execute signature: (Lorg/armedbear/lisp/LispObject;)Lorg/armedbear/lisp/LispObject;) Expecting to find object/array on stack at java.base/java.lang.Class.getDeclaredConstructors0(Native Method) at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137) at java.base/java.lang.Class.getConstructor0(Class.java:3342) at java.base/java.lang.Class.newInstance(Class.java:556) at org.armedbear.lisp.FaslClassLoader.loadFunction(FaslClassLoader.java:130) at org.armedbear.lisp.FaslClassLoader$pf_get_fasl_function.execute(FaslClassLoader.java:165) at org.armedbear.lisp.LispThread.execute(LispThread.java:832) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:582) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.evalCall(Lisp.java:577) at org.armedbear.lisp.Lisp.eval(Lisp.java:540) at org.armedbear.lisp.Lisp.progn(Lisp.java:709) at org.armedbear.lisp.SpecialOperators$sf_progn.execute(SpecialOperators.java:273) at org.armedbear.lisp.Lisp.eval(Lisp.java:530) at org.armedbear.lisp.Load.faslLoadStream(Load.java:667) at org.armedbear.lisp.Load$init_fasl.execute(Load.java:457)
(in-package :cl-user) (progn (defvarsxhash-crosscheck nil) (defun sxhash (x) (let ((answer (if (string= x "NIL") (ash 1343225879 (- 1))))) (push (cons x answer)sxhash-crosscheck) answer))) -- Eric Marsden https://risk-engineering.org/
On 18/12/2019 02:17, somewhat-functional-programmer wrote:
Patch attached that should fix this issue. Root cause: when I added the with-operand-accumulation forms for opstack safety (to fix the stack inconsistency bug reported in #69 [0]), I had incorrect types specified in some of the p2-ash code paths. The form (ash 1343225879 (- 1)) also was not being compiled as a constant, and took a code path to actually calculate the shift based on the -1 constant.
Thanks, I can confirm that this fixes the problem, and in fact with this patch ABCL can build current SBCL (tested on Linux/AMD64).
Eric
Glad to hear it, I appreciate you trying it again. I'll try to go through the building of SBCL myself too -- it's really a blessing to have these types of existing resources already available to help test (the ansi-test suite is also fantastic).
Again, sorry for the trouble,
-Mark
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, December 18, 2019 2:00 PM, Eric Marsden eric.marsden@free.fr wrote:
On 18/12/2019 02:17, somewhat-functional-programmer wrote:
Patch attached that should fix this issue. Root cause: when I added the with-operand-accumulation forms for opstack safety (to fix the stack inconsistency bug reported in #69 [0]), I had incorrect types specified in some of the p2-ash code paths. The form (ash 1343225879 (- 1)) also was not being compiled as a constant, and took a code path to actually calculate the shift based on the -1 constant.
Thanks, I can confirm that this fixes the problem, and in fact with this patch ABCL can build current SBCL (tested on Linux/AMD64).
Eric
On Dec 19, 2019, at 00:29, somewhat-functional-programmer somewhat-functional-programmer@protonmail.com wrote:
Glad to hear it, I appreciate you trying it again. I'll try to go through the building of SBCL myself too -- it's really a blessing to have these types of existing resources already available to help test (the ansi-test suite is also fantastic).
Somehow, I missed receiving Mark’s patch for yesterday for reasons that I can’t explain.
Fortunately, I did receive Eric’s confirmation that Mark’s patch works for him and was able to retrieve the patch from the mailing list archives.
[I have now applied the patch to ABCL][1].
This will be part of abcl-1.6.1, which I intend to release over the holidays.
Eric: Thanks again for the concise test case from hosting the SBCL compilation via ABCL.
Mark: Thank you for the fix, and especially the use of PROVE tests.
[1]: https://github.com/armedbear/abcl/commit/bffb8433987453b9574519b80a7fa66090a...
Thanks! BTW I built SBCL (latest github master) today too using the following command (Linux,X86_64, sbcl clone directory):
sh make.sh --xc-host='<path-to-abcl>/abcl --noinit'
-Mark
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, December 19, 2019 11:19 AM, Mark Evenson evenson@panix.com wrote:
On Dec 19, 2019, at 00:29, somewhat-functional-programmer somewhat-functional-programmer@protonmail.com wrote: Glad to hear it, I appreciate you trying it again. I'll try to go through the building of SBCL myself too -- it's really a blessing to have these types of existing resources already available to help test (the ansi-test suite is also fantastic).
Somehow, I missed receiving Mark’s patch for yesterday for reasons that I can’t explain.
Fortunately, I did receive Eric’s confirmation that Mark’s patch works for him and was able to retrieve the patch from the mailing list archives.
[I have now applied the patch to ABCL][1].
This will be part of abcl-1.6.1, which I intend to release over the holidays.
Eric: Thanks again for the concise test case from hosting the SBCL compilation via ABCL.
Mark: Thank you for the fix, and especially the use of PROVE tests.
armedbear-devel@common-lisp.net