Two test errors in latest slime from github
Hello everyone, This morning I downloaded the latest slime from github. I have put together a simple bash script to automate the process and as part of that script I run the test suit using: make clean check make check-fancy The results of the test were: Ran 112 tests, 110 results as expected, 2 unexpected (2014-02-21 07:20:38-0800) 1 expected failures 2 unexpected results: FAILED autodoc-tests-23 FAILED autodoc-tests-24 I ran the test suite from the command prompt to check if it was my script causing the problem, but I got the same two errors. This is different from the previous download where I ran the test suite from the command prompt using the same two commands and got no unexpected errors. Respectfully, Paul Bowyer
On Fri, Feb 21, 2014 at 3:34 PM, Paul Bowyer <pbowyer@olynet.com> wrote:
2 unexpected results: FAILED autodoc-tests-23 FAILED autodoc-tests-24
I ran the test suite from the command prompt to check if it was my script causing the problem, but I got the same two errors.
I believe this is a known issue, registered in this very spartan issue: <https://github.com/slime/slime/issues/106>. Will look into it soon. Cheers, -- Luís Oliveira http://kerno.org/~luis/
Luís Oliveira <luismbo@gmail.com> writes:
On Fri, Feb 21, 2014 at 3:34 PM, Paul Bowyer <pbowyer@olynet.com> wrote:
2 unexpected results: FAILED autodoc-tests-23 FAILED autodoc-tests-24
I ran the test suite from the command prompt to check if it was my script causing the problem, but I got the same two errors.
I believe this is a known issue, registered in this very spartan issue: <https://github.com/slime/slime/issues/106>.
Paul, The autodoc tests, were, until fairly recently, hidden from the automated test run by a mistake of mine. They were "unhidden" recently and hence that failure shows. Alternatively, it might be that you have updated your SBCL, since those two tests don't fail in the 32bit version, which is the version that Travis CI runs. They also don't fail on some cmucl version. The problem is that autodoc tests are fairly brittle and when the swank-side reports a little more or less info, it thinks it's a failure. Part of Luís's work is determining when that is true and when it isn't. Maybe you can help. A good place to start would be to register in the issue, for each problematic test case, exactly: what is happening across different versions; what the tests believe should be happening; what you think should be happening. João
On 02/21/2014 09:04 AM, João Távora wrote:
Luís Oliveira<luismbo@gmail.com> writes:
On Fri, Feb 21, 2014 at 3:34 PM, Paul Bowyer<pbowyer@olynet.com> wrote:
2 unexpected results: FAILED autodoc-tests-23 FAILED autodoc-tests-24
I ran the test suite from the command prompt to check if it was my script causing the problem, but I got the same two errors. I believe this is a known issue, registered in this very spartan issue:<https://github.com/slime/slime/issues/106>. Paul,
The autodoc tests, were, until fairly recently, hidden from the automated test run by a mistake of mine. They were "unhidden" recently and hence that failure shows.
Alternatively, it might be that you have updated your SBCL, since those two tests don't fail in the 32bit version, which is the version that Travis CI runs. They also don't fail on some cmucl version.
The problem is that autodoc tests are fairly brittle and when the swank-side reports a little more or less info, it thinks it's a failure.
Part of Luís's work is determining when that is true and when it isn't.
Maybe you can help. A good place to start would be to register in the issue, for each problematic test case, exactly:
what is happening across different versions; what the tests believe should be happening; what you think should be happening.
João
Hello João, I've been able to get both the 32-bit and 64-bit versions of CCL running on my system and they load into slime OK, but I've not yet tried to work with any lisp code with them. I tried the following with the 64-bit version of CCL from a command prompt: cd ~/my-slime-sandbox/slime make LISP=ccl clean make LISP=ccl compile make LISP=ccl clean check make LISP=ccl check-fancy But it gets to a point in one of the tests and produces no further output. It may be just very slow, but I waited for a very long time for more output. The point where it stops producing output: <clip> Fontifying *slime-macroexpansion*... (regexps.............) Fontifying *slime-macroexpansion*... (regexps..............) Fontifying *slime-macroexpansion*... (regexps...............) waiting for condition: top-level [21:08:22.373847] Undo! passed 58/184 macroexpand-1 Saving file /tmp/slime-recipe-11858GDD.el... Wrote /tmp/slime-recipe-11858GDD.el passed 59/184 readme-recipe Saving file /tmp/slime-recipe-11858DnF.el... Wrote /tmp/slime-recipe-11858DnF.el passed 60/184 readme-recipe-autoload-on-lisp-visit => 3 (2 bits, #x3, #o3, #b11) passed 61/184 report-condition-with-circular-list-1 waiting for condition: top-level [21:15:46.975566] waiting for condition: top-level [21:15:47.075842] => 3 (2 bits, #x3, #o3, #b11) passed 62/184 report-condition-with-circular-list-2 When I run the tests using SBCL, it is very fast and I have backtrace info and and the cause of the error. Using the following: cd ~/my-slime-sandbox/slime make clean make compile make clean check make check-fancy I got the this partial output. <snip> Test autodoc-tests-23 backtrace: signal(ert-test-failed (((should (equal "(error 'simple-condition &r ert-fail(((should (equal "(error 'simple-condition &rest arguments & #[nil "\306\307!\210\310 \210\311\312!▒q\210\313\216\314\315 \210 byte-code("\306\307\310\311\31D\313FE\314\211\315\211\316H byte-code("\301\302!▒q\210\303\216\304\213\210+\305 \207" [temp-bu ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc byte-code("\306\307!\211▒r\310\311!q\210\312 d\313\223)L\210\314\216 ert-run-test([cl-struct-ert-test autodoc-tests-23 "Check autodoc wor ert-run-or-rerun-test([cl-struct-ert--stats (tag contrib) [[cl-struc ert-run-tests((tag contrib) #[(event-type &rest event-args) \306=\ ert-run-tests-batch((tag contrib)) slime-batch-test((tag contrib)) eval((slime-batch-test (quote (tag contrib)))) command-line-1(("-L" "." "-L" ".." "-L" "test" "--eval" "(require (q command-line() normal-top-level() Test autodoc-tests-23 condition: (ert-test-failed ((should (equal "(error 'simple-condition &rest arguments &key format-arguments format-control)" (slime-autodoc-to-string))) :form (equal "(error 'simple-condition &rest arguments &key format-arguments format-control)" "(error 'simple-condition &rest arguments &key (format-arguments 'nil) (format-control 'nil))") :value nil :explanation (arrays-of-different-length 78 92 "(error 'simple-condition &rest arguments &key format-arguments format-control)" "(error 'simple-condition &rest arguments &key (format-arguments 'nil) (format-control 'nil))" first-mismatch-at 46))) FAILED 16/112 autodoc-tests-23 Test autodoc-tests-24 backtrace: signal(ert-test-failed (((should (equal "(cerror \"Foo\" 'simple-con ert-fail(((should (equal "(cerror \"Foo\" 'simple-condition &rest ar #[nil "\306\307!\210\310 \210\311\312!▒q\210\313\216\314\315 \210 byte-code("\306\307\310\311\31D\313FE\314\211\315\211\316H byte-code("\301\302!▒q\210\303\216\304\213\210+\305 \207" [temp-bu ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc byte-code("\306\307!\211▒r\310\311!q\210\312 d\313\223)L\210\314\216 ert-run-test([cl-struct-ert-test autodoc-tests-24 "Check autodoc wor ert-run-or-rerun-test([cl-struct-ert--stats (tag contrib) [[cl-struc ert-run-tests((tag contrib) #[(event-type &rest event-args) \306=\ ert-run-tests-batch((tag contrib)) slime-batch-test((tag contrib)) eval((slime-batch-test (quote (tag contrib)))) command-line-1(("-L" "." "-L" ".." "-L" "test" "--eval" "(require (q command-line() normal-top-level() Test autodoc-tests-24 condition: (ert-test-failed ((should (equal "(cerror \"Foo\" 'simple-condition &rest arguments &key format-arguments format-control)" (slime-autodoc-to-string))) :form (equal "(cerror \"Foo\" 'simple-condition &rest arguments &key format-arguments format-control)" "(cerror \"Foo\" 'simple-condition &rest arguments &key (format-arguments 'nil) (format-control 'nil))") :value nil :explanation (arrays-of-different-length 85 99 "(cerror \"Foo\" 'simple-condition &rest arguments &key format-arguments format-control)" "(cerror \"Foo\" 'simple-condition &rest arguments &key (format-arguments 'nil) (format-control 'nil))" first-mismatch-at 53))) <snip> I tried again with CCL to run the the tests, but it stopped at the same place and never returned. I finally had to close the konsole window to stop the test. Paul Bowyer
On Sat, Feb 22 2014, Paul Bowyer wrote:
I tried again with CCL to run the the tests, but it stopped at the same place and never returned. I finally had to close the konsole window to stop the test.
CCL doesn't detect cycles for this example: (let ((*print-pretty* t) (*print-circle* t)) (princ-to-string (make-condition 'simple-error :format-control "~a" :format-arguments (list (let ((x (cons nil nil))) (setf (cdr x) x)))))) is that a CCL bug or is this example not covered by the spec? Helmut
On 02/22/2014 01:34 PM, Helmut Eller wrote:
On Sat, Feb 22 2014, Paul Bowyer wrote:
I tried again with CCL to run the the tests, but it stopped at the same place and never returned. I finally had to close the konsole window to stop the test. CCL doesn't detect cycles for this example:
(let ((*print-pretty* t) (*print-circle* t)) (princ-to-string (make-condition 'simple-error :format-control "~a" :format-arguments (list (let ((x (cons nil nil))) (setf (cdr x) x))))))
is that a CCL bug or is this example not covered by the spec?
Helmut
Hello Helmut, I didn't really fully understand this example or the question because I have only just recently begun to look at elisp, which I understand is the language in which the test suite is written. What spec are you referring to? I did find the following in the CCL documentation if that has any bearing: 9.3. Limitations and known bugs Clozure CL and the external process may get confused about who owns which streams when input, output, or error are specified as T and wait is specified as NIL. External processes that need to talk to a terminal device may not work properly; the environment (SLIME, ILISP) under which Clozure CL is run can affect this. Paul Bowyer
On Sun, Feb 23 2014, Paul Bowyer wrote:
CCL doesn't detect cycles for this example:
(let ((*print-pretty* t) (*print-circle* t)) (princ-to-string (make-condition 'simple-error :format-control "~a" :format-arguments (list (let ((x (cons nil nil))) (setf (cdr x) x))))))
is that a CCL bug or is this example not covered by the spec?
Helmut
Hello Helmut,
I didn't really fully understand this example or the question because I have only just recently begun to look at elisp, which I understand is the language in which the test suite is written. What spec are you referring to?
The example is Common Lisp and the spec I meant is the Common Lisp spec. The report-condition-with-circular-list test executes similar code on the Lisp side. If you run the example in CCL without SLIME it will end up in some out-of-memory condition. Obviously CCL doesn't detect the cycle, and the question was whether CCL has a bug (ansi-compliance) given that *print-circle* is t. But it doesn't matter now as SLIME passes this test now. Helmut
participants (4)
-
Helmut Eller -
joaotavora＠gmail.com -
Luís Oliveira -
Paul Bowyer