Just pulled from git this afternoon and reran the tests on ccl clisp sbcl ecl abcl allegro allegromodern
* ECL 12.12.1 fails tests test-multiple.script test-run-program.script
test-multiple:
TEST ABORTED: Subprocess run with command "/bin/ln -sf /Users/rpg/lisp/asdf/test/test-multiple.asd /Users/rpg/lisp/asdf/build/test-multiple-too.asd 2>&1" exited with error code 1
No backtrace for this.
TEST ABORTED: Subprocess run with command "echo ok 1" exited with error code 1
For test-run-program I see:
exec: Too few arguments. TEST ABORTED: Subprocess run with command "echo ok 1" exited with error code 1
* ABCL 1.1.0.2 failed test-encodings: TEST ABORTED: These two expressions fail comparison with EQUAL: (STRING-CHAR-CODES "λ") evaluates to (338 170) (EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Above backtrace due to this condition: These two expressions fail comparison with EQUAL: (ASDF-TEST::STRING-CHAR-CODES "λ") evaluates to (338 170) (ASDF-TEST::EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Script failed
* Allegro Modern 8.2 failed test-utilities with this error:
TEST ABORTED: Package "ASDF/ACTION" not found. [file position = 3334]
The problem is the defparameter for *ok-symbols*, which gives the symbol names in all caps. I am attaching a patch for this.
Best, r
On Sun, Feb 24, 2013 at 5:16 PM, Robert Goldman rpgoldman@sift.info wrote:
Just pulled from git this afternoon and reran the tests on ccl clisp sbcl ecl abcl allegro allegromodern
- ECL 12.12.1 fails tests test-multiple.script test-run-program.script
test-multiple:
TEST ABORTED: Subprocess run with command "/bin/ln -sf /Users/rpg/lisp/asdf/test/test-multiple.asd /Users/rpg/lisp/asdf/build/test-multiple-too.asd 2>&1" exited with error code 1
No backtrace for this.
TEST ABORTED: Subprocess run with command "echo ok 1" exited with error code 1
For test-run-program I see:
exec: Too few arguments. TEST ABORTED: Subprocess run with command "echo ok 1" exited with error code 1
OK, these are all instances of problems with run-program. Is it run-program itself failing, or something wrong with programs missing from the $PATH ?
Works for me on ECL 12.12.1 on Linux.
- ABCL 1.1.0.2 failed test-encodings:
TEST ABORTED: These two expressions fail comparison with EQUAL: (STRING-CHAR-CODES "λ") evaluates to (338 170) (EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Above backtrace due to this condition: These two expressions fail comparison with EQUAL: (ASDF-TEST::STRING-CHAR-CODES "λ") evaluates to (338 170) (ASDF-TEST::EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Script failed
Weird. Looks like the UTF-8-encoded characters were read as MacRoman. Within which sub-test does that occur? A bit more context above would help.
- Allegro Modern 8.2 failed test-utilities with this error:
TEST ABORTED: Package "ASDF/ACTION" not found. [file position = 3334]
The problem is the defparameter for *ok-symbols*, which gives the symbol names in all caps. I am attaching a patch for this.
Oops. You know, you have commit rights, and should readily push quite uncontroversial patches like this.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The curious task of economics is to demonstrate to men how little they really know about what they imagine they can design. — F.A. Hayek
On 2/24/13 Feb 24 -5:16 PM, Faré wrote:
On Sun, Feb 24, 2013 at 5:16 PM, Robert Goldman rpgoldman@sift.info wrote:
Just pulled from git this afternoon and reran the tests on ccl clisp sbcl ecl abcl allegro allegromodern
- ECL 12.12.1 fails tests test-multiple.script test-run-program.script
test-multiple:
TEST ABORTED: Subprocess run with command "/bin/ln -sf /Users/rpg/lisp/asdf/test/test-multiple.asd /Users/rpg/lisp/asdf/build/test-multiple-too.asd 2>&1" exited with error code 1
No backtrace for this.
I believe the problem is that EXT:SYSTEM on ECL runs whatever shell is the value of SHELL, but your test code is all written for bash, without checking for it.
I use the T shell.
Probably a good thing that I am testing with this unusual configuration!
I suppose the test code should force SHELL to be bash. I don't know if there's a portable way to do this. ASDF used to invoke /bin/sh, in RUN-SHELL-COMMAND, IIRC.
- ABCL 1.1.0.2 failed test-encodings:
TEST ABORTED: These two expressions fail comparison with EQUAL: (STRING-CHAR-CODES "λ") evaluates to (338 170) (EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Above backtrace due to this condition: These two expressions fail comparison with EQUAL: (ASDF-TEST::STRING-CHAR-CODES "λ") evaluates to (338 170) (ASDF-TEST::EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Script failed
Weird. Looks like the UTF-8-encoded characters were read as MacRoman. Within which sub-test does that occur? A bit more context above would help.
This is the one that failed:
(with-encoding-test (explicit-u8 :utf-8) (def-test-system :test-encoding-explicit-u8 :components ((:file "lambda" :encoding :utf-8))))
Note that to make this work, I added a dynamic variable that bound asdf-test::*test-name* and made the assert-compare-helper incorporate that name in its output.
The dynamic variable was a little yucky, but otherwise I would have had to push the name (as an optional) through the arglist of all of the comparison macros....
This might be useful for debugging, but the implementation is super nasty.
- Allegro Modern 8.2 failed test-utilities with this error:
TEST ABORTED: Package "ASDF/ACTION" not found. [file position = 3334]
The problem is the defparameter for *ok-symbols*, which gives the symbol names in all caps. I am attaching a patch for this.
Oops. You know, you have commit rights, and should readily push quite uncontroversial patches like this.
I haven't been attending as much as I should, so I wasn't sure how this interacted with the versioning (didn't seem worth a patch level bump, but I wasn't certain).
r
On Sun, Feb 24, 2013 at 7:14 PM, Robert Goldman rpgoldman@sift.info wrote:
TEST ABORTED: Subprocess run with command "/bin/ln -sf /Users/rpg/lisp/asdf/test/test-multiple.asd /Users/rpg/lisp/asdf/build/test-multiple-too.asd 2>&1" exited with error code 1
No backtrace for this.
I believe the problem is that EXT:SYSTEM on ECL runs whatever shell is the value of SHELL, but your test code is all written for bash, without checking for it.
I use the T shell.
Probably a good thing that I am testing with this unusual configuration!
I suppose the test code should force SHELL to be bash. I don't know if there's a portable way to do this. ASDF used to invoke /bin/sh, in RUN-SHELL-COMMAND, IIRC.
It is possible that there is a bug in asdf/driver:run-program, or maybe one in ECL itself. Can you run again while tracing as follows? (trace asdf/driver:run-program ext:run-program ext:shell)
I think the test-multiple might pass if you the /bin/ in /bin/ln.
Is the "T shell" the same thing as tcsh? Let me try it... Indeed, I can reproduce the failure with SHELL=tcsh make t l=ecl t='test-run-program.script even though this works: SHELL=tcsh make t l=ccl t='test-run-program.script That's because somehow ECL, to get the exit value of a process, reverts to a more primitive call to system, and then relies on "exec > tmpfile ; command" for redirection, independently from the content of command. It looks like this is not portable to tcsh; and probably not to Windows, either. I don't know how to solve this problem portably or to automatically detect a workaround.
Maybe the test script runner should just export SHELL=/bin/sh ?
- ABCL 1.1.0.2 failed test-encodings: [...]
Weird. Looks like the UTF-8-encoded characters were read as MacRoman. Within which sub-test does that occur? A bit more context above would help.
This is the one that failed:
(with-encoding-test (explicit-u8 :utf-8) (def-test-system :test-encoding-explicit-u8 :components ((:file "lambda" :encoding :utf-8))))
Note that to make this work, I added a dynamic variable that bound asdf-test::*test-name* and made the assert-compare-helper incorporate that name in its output.
The dynamic variable was a little yucky, but otherwise I would have had to push the name (as an optional) through the arglist of all of the comparison macros....
This might be useful for debugging, but the implementation is super nasty.
I'm not sure what you changed to "make it work". If explicit request for utf-8 encoding doesn't work, something is probably broken in either asdf and/or abcl.
- Allegro Modern 8.2 failed test-utilities with this error:
TEST ABORTED: Package "ASDF/ACTION" not found. [file position = 3334]
The problem is the defparameter for *ok-symbols*, which gives the symbol names in all caps. I am attaching a patch for this.
Oops. You know, you have commit rights, and should readily push quite uncontroversial patches like this.
I haven't been attending as much as I should, so I wasn't sure how this interacted with the versioning (didn't seem worth a patch level bump, but I wasn't certain).
My principle is I only bump the version when there's a change to the code itself (i.e. asdf.lisp in this case). all the rest I consider ancillary.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "There could hardly be a more unbearable — and more irrational — world than one in which the most eminent specialists in each field were allowed to proceed unchecked with the realization of their ideals" — Friedrich A. Hayek
On 2/24/13 Feb 24 -7:05 PM, Faré wrote:
On Sun, Feb 24, 2013 at 7:14 PM, Robert Goldman rpgoldman@sift.info wrote:
TEST ABORTED: Subprocess run with command "/bin/ln -sf /Users/rpg/lisp/asdf/test/test-multiple.asd /Users/rpg/lisp/asdf/build/test-multiple-too.asd 2>&1" exited with error code 1
No backtrace for this.
I believe the problem is that EXT:SYSTEM on ECL runs whatever shell is the value of SHELL, but your test code is all written for bash, without checking for it.
I use the T shell.
Probably a good thing that I am testing with this unusual configuration!
I suppose the test code should force SHELL to be bash. I don't know if there's a portable way to do this. ASDF used to invoke /bin/sh, in RUN-SHELL-COMMAND, IIRC.
It is possible that there is a bug in asdf/driver:run-program, or maybe one in ECL itself. Can you run again while tracing as follows? (trace asdf/driver:run-program ext:run-program ext:shell)
I think the test-multiple might pass if you the /bin/ in /bin/ln.
Is the "T shell" the same thing as tcsh? Let me try it... Indeed, I can reproduce the failure with SHELL=tcsh make t l=ecl t='test-run-program.script even though this works: SHELL=tcsh make t l=ccl t='test-run-program.script That's because somehow ECL, to get the exit value of a process, reverts to a more primitive call to system, and then relies on "exec > tmpfile ; command" for redirection, independently from the content of command. It looks like this is not portable to tcsh; and probably not to Windows, either. I don't know how to solve this problem portably or to automatically detect a workaround.
Maybe the test script runner should just export SHELL=/bin/sh ?
That sounds right for the short term.
I don't claim to have fully grokked ASDF:RUN-PROGRAM yet. Would it be reasonable to modify it so that when :force-shell is T (a) it prefers /bin/sh to alternatives, or CMD.EXE on windows? and (b) it allows the user to override and request her shell to be used?
The reason I suggest this is that it would provide a consistent way to encapsulate the behavior we see from ECL. We default to a standard shell for the maximum in portability, but we allow the programmer to demand his shell under some circumstances?
I have given this exactly 2.4 seconds of thought, so this might be A Very Stupid Idea.
I was just looking for a portable way to keep ecl's behavior and other systems behaviors consistent when viewed through ASDF:RUN-PROGRAM.
- ABCL 1.1.0.2 failed test-encodings: [...]
Weird. Looks like the UTF-8-encoded characters were read as MacRoman. Within which sub-test does that occur? A bit more context above would help.
This is the one that failed:
(with-encoding-test (explicit-u8 :utf-8) (def-test-system :test-encoding-explicit-u8 :components ((:file "lambda" :encoding :utf-8))))
Note that to make this work, I added a dynamic variable that bound asdf-test::*test-name* and made the assert-compare-helper incorporate that name in its output.
The dynamic variable was a little yucky, but otherwise I would have had to push the name (as an optional) through the arglist of all of the comparison macros....
This might be useful for debugging, but the implementation is super nasty.
I'm not sure what you changed to "make it work". If explicit request for utf-8 encoding doesn't work, something is probably broken in either asdf and/or abcl.
Sorry -- "make it work" just meant "make it so that I could determine which test failed," not "make the encoding behave correctly."
best, r
Maybe the test script runner should just export SHELL=/bin/sh ?
That sounds right for the short term.
I don't claim to have fully grokked ASDF:RUN-PROGRAM yet. Would it be reasonable to modify it so that when :force-shell is T (a) it prefers /bin/sh to alternatives, or CMD.EXE on windows? and (b) it allows the user to override and request her shell to be used?
I thought it already did that by using the system() libc call, the manpage which, on Linux at least, promises to use /bin/sh, not $SHELL. But looking at ecl/src/lsp/process.lsp I see it uses $SHELL. Probably a mistake. It makes the function mostly unusable to the programmer. Whereas if sh were guaranteed, it would be trivial to execute $SHELL in the rare cases that it is desired (who needs a shell escape when you have a REPL?)
Also, the manual page for run-program doesn't explain the return values. And has at least one typo "vaule" for value: http://ecls.sourceforge.net/new-manual/rn01re63.html
Sorry -- "make it work" just meant "make it so that I could determine which test failed," not "make the encoding behave correctly."
Uh, doesn't this test print the test name just before, in the logs? As to making it work -- this smells like an ABCL bug. Any ABCL hacker listening?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org So from the depths of its enchantment, Terra was able to calculate a course of action. Here at last was an opportunity to consort with Dirbanu on a friendly basis — great Durbanu which, since it had force fields which Earth could not duplicate, must of necessity have many other things Earth could use; mighty Durbanu before whom we would kneel in supplication (with purely- for-defense bombs hidden in our pockets) with lowered heads (making invisible the knife in our teeth) and ask for crumbs from their table (in order to extrapolate the location of their kitchens). — Theodore Sturgeon, "The World Well Lost"
On 2/24/13 Feb 24 -11:16 PM, Faré wrote: ....
re: test-encodings
Sorry -- "make it work" just meant "make it so that I could determine which test failed," not "make the encoding behave correctly."
Uh, doesn't this test print the test name just before, in the logs?
It prints the test name, but my stuff adds a sub-test name. There were, IIRC, three places where we check UTF-8 encoding of the lambda character.
I made them distinguishable, but in a kludgy way. See attached patch. Could tidy it up, but possibly not worth it. LMK what you think.
Cheers, r
On Mon, Feb 25, 2013 at 10:20 AM, Robert Goldman rpgoldman@sift.info wrote:
On 2/24/13 Feb 24 -11:16 PM, Faré wrote: ....
re: test-encodings
Uh, doesn't this test print the test name just before, in the logs?
It prints the test name, but my stuff adds a sub-test name. There were, IIRC, three places where we check UTF-8 encoding of the lambda character.
I made them distinguishable, but in a kludgy way. See attached patch. Could tidy it up, but possibly not worth it. LMK what you think.
Wait, in with-encoding-test, there is a (format t "~&Test ~A: should be ~A~%" ',sys ',encoding) that ought to tell you which subtest it is already.
Are you providing more information?
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org On-line, adj.: The idea that a human being should always be accessible to a computer.
On 2/25/13 Feb 25 -9:26 AM, Faré wrote:
On Mon, Feb 25, 2013 at 10:20 AM, Robert Goldman rpgoldman@sift.info wrote:
On 2/24/13 Feb 24 -11:16 PM, Faré wrote: ....
re: test-encodings
Uh, doesn't this test print the test name just before, in the logs?
It prints the test name, but my stuff adds a sub-test name. There were, IIRC, three places where we check UTF-8 encoding of the lambda character.
I made them distinguishable, but in a kludgy way. See attached patch. Could tidy it up, but possibly not worth it. LMK what you think.
Wait, in with-encoding-test, there is a (format t "~&Test ~A: should be ~A~%" ',sys ',encoding) that ought to tell you which subtest it is already.
Are you providing more information?
Possibly not.
But when I look for "should be" in my test output file, it is many lines away from the failure message.
See attached transcript.... There's a pile of garbage about jar file loading between the output of that format line and the message about the test failing. That format output is 24 lines away from:
TEST ABORTED: On test Encoding test EXPLICIT-U8. These two expressions fail comparison with EQUAL: (STRING-CHAR-CODES "λ") evaluates to (338 170) (EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955)
cheers, r
But when I look for "should be" in my test output file, it is many lines away from the failure message.
That's autoloading for you.
Can you trace LOAD* COMPILE-FILE* and/or COMPILE-FILE LOAD and check that indeed :EXTERNAL-FORMAT :UTF-8 is used?
If it isn't, then it's a bug in our code or test (Note: considering the portable nature of the code, and the fact that it works for me, that is unlikely.)
If it is, then it's time to file a bug against ABCL. (Note: considering that I already had to disable most of the test for ABCL, that is very likely.)
rlwrap abcl (load "test/script-support.lisp") (da) (la) (trace load compile-file load* compile-file*) (load "test/test-encodings.script")
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org No man would listen to you talk if he didn't know it was his turn next. — Edgar Watson Howe
On 2/25/13 Feb 25 -10:34 AM, Faré wrote:
But when I look for "should be" in my test output file, it is many lines away from the failure message.
That's autoloading for you.
Can you trace LOAD* COMPILE-FILE* and/or COMPILE-FILE LOAD and check that indeed :EXTERNAL-FORMAT :UTF-8 is used?
If it isn't, then it's a bug in our code or test (Note: considering the portable nature of the code, and the fact that it works for me, that is unlikely.)
If it is, then it's time to file a bug against ABCL. (Note: considering that I already had to disable most of the test for ABCL, that is very likely.)
rlwrap abcl (load "test/script-support.lisp") (da) (la) (trace load compile-file load* compile-file*) (load "test/test-encodings.script")
Done. I'm afraid the results are Greek to me. Or would be if I could get ABCL to print non-ASCII! ;-)
ASDF-TEST(7): (load "test-encodings.script") 0: (LOAD "test-encodings.script") ; Loading /Users/rpg/lisp/asdf/test/test-encodings.script ... Test TEST-ENCODING-EXPLICIT-U8: should be UTF-8 ; Registering #<ASDF/SYSTEM:SYSTEM "test-encoding-explicit-u8"> 1: (ASDF/LISP-BUILD:LOAD* #P"/Users/rpg/lisp/asdf/test/lambda.lisp" :EXTERNAL-FORMAT :UTF-8) 2: (LOAD #P"/Users/rpg/lisp/asdf/test/lambda.lisp" :EXTERNAL-FORMAT :UTF-8) ; Loading /Users/rpg/lisp/asdf/test/lambda.lisp ... ; Loaded /Users/rpg/lisp/asdf/test/lambda.lisp (0.0020 seconds) 2: LOAD returned T 1: LOAD* returned T Error loading /Users/rpg/lisp/asdf/test/test-encodings.script at line 43 (offset 1675) #<THREAD "interpreter" {9C2DF08}>: Debugger invoked on condition of type SIMPLE-ERROR On sub-test Encoding test EXPLICIT-U8. These two expressions fail comparison with EQUAL: (ASDF-TEST::STRING-CHAR-CODES "λ") evaluates to (338 170) (ASDF-TEST::EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955)
Best, r
On Mon, Feb 25, 2013 at 1:14 AM, Robert Goldman rpgoldman@sift.info wrote:
I believe the problem is that EXT:SYSTEM on ECL runs whatever shell is the value of SHELL, but your test code is all written for bash, without checking for it.
I have changed this. SYSTEM now calls /bin/sh on all POSIX platforms. Thanks for pointing out this problem -- I was not aware of this behavior.
Juanjo
-
Faré -
Juan Jose Garcia-Ripoll -
Robert Goldman