On 2/24/13 Feb 24 -5:16 PM, Faré wrote:

On Sun, Feb 24, 2013 at 5:16 PM, Robert Goldman rpgoldman@sift.info wrote:

...

Just pulled from git this afternoon and reran the tests on ccl clisp sbcl ecl abcl allegro allegromodern

• ECL 12.12.1 fails tests test-multiple.script test-run-program.script

test-multiple:

TEST ABORTED: Subprocess run with command "/bin/ln -sf /Users/rpg/lisp/asdf/test/test-multiple.asd /Users/rpg/lisp/asdf/build/test-multiple-too.asd 2>&1" exited with error code 1

No backtrace for this.

I believe the problem is that EXT:SYSTEM on ECL runs whatever shell is the value of SHELL, but your test code is all written for bash, without checking for it.

I use the T shell.

Probably a good thing that I am testing with this unusual configuration!

I suppose the test code should force SHELL to be bash. I don't know if there's a portable way to do this. ASDF used to invoke /bin/sh, in RUN-SHELL-COMMAND, IIRC.

...

• ABCL 1.1.0.2 failed test-encodings:

TEST ABORTED: These two expressions fail comparison with EQUAL: (STRING-CHAR-CODES "λ") evaluates to (338 170) (EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Above backtrace due to this condition: These two expressions fail comparison with EQUAL: (ASDF-TEST::STRING-CHAR-CODES "λ") evaluates to (338 170) (ASDF-TEST::EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955) Script failed

Weird. Looks like the UTF-8-encoded characters were read as MacRoman. Within which sub-test does that occur? A bit more context above would help.

This is the one that failed:

(with-encoding-test (explicit-u8 :utf-8) (def-test-system :test-encoding-explicit-u8 :components ((:file "lambda" :encoding :utf-8))))

Note that to make this work, I added a dynamic variable that bound asdf-test::*test-name* and made the assert-compare-helper incorporate that name in its output.

The dynamic variable was a little yucky, but otherwise I would have had to push the name (as an optional) through the arglist of all of the comparison macros....

This might be useful for debugging, but the implementation is super nasty.

...

• Allegro Modern 8.2 failed test-utilities with this error:

TEST ABORTED: Package "ASDF/ACTION" not found. [file position = 3334]

The problem is the defparameter for *ok-symbols*, which gives the symbol names in all caps. I am attaching a patch for this.

Oops. You know, you have commit rights, and should readily push quite uncontroversial patches like this.

I haven't been attending as much as I should, so I wasn't sure how this interacted with the versioning (didn't seem worth a patch level bump, but I wasn't certain).

r

On 2/24/13 Feb 24 -7:05 PM, Faré wrote:

On Sun, Feb 24, 2013 at 7:14 PM, Robert Goldman rpgoldman@sift.info wrote:

...

...

...

TEST ABORTED: Subprocess run with command "/bin/ln -sf /Users/rpg/lisp/asdf/test/test-multiple.asd /Users/rpg/lisp/asdf/build/test-multiple-too.asd 2>&1" exited with error code 1

No backtrace for this.

I believe the problem is that EXT:SYSTEM on ECL runs whatever shell is the value of SHELL, but your test code is all written for bash, without checking for it.

I use the T shell.

Probably a good thing that I am testing with this unusual configuration!

I suppose the test code should force SHELL to be bash. I don't know if there's a portable way to do this. ASDF used to invoke /bin/sh, in RUN-SHELL-COMMAND, IIRC.

It is possible that there is a bug in asdf/driver:run-program, or maybe one in ECL itself. Can you run again while tracing as follows? (trace asdf/driver:run-program ext:run-program ext:shell)

I think the test-multiple might pass if you the /bin/ in /bin/ln.

Is the "T shell" the same thing as tcsh? Let me try it... Indeed, I can reproduce the failure with SHELL=tcsh make t l=ecl t='test-run-program.script even though this works: SHELL=tcsh make t l=ccl t='test-run-program.script That's because somehow ECL, to get the exit value of a process, reverts to a more primitive call to system, and then relies on "exec > tmpfile ; command" for redirection, independently from the content of command. It looks like this is not portable to tcsh; and probably not to Windows, either. I don't know how to solve this problem portably or to automatically detect a workaround.

Maybe the test script runner should just export SHELL=/bin/sh ?

That sounds right for the short term.

I don't claim to have fully grokked ASDF:RUN-PROGRAM yet. Would it be reasonable to modify it so that when :force-shell is T (a) it prefers /bin/sh to alternatives, or CMD.EXE on windows? and (b) it allows the user to override and request her shell to be used?

The reason I suggest this is that it would provide a consistent way to encapsulate the behavior we see from ECL. We default to a standard shell for the maximum in portability, but we allow the programmer to demand his shell under some circumstances?

I have given this exactly 2.4 seconds of thought, so this might be A Very Stupid Idea.

I was just looking for a portable way to keep ecl's behavior and other systems behaviors consistent when viewed through ASDF:RUN-PROGRAM.

...

...

...

• ABCL 1.1.0.2 failed test-encodings: [...]

Weird. Looks like the UTF-8-encoded characters were read as MacRoman. Within which sub-test does that occur? A bit more context above would help.

This is the one that failed:

(with-encoding-test (explicit-u8 :utf-8) (def-test-system :test-encoding-explicit-u8 :components ((:file "lambda" :encoding :utf-8))))

Note that to make this work, I added a dynamic variable that bound asdf-test::*test-name* and made the assert-compare-helper incorporate that name in its output.

The dynamic variable was a little yucky, but otherwise I would have had to push the name (as an optional) through the arglist of all of the comparison macros....

This might be useful for debugging, but the implementation is super nasty.

I'm not sure what you changed to "make it work". If explicit request for utf-8 encoding doesn't work, something is probably broken in either asdf and/or abcl.

Sorry -- "make it work" just meant "make it so that I could determine which test failed," not "make the encoding behave correctly."

best, r

On 2/24/13 Feb 24 -11:16 PM, Faré wrote: ....

re: test-encodings

...

Sorry -- "make it work" just meant "make it so that I could determine which test failed," not "make the encoding behave correctly."

Uh, doesn't this test print the test name just before, in the logs?

It prints the test name, but my stuff adds a sub-test name. There were, IIRC, three places where we check UTF-8 encoding of the lambda character.

I made them distinguishable, but in a kludgy way. See attached patch. Could tidy it up, but possibly not worth it. LMK what you think.

Cheers, r

On 2/25/13 Feb 25 -9:26 AM, Faré wrote:

On Mon, Feb 25, 2013 at 10:20 AM, Robert Goldman rpgoldman@sift.info wrote:

...

On 2/24/13 Feb 24 -11:16 PM, Faré wrote: ....

re: test-encodings

...

Uh, doesn't this test print the test name just before, in the logs?

It prints the test name, but my stuff adds a sub-test name. There were, IIRC, three places where we check UTF-8 encoding of the lambda character.

I made them distinguishable, but in a kludgy way. See attached patch. Could tidy it up, but possibly not worth it. LMK what you think.

Wait, in with-encoding-test, there is a (format t "~&Test ~A: should be ~A~%" ',sys ',encoding) that ought to tell you which subtest it is already.

Are you providing more information?

Possibly not.

But when I look for "should be" in my test output file, it is many lines away from the failure message.

See attached transcript.... There's a pile of garbage about jar file loading between the output of that format line and the message about the test failing. That format output is 24 lines away from:

TEST ABORTED: On test Encoding test EXPLICIT-U8. These two expressions fail comparison with EQUAL: (STRING-CHAR-CODES "λ") evaluates to (338 170) (EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955)

cheers, r

On 2/25/13 Feb 25 -10:34 AM, Faré wrote:

...

But when I look for "should be" in my test output file, it is many lines away from the failure message.

That's autoloading for you.

Can you trace LOAD* COMPILE-FILE* and/or COMPILE-FILE LOAD and check that indeed :EXTERNAL-FORMAT :UTF-8 is used?

If it isn't, then it's a bug in our code or test (Note: considering the portable nature of the code, and the fact that it works for me, that is unlikely.)

If it is, then it's time to file a bug against ABCL. (Note: considering that I already had to disable most of the test for ABCL, that is very likely.)

rlwrap abcl (load "test/script-support.lisp") (da) (la) (trace load compile-file load* compile-file*) (load "test/test-encodings.script")

Done. I'm afraid the results are Greek to me. Or would be if I could get ABCL to print non-ASCII! ;-)

ASDF-TEST(7): (load "test-encodings.script") 0: (LOAD "test-encodings.script") ; Loading /Users/rpg/lisp/asdf/test/test-encodings.script ... Test TEST-ENCODING-EXPLICIT-U8: should be UTF-8 ; Registering #<ASDF/SYSTEM:SYSTEM "test-encoding-explicit-u8"> 1: (ASDF/LISP-BUILD:LOAD* #P"/Users/rpg/lisp/asdf/test/lambda.lisp" :EXTERNAL-FORMAT :UTF-8) 2: (LOAD #P"/Users/rpg/lisp/asdf/test/lambda.lisp" :EXTERNAL-FORMAT :UTF-8) ; Loading /Users/rpg/lisp/asdf/test/lambda.lisp ... ; Loaded /Users/rpg/lisp/asdf/test/lambda.lisp (0.0020 seconds) 2: LOAD returned T 1: LOAD* returned T Error loading /Users/rpg/lisp/asdf/test/test-encodings.script at line 43 (offset 1675) #<THREAD "interpreter" {9C2DF08}>: Debugger invoked on condition of type SIMPLE-ERROR On sub-test Encoding test EXPLICIT-U8. These two expressions fail comparison with EQUAL: (ASDF-TEST::STRING-CHAR-CODES "λ") evaluates to (338 170) (ASDF-TEST::EXPECTED-CHAR-CODES (QUOTE :UTF-8)) evaluates to (955)

Best, r