January 2013 - asdf-devel - mailman3.common-lisp.net

[asdf-devel] Upgrade issues
by Faré 21 Jan '13

21 Jan '13

ABCL is one of the three implementations, together with CLISP and CMUCL, where I ultimately punted on ASDF hot-upgrade, by just trying to rename away the ASDF package if it's too old. I don't know if you're interested in making the kind of package surgery I was indulging in work, but it involves rehoming symbols from ASDF to subpackages such as ASDF/COMPONENT, including for class slot names. Anyway, please test the latest ASDF with ABCL, I think it's ripe for release, but I'd like your feedback considering the massive amount of hacking that happened. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If the human mind were simple enough to understand, we'd be too simple to understand it. — Pat Bahn

2 2

[asdf-devel] Just pulled an update and now CCL fails the tests
by Robert Goldman 20 Jan '13

20 Jan '13

I am now getting 8 test failures: Ran 42 tests: 34 passing and 8 failing failing test(s): test-bundle.script test-compile-file-failure.script test-concatenate-source.script test-force.script test-nested-components.script test-package.script test-touch-system-2.script test-xach-update-bug.script Testing: test-bundle.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --batch --eval '(load "test-bundle.script")' ; Registering #<SYSTEM "test-bundle-1"> ; Registering #<SYSTEM "test-bundle-2"> TEST-BUNDLE ASDF-TEST::BUNDLE-1 => #P"/Users/rpg/lisp/asdf/build/fasls/ccl-1.8-f95-macosx-x64/asdf/test/test-bundle-1\\.system.dx64fsl" ASDF-TEST::BUNDLE-2 => #P"/Users/rpg/lisp/asdf/build/fasls/ccl-1.8-f95-macosx-x64/asdf/test/test-bundle-2\\.system.dx64fsl" TEST ABORTED: Failed assertion: (PROBE-FILE BUNDLE-1) Testing: test-compile-file-failure.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --batch --eval '(load "test-compile-file-failure.script")' ; Warning: Warning. ; While executing: #<Anonymous Function #x30200099F98F>, in process listener(1). TEST ABORTED: Failed assertion: (HANDLER-CASE (LET ((*COMPILE-FILE-FAILURE-BEHAVIOUR* :ERROR)) (LOAD-SYSTEM 'TEST-COMPILE-FILE-FAILURE :FORCE T) NIL) (COMPILE-ERROR NIL T)) Testing: test-concatenate-source.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --batch --eval '(load "test-concatenate-source.script")' ; Registering #<SYSTEM "test-concatenate-source"> ; Loading system definition from /Users/rpg/lisp/asdf/test/file3-only.asd into #<Package "ASDF-0"> ; Registering #<SYSTEM "file3-only"> TEST ABORTED: These two expressions fail comparison with EQUAL: (PRINC-TO-STRING (INPUT-FILES MCSO SYS)) evaluates to "NIL" (PRINC-TO-STRING (LOOP :FOR N :IN '(3 1 2) :COLLECT (SUBPATHNAME *TEST-DIRECTORY* (FORMAT NIL "file~D.lisp" N)))) evaluates to "(/Users/rpg/lisp/asdf/test/file3.lisp /Users/rpg/lisp/asdf/test/file1.lisp /Users/rpg/lisp/asdf/test/file2.lisp)" Testing: test-force.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --batch --eval '(load "test-force.script")' TEST ABORTED: Failed assertion: (>= (FILE-WRITE-DATE FILE1) FILE1-DATE) Testing: test-nested-components.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --batch --eval '(load "test-nested-components.script")' ; Registering #<SYSTEM "test-nested-components-a"> TEST ABORTED: Invalid pathname #P"test-nested-components-1.asd": Expected an absolute pathname Testing: test-package.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --batch --eval '(load "test-package.script")' ; Registering #<SYSTEM "test-package"> TEST ABORTED: Invalid relative pathname #P"test-package/" for component ("test-package") Testing: test-touch-system-2.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --batch --eval '(load "test-touch-system-2.script")' ; Registering #<SYSTEM "test1"> TEST ABORTED: Invalid pathname #P"test1.asd": Expected an absolute pathname Now, change registry so foo is found from xach-foo-2/ load foo yet again. It should see the pathname has changed and load it anew TEST ABORTED: No package named "SECOND-VERSION" cheers, r

4 5

[asdf-devel] Tests still failing on allegro
by Robert Goldman 19 Jan '13

19 Jan '13

I checked test-force. This is failing because after the test-force system is loaded, we have this: ASDF-TEST> (asdf:compile-file-pathname* "file1") #P"/Users/rpg/lisp/asdf/build/fasls/acl-8.2-macosx-x64/root/file1.fasl" ASDF-TEST> (probe-file (asdf:compile-file-pathname* "file1")) NIL This despite the fact that the system is loaded: ASDF-TEST> (asdf:operate 'asdf:load-op 'test-force) #<LOAD-OP > ((#<PREPARE-OP > . #<SYSTEM "test-force">) (#<PREPARE-OP > . #<CL-SOURCE-FILE "test-force" "file1">) (#<LOAD-OP > . #<CL-SOURCE-FILE "test-force" "file1">) (#<COMPILE-OP > . #<SYSTEM "test-force">) (#<LOAD-OP > . #<SYSTEM "test-force">)) Looks like this might be the problem: ASDF-TEST> (asdf:output-files (make-instance 'asdf:compile-op) (asdf:find-component "test-force" "file1")) (#P"/Users/rpg/lisp/asdf/build/fasls/acl-8.2-macosx-x64/asdf/test/file1.fasl") See that the file that is generated by compiling "file1" does not have the same pathname as (ASDF:COMPILE-FILE-PATHNAME* "file1") This seems to give the right answer: ASDF-TEST> (asdf:compile-file-pathname* (asdf:component-pathname (asdf:find-component "test-force" "file1"))) #P"/Users/rpg/lisp/asdf/build/fasls/acl-8.2-macosx-x64/asdf/test/file1.fasl" Is this simply a bug in the test? Best, r

1 0

[asdf-devel] deferred warnings in asdf, and more
by Faré 19 Jan '13

19 Jan '13

Here's another one for James Knight: ASDF now correctly (I hope) saves undefined function warnings at compile-time in SBCL, to check them at the end of a system's build. All the low level support is in asdf-driver, and the necessary hooks in asdf-defsystem. This builds upon work by Douglas Katzman I had once again a *lot* of pain with pathnames, and the pathname compatibility file has soared in length to 1157 lines (not counting hundreds of related lines in os support). Go grab asdf 2.26.124 (also available separately as asdf-driver and asdf-defsystem). —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org You can learn many things from children. How much patience you have, for instance. — Franklin P. Jones

1 0

Re: [asdf-devel] fare-utils build failure
by Faré 18 Jan '13

18 Jan '13

Cc'ng asdf-devel. On Wed, Jan 16, 2013 at 9:30 AM, Zach Beane <xach(a)xach.com> wrote: > Faré <fahree(a)gmail.com> writes: > >>> How do I get asdf-driver? >>> >> It's currently part of the asdf git tree. Should I make it separate? >> I think you can already include asdf-driver.asd without including asdf. > > What is the relationship between asdf-driver and asdf? What is a driver? > asdf uses asdf-driver. asdf-driver is currently distributed as part of the asdf source tree, but if that is an issue, there is no barrier to splitting it apart. For bootstrap purposes, the source code of asdf-driver is transcluded into asdf.lisp as a prelude to asdf, so asdf can rely on it. asdf-driver is a general purpose portability and runtime management library. It provides various abstraction layers for packages, pathnames, command-line arguments, compilation, condition-muffling, run-program, image dump and restore with hooks, basic I/O, etc. Basically, it's all the portability work I did over the last few years as part of asdf and xcvb, so as to be able to run user code unchanged (above the abstraction layer) on all supported lisp implementations. When rpgoldman convinced me that a portable, usable run-program should be part of asdf, whereas the legacy run-shell-command is an unusable unmaintainable unfixable crock, I decided to pull in the implementation from xcvb-driver. At the same time, drewc convinced me that having plenty of smaller files, with one package per file, was a good idea. I have been haunted for a few years for having failed janderson when he proposed to split asdf in parts; yet now asdf.lisp is much bigger than it was then, and at the same, having integrated bundle operations and figured out the dependency mess means it's easier and safer to write a general purpose source-concatenation operation. Finally you (Xach) convinced me that ASDF itself should not be a utility library, but only a build system; at the same time I really need the portability library, for XCVB as well as for ASDF and for any portable CL code I write. To top it all, after having completely reimplemented ASDF several times, I have started feeling much more comfortable with taking it apart and reassembling it in better ways. So there. I split ASDF in parts, and the part that's not specific to the system definition facility has been separated into asdf-driver. On most implementations, you can upgrade asdf-driver on top of an old ASDF -- though I admit I haven't yet added this situation to my automated testing regime. (The outlier as usual is CLISP, which is very funky to upgrade, and I don't know how to do things short of deleting the ASDF package when there's an incompatibility, which isn't conducive to upgrading ASDF-DRIVER far beyond ASDF; but I have ideas on how to make things a little bit more usable.) —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If I go through life free and rich, I shall not cry because my neighbour, equally free, is richer. Liberty will ultimately make all men rich; it will not make all men equally rich. Authority may (and may not) make all men equally rich in purse; it certainly will make them equally poor in all that makes life best worth living. — Benjamin Tucker

2 6

[asdf-devel] More test results
by Robert Goldman 16 Jan '13

16 Jan '13

ccl now passes. clisp fails to build ASDF successfully: + clisp -norc --silent -ansi -I -on-error exit -x '(or #.(load "test/script-support.lisp") #.(asdf-test::compile-asdf-script))' [3]> ;; Loading file test/script-support.lisp ... ;; Loading file /Users/rpg/lisp/asdf/build/asdf.lisp ... ;; Loaded file /Users/rpg/lisp/asdf/build/asdf.lisp TEST ABORTED: READ from #<INPUT BUFFERED FILE-STREAM CHARACTER #P"/Users/rpg/lisp/asdf/build/asdf.lisp" @7809>: there is no package with name "ASDF" Script failed Compilation FAILED still 9 tests failing on Allegro 8.2: asdf-pathname-test.script test-force.script test-module-depend.script test-module-excessive-depend.script test-static-and-serial.script test-touch-system-1.script test-touch-system-2.script test1.script test3.script didn't bother with allegromodern yet... SBCL passes ECL goes ape-execrement, and goes into what looks like an infinite loop in error-handling on test test-retry-loading-component-1.script. Seemingly infinite flood of the following: ASDF/LISP-BUILD::STATUS Caught error #<a UNBOUND-VARIABLE> trlc1 5 trlc1 6 ASDF-TEST::NAME => ASDF/FIND-COMPONENT:RETRY ASDF-TEST::MODE => :EXTERNAL trlc1 7 ASDF-TEST::NAME => ASDF/FIND-COMPONENT:RETRY RESTART => #<RESTART.4389276816> ABCL compilation failed for me: Testsuite failed: ASDF compiled with unexpected warnings Compilation FAILED + abcl --noinit --nosystem --noinform --eval '(or #.(load "test/script-support.lisp") #.(asdf-test::compile-asdf-script))' ; in (ASDF/UTILITY:DEFUN* BUNDLABLE-FILE-P ...) ; Caught STYLE-WARNING: ; The variable TYPE is defined but never used. Looks like this happens because none of the #+ conditional compilation cases gives ABCL any love in this function. Best, r

2 1

[asdf-devel] asdf-encodings?
by Raymond Toy 16 Jan '13

16 Jan '13

Is this the right place to talk about asdf-encodings? Apologies if this is the wrong place. One minor issue with asdf-encodings and cmucl: It looks like find-implementation-encoding does not expect an error for unknown encodings, so for cmucl, the implementation should be #+cmu (ignore-errors (stream::find-external-format encoding)) Also, once asdf-encodings is loaded, compiles with asdf fail because cl:compile-file is called with :external-format set to the external format object instead of the name. To get the name from the object, use (stream::ef-name obj). I don't know where that should be applied. BTW, make test lisp=cmucl CMUCL=cmulisp passes 41 tests, but there are some warnings like loading level1/file1 Warning: compiling #<STATIC-FILE "test-module-pathnames" "sources/level1" "level2/static.file"> completed without its input file #P"/home/rtoy/src/clnet/asdf/test/sources/level1/level2/static.file" Don't know if that's significant or not. Ray

2 2

[asdf-devel] News on the front
by Faré 15 Jan '13

15 Jan '13

Dear Lisp hackers, I've completed yet another major pass of refactoring of ASDF. It is now divided into many files, that are concatenated together for single-file delivery. (An idea that I once rejected when James Anderson proposed it 3 years ago; he might have known better. The file has more than quadrupled in size since.) >From a pristine git checkout, you can concatenate them from the command-line with make build/asdf.lisp or from lisp with a recent bootstrapped asdf.lisp with (asdf:operate 'asdf::concatenate-source-op :generate-asdf) About 45% of ASDF is utilities, and those are now available through a semi-independent system asdf-driver. See make wc wc-driver wc-asdf. I'have notably moved in run-program/ from xcvb-driver, and its native-namestring support, by popular demand, which increased this number. I brought in the condition-control code and image support from xcvb-driver as well; I'll be working to package them better. Hot upgrade is preserved through the heroic development of define-package, an extension to defpackage that makes things as declared from however they were, instead of throwing an error at the first discrepancy. You better know what you want. As a result from this and all the package declarations, plus the bundle and concatenate support, ASDF has swollen 60% in size since 2.26, or 42% since I merged in asdf-bundle in 2.26.7. I'll argue that the functionality is also much improved, including the fixing of the fundamental build model that I've discussed in previous emails. I won't release anything until all the packages in quicklisp work, and I'll have to install cl-test-grid for that, but I invite those of you who are interested to test it and/or comment on the changes. It looks like I mostly understand ASDF these days. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Every technique is first developed, then used, important, obsolete, normalized, and finally understood.

3 5

[asdf-devel] Status update on tests
by Robert Goldman 15 Jan '13

15 Jan '13

CCL fails on test-static-and-serial.script TEST ABORTED: These two expressions fail comparison with EQUAL: (FILE-WRITE-DATE FILE1) evaluates to 3567182153 DATE3 evaluates to 3567182002 Same failure on allegro. Allegro (8.2) has 6 failures all told: test-force.script test-static-and-serial.script test-touch-system-1.script test-touch-system-2.script test1.script test3.script On allegro there are a large number of date comparison failures. TEST ABORTED: the assertion (EQUAL (FILE-WRITE-DATE FILE1) DATE2) TEST ABORTED: These two expressions fail comparison with EQUAL: TEST ABORTED: These two expressions fail comparison with EQUAL: TEST ABORTED: the assertion (> DATE2 DATE1) failed. TEST ABORTED: These two expressions fail comparison with EQUAL: TEST ABORTED: the assertion (NOT (PROBE-FILE FASL2)) failed. More results as I can grab time to collect them. cheers, r

2 4

[asdf-devel] Test results: one failure on CCL
by Robert Goldman 14 Jan '13

14 Jan '13

Running test-all-noupgrade, CCL fails for me on test-encodings.script: Testing: test-encodings.script + /Users/rpg/ccl/dx86cl64 --no-init --quiet --eval '(load "test-encodings.script")' Test TEST-ENCODING-EXPLICIT-U8: should be UTF-8 ; Registering #<SYSTEM "test-encoding-explicit-u8"> TEST ABORTED: Component "asdf-encodings" not found Looks like this might be a simple bundling problem? Mac OS X, CCL 1.8 cheers, r

2 4