- asdf-devel - mailman3.common-lisp.net

[asdf-devel] 1.671 compiles in abc, acl, ccl, ecl, cmucl, sbcl, lw
by james anderson 08 Apr '10

08 Apr '10

asdf@ip-10-251-122-82:/development/source/library$ /development/bin/ abcl --load build-init.lisp Armed Bear Common Lisp 0.19.1 Java 1.6.0_0 Sun Microsystems Inc. OpenJDK Client VM Low-level initialization completed in 0.963 seconds. Startup completed in 2.205 seconds. ; Compiling /ebs/source/library/net/common-lisp/asdf/asdf.lisp ... ; [ ... ] ; Wrote /ebs/source/library/net/common-lisp/asdf/asdf.abcl (13.859 seconds) Type ":help" for a list of available commands. CL-USER(1): asdf@ip-10-251-122-82:/development/source/library$ /development/bin/ alisp -qq -L build-init.lisp ;Build root: #P"/ebs/source/library/build-init.lisp". ;Incorporating asdf anew from #P"/ebs/source/library/net/common-lisp/ asdf/asdf.lisp". ;;; Compiling file /ebs/source/library/net/common-lisp/asdf/asdf.lisp ; [...] ;;; Writing fasl file /ebs/source/library/net/common-lisp/asdf/asdf.fasl ;;; Fasl write complete ;;; [ ... ] CL-USER(1): EOF Really exit lisp [n]? ; Exiting asdf@ip-10-251-122-82:/development/source/library$ /development/bin/ clisp -i build-init.lisp i i i i i i i ooooo o ooooooo ooooo ooooo I I I I I I I 8 8 8 8 8 o 8 8 I \ `+' / I 8 8 8 8 8 8 \ `-+-' / 8 8 8 ooooo 8oooo `-__|__-' 8 8 8 8 8 | 8 o 8 8 o 8 8 ------+------ ooooo 8oooooo ooo8ooo ooooo 8 Welcome to GNU CLISP 2.48+ (2009-08-24) <http://clisp.cons.org/> Copyright (c) Bruno Haible, Michael Stoll 1992, 1993 Copyright (c) Bruno Haible, Marcus Daniels 1994-1997 Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998 Copyright (c) Bruno Haible, Sam Steingold 1999-2000 Copyright (c) Sam Steingold, Bruno Haible 2001-2009 Type :h and hit Enter for context help. ;; Loading file build-init.lisp ... ;Build root: #P"/ebs/source/library/build-init.lisp". ;Incorporating asdf anew from #P"/ebs/source/library/net/common-lisp/ asdf/asdf.lisp". ;; Compiling file /ebs/source/library/net/common-lisp/asdf/asdf.lisp ... ;; Wrote file /ebs/source/library/net/common-lisp/asdf/asdf.fas 0 errors, 0 warnings ;; Loading file /ebs/source/library/net/common-lisp/asdf/asdf.fas ... ;; Loaded file /ebs/source/library/net/common-lisp/asdf/asdf.fas ;; [ ... ] [1]> Bye. asdf@ip-10-251-122-82:/development/source/library$ ccl --no-init -l build-init.lisp ;Build root: #P"/ebs/source/library/build-init.lisp". ;Incorporating asdf anew from #P"/ebs/source/library/net/common-lisp/ asdf/asdf.lisp". ;Compiling "/ebs/source/library/net/common-lisp/asdf/asdf.lisp"... ;Loading #P"/ebs/source/library/net/common-lisp/asdf/asdf.lx32fsl"... ;Loading #P"/ebs/source/library/de/setf/utility/asdf/hierarchical- names.lisp"... ;Loading #P"/ebs/source/library/de/setf/utility/pathnames.lisp"... ;Loading #P"/ebs/source/library/de/setf/utility/package.lisp"... Host translations: LIBRARY base: #P"/ebs/source/library/de/setf/utility/pathnames.lisp". location: NIL. root-target: #P"/ebs/source/library/**/*.*". binary: #P"/ebs/source/library/bin/omcl/**/*.lx32fsl". Welcome to Clozure Common Lisp Version 1.4-r13119 (LinuxX8632)! ? asdf@ip-10-251-122-82:/development/source/library$ /development/bin/ cmucl -load build-init.lisp ; Loading #P"/ebs/source/library/build-init.lisp". ;Build root: #P"/ebs/source/library/build-init.lisp". ; Python version 1.1, VM version Intel x86 on 08 APR 10 10:32:32 pm. ; Compiling: /ebs/source/library/net/common-lisp/asdf/asdf.lisp 08 APR 10 10:26:56 pm ; [ ... ]. ; /ebs/source/library/net/common-lisp/asdf/asdf.sse2f written. ; Compilation finished in 0:00:06. ;Incorporating asdf anew from #P"/ebs/source/library/net/common-lisp/ asdf/asdf.lisp". ;; Loading #P"/ebs/source/library/net/common-lisp/asdf/asdf.sse2f". ; [GC threshold exceeded with 19,203,144 bytes in use. Commencing GC.] ; [GC completed with 2,603,824 bytes retained and 16,599,320 bytes freed.] ; [GC will next occur when at least 14,603,824 bytes are in use.] CMU Common Lisp 20a (20A), running on ip-10-251-122-82 With core: /development/downloads/cmucl-20a-non-unicode-x86-linux/lib/ cmucl/lib/lisp-sse2.core Dumped on: Mon, 2009-09-28 16:06:53Z on usrtc3154 See <http://www.cons.org/cmucl/> for support information. Loaded subsystems: Python 1.1, target Intel x86 CLOS based on Gerd's PCL 2008-11-12 16:36:41 * (quit) asdf@ip-10-251-122-82:/development/source/library$ /development/bin/ ecl -load build-init.lisp ;;; Loading "/ebs/source/library/build-init.lisp" ;Build root: #P"/ebs/source/library/build-init.lisp". ;Incorporating asdf anew from #P"/ebs/source/library/net/common-lisp/ asdf/asdf.lisp". ;;; Loading #P"/development/lib/ecl-10.2.1/cmp.fas" ;;; Loading #P"/development/lib/ecl-10.2.1/sysfun.lsp" ;;; ;;; Compiling /ebs/source/library/net/common-lisp/asdf/asdf.lisp. ;;; [...] ;;; Note: ;;; Invoking external command: ;;; gcc -o "/ebs/source/library/net/common-lisp/asdf/asdf-ecl.fas" - L"/development/lib/" "/ebs/source/library/net/common-lisp/asdf/asdf- ecl.o" -shared -lecl -lgmp -ldl -lm ;;; ;;; Finished compiling /ebs/source/library/net/common-lisp/asdf/asdf- ecl.lisp. ;;; ;;; Loading "/ebs/source/library/net/common-lisp/asdf/asdf-ecl.fas" ;;; Loading "/ebs/source/library/de/setf/utility/asdf/hierarchical- names.lisp" ;;; Loading "/ebs/source/library/de/setf/utility/asdf/../pathnames.lisp" ;;; Loading "/ebs/source/library/de/setf/utility/asdf/../package.lisp" ;;; OPTIMIZE levels: Safety=3, Space=0, Speed=2, Debug=2 ;;; ;;; End of Pass 1. ;;; Note: ;;; Invoking external command: ;;; gcc "-I/development/include/" -D_GNU_SOURCE - D_FILE_OFFSET_BITS=64 -g -O2 -fPIC -Dlinux -O -w -c "/tmp/ ECL001G986OB.c" -o "/tmp/ECL001G986OB.o" ;;; ;;; Note: ;;; Invoking external command: ;;; gcc -o "/tmp/ECL001G986OB.fas" -L"/development/lib/" "/tmp/ ECL001G986OB.o" -shared -lecl -lgmp -ldl -lm ;;; Host translations: LIBRARY base: #P"/ebs/source/library/de/setf/utility/pathnames.lisp". location: NIL. root-target: #P"/ebs/source/library/**/*.*". binary: #P"/ebs/source/library/bin/ECL/**/*.fas".ECL (Embeddable Common-Lisp) 10.2.1 Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya Copyright (C) 1993 Giuseppe Attardi Copyright (C) 2000 Juan J. Garcia-Ripoll ECL is free software, and you are welcome to redistribute it under certain conditions; see file 'Copyright' for details. Type :h for Help. Top level. > asdf::*asdf-version* "1.671" > asdf@ip-10-251-122-82:/development/source/library$ /usr/local/bin/ sbcl --no-sysinit --userinit build-init.lisp This is SBCL 1.0.36, an implementation of ANSI Common Lisp. More information about SBCL is available at <http://www.sbcl.org/>. SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. ;Build root: #P"/ebs/source/library/". ;Incorporating asdf anew from #P"/ebs/source/library/net/common-lisp/ asdf/asdf.lisp". ; compiling file "/ebs/source/library/net/common-lisp/asdf/ asdf.lisp" (written 08 APR 2010 10:26:56 PM): ; compiling (IN-PACKAGE :CL-USER) ; compiling (DECLAIM (OPTIMIZE # ...)) ; compiling (LET* (# # ...) ...) ; file: /ebs/source/library/net/common-lisp/asdf/asdf.lisp ; [ ... ] ; /ebs/source/library/net/common-lisp/asdf/asdf.fasl written ; compilation finished in 0:00:05.487 ; loading #P"/ebs/source/library/net/common-lisp/asdf/asdf.fasl" ; loading #P"/ebs/source/library/de/setf/utility/asdf/hierarchical- names.lisp" ;; loading #P"/ebs/source/library/de/setf/utility/asdf/../ pathnames.lisp" ;;; loading #P"/ebs/source/library/de/setf/utility/asdf/../package.lisp" Host translations: LIBRARY base: #P"/ebs/source/library/de/setf/utility/pathnames.lisp". location: NIL. root-target: #P"/ebs/source/library/**/*.*". binary: #P"/ebs/source/library/bin/sbcl-linux/**/*.fasl". * asdf@ip-10-251-122-82:/development/source/library$ lispworks LispWorks(R): The Common Lisp Programming Environment Personal Edition Copyright (C) 1987-2008 LispWorks Ltd. All rights reserved. Version 5.1.1 Saved by LispWorks as lispworks-personal-5-1-1-x86-linux, at 13 May 2008 17:15 User asdf on ip-10-251-122-82 Error: Could not register handle for external module X- UTILITIES::CAPIX11: libXm.so.3: cannot open shared object file: No such file or directory. 1 (abort) Quit process. Type :b for backtrace, :c <option number> to proceed, or :? for other options CL-USER 1 : 1 > (load "build-init.lisp") ; Loading text file /ebs/source/library/build-init.lisp ;Build root: #P"/ebs/source/library/build-init.lisp". ;Incorporating asdf anew from #P"/ebs/source/library/net/common-lisp/ asdf/asdf.lisp". ;;; Compiling file /ebs/source/library/net/common-lisp/asdf/ asdf.lisp ... ;;; Safety = 3, Speed = 1, Space = 1, Float = 1, Interruptible = 0 ;;; Compilation speed = 1, Debug = 2, Fixnum safety = 3 ;;; [ ... ] ; Loading fasl file /ebs/source/library/net/common-lisp/asdf/asdf.ufasl #P"/ebs/source/library/build-init.lisp" CL-USER 2 : 1 >

1 0

[asdf-devel] duplicate-name condition
by Tobias C. Rittweiler 07 Apr '10

07 Apr '10

I was just bitten by a duplicate-name condition. It was on ASDF1, but the code still seems to be exactly the same in ASDF2. Please add a proper error message to that condition. I first thought there are two systems with the same name, but it actually complains about two files with the same name in the same component. -T.

2 1

Re: [asdf-devel] Patches for ASDF2 for translation function
by Faré 06 Apr '10

06 Apr '10

Dear Mark, I merged your patch in, after rearranging it a little bit. Tell me how it's working for you... [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Procrastination is great. It gives me a lot more time to do things that I'm never going to do. On 5 April 2010 14:09, Mark Evenson <evenson(a)panix.com> wrote: > [With the attached patches, the following email get rejected by both the > asdf-devel and armedbear-devel mailing lists. If the mail doesn't get > through in the next day, I'll stash the patches at a URL and resend.] > > -------- Original Message -------- > Subject: Patches for ASDF2 for translation function > Date: Mon, 05 Apr 2010 14:52:00 +0200 > From: Mark Evenson <evenson(a)panix.com> > To: ASDF-devel <asdf-devel(a)common-lisp.net> > CC: Armed Bear <armedbear-devel(a)common-lisp.net> > > On 3/29/10 11:33 AM, Mark Evenson wrote: > […] > >>> I'd favor either a keyword argument or :function as a magic marker >>> to be inserted in FIRST position, followed by a function designator >>> (and additionally, parameters to curry?). Unless of course, you >>> symmetrically accept a FUNCTION in first position to denote a >>> "recognize if this matching rule applies". >> >> >> I would think that we want to preserve the current ability for the >> user to vistually scan down the list of translations to determine >> what should match, so I would favor going for your second proposal. >> It doesn't make sense that there be a match predicate paired with a >> translation pathname right? So the two new possibilities for output >> translation lists would be: >> >> ("/**/foo/*.*" (:function FOO-OUTPUT-TRANSLATION)) >> ((:function BAR-PATHNAME-P) (:function BAR-OUTPUT-TRANSLATION)) > > Attached is a patch against ASDF that allows a user specified function > to take the place of TRANSLATE-PATHNAME as the ASDF2 output translation > function. > > Additionally it adds an ABCL specific extension to ASDF2 that uses this > ability by providing the following translations by default > > (#p"jar:file:/**/*.jar!/**/*.*" (:function translate-jar-pathname)) > (#p"/:jar:file/**/*.*" (:user-cache #p"**/*.*"))) > > that will map all ASDF loads from jar pathnames to an output location > under the per user cache location. This will allow the loading of ASDF > systems packaged in jar files which first compiles and then loads FASLs > to the local filesystem. > > To run this with ABCL you will need to build a custom build of ABCL with > the attached patch which is currently necessary to OPEN a jar pathname. > This patch is part of a large set of modifications that allow ABCL to > open a pathname specified as a url, that I am developing and testing as > a unit. These modiictions will be placed in ABCL trunk soon, but first > I need to add create more unit tests for the introduced functionality > and implement a few more suggestions (like URI encoding functions). > > I did not implement Faré's suggestion of including a syntax for a > predicate function as the first argument for a output translation pair, > because I couldn't come up with a use case in which the current default > function PATHNAME-MATCH-P would not work. If someone can come up with > such a case, or this deficiency prevents ASDF from accepting this patch, > I can have a go at it again. > > -- > "A screaming comes across the sky. It has happened before, but there > is nothing to compare to it now."

2 1

[asdf-devel] ASDF2 cache control?
by Jean-Claude Beaudoin 06 Apr '10

06 Apr '10

Hello ASDF developers, I just upgraded my version of ASDF to 1.661, more or less the latest and greatest. I would like to know how to turn off the compiler output cache that ASDF now uses. In the currently available manual (on the web site) there is a mention of variable *centralize-lisp-binaries* and a few other variables that would control that area of ASDF. But *centralize-lisp-binaries* is no where to be found in the current source code. The manual obviously lags significantly behind the evolution of the code... So how does one turn off the compiler output cache in this new version of ASDF? Thanks, Jean-Claude Beaudoin

6 18

[asdf-devel] ? interaction between asdf and cmucl on the matter of "home:"
by james anderson 05 Apr '10

05 Apr '10

good evening; an attempt to use asdf in cmucl to build alexandria fails in connection with operations on a pathname designator which begins with "home:". one observes, that it is not only commercial vendors who know what is best for us. is cmu 20* expected to work with asdf 1.6*? is there some this to configure to eliminate the problem? i have read about ext:clear-search-list, but its effect is somehow not sufficient. the problem persists even once (ext;search-list-defined-p "home:") yields nil prior to compile-loading asdf. asdf@ip-10-251-122-82:/development/source/metadata/net/common-lisp/ alexandria$ /development/bin/cmucl CMU Common Lisp 20a (20A), running on ip-10-251-122-82 With core: /development/downloads/cmucl-20a-non-unicode-x86-linux/lib/ cmucl/lib/lisp-sse2.core Dumped on: Mon, 2009-09-28 16:06:53Z on usrtc3154 See <http://www.cons.org/cmucl/> for support information. Loaded subsystems: Python 1.1, target Intel x86 CLOS based on Gerd's PCL 2008-11-12 16:36:41 * (load (compile-file-pathname "/development/source/library/net/ common-lisp/asdf/asdf.lisp")) ; Loading #P"/ebs/source/library/net/common-lisp/asdf/asdf.sse2f". T * asdf::*asdf-version* "1.661" * (push #p"/development/source/upload/net/common-lisp/alexandria/" asdf:*central-registry*) (#P"/development/source/upload/net/common-lisp/alexandria/") * (asdf:load-system :alexandria) ; loading system definition from ; /development/source/upload/net/common-lisp/alexandria/ alexandria.asd into ; #<The ASDF0 package> ; Loading #P"/ebs/source/upload/net/common-lisp/alexandria/ alexandria.asd". ; registering #<SYSTEM :ALEXANDRIA {58B015CD}> as ALEXANDRIA ; Compiling LAMBDA (.PV-CELL. .NEXT-METHOD-CALL. O C): ; Compiling Top-Level Form: ; [GC threshold exceeded with 12,008,296 bytes in use. Commencing GC.] ; [GC completed with 1,578,760 bytes retained and 10,429,536 bytes freed.] ; [GC will next occur when at least 13,578,760 bytes are in use.] ; Compiling LAMBDA (.PV-CELL. .NEXT-METHOD-CALL. O C): ; Compiling Top-Level Form: Type-error in KERNEL::OBJECT-NOT-TYPE-ERROR-HANDLER: #<SEARCH-LIST home> is not of type (OR LISP::PATTERN SYMBOL SIMPLE-BASE-STRING INTEGER) [Condition of type TYPE-ERROR] Restarts: 0: [ABORT] Return to Top-Level. Debug (type H for help) (LISP::COMPONENTS-MATCH 2 "ebs" #<SEARCH-LIST home>)[:EXTERNAL] Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/pathname.lisp. 0] (logical-pathname-translations "HOME") File-error in function LISP::FIND-LOGICAL-HOST: Logical host not yet defined: "HOME" Error flushed ... 0] (extensions:search-list "home:") (#P"/home/asdf/") 0] :back 0: (LISP::COMPONENTS-MATCH 2 "ebs" #<SEARCH-LIST home>)[:EXTERNAL] 1: (LISP::DIRECTORY-COMPONENTS-MATCH ("ebs" "source" "upload" "net" "common-lisp" ...) (#<SEARCH-LIST home> ".cache" "common-lisp" "cmu-20a__20a_-linux- x86" :WILD-INFERIORS)) 2: (LISP::DIRECTORY-COMPONENTS-MATCH 2 (:ABSOLUTE "ebs" "source" "upload" "net" ...) (:ABSOLUTE #<SEARCH-LIST home> ".cache" "common-lisp" "cmu-20a__20a_- linux-x86" ...))[:EXTERNAL] 3: (PATHNAME-MATCH-P #P"/ebs/source/upload/net/common-lisp/alexandria/package.sse2f" #P"home:.cache/common-lisp/cmu-20a__20a_-linux-x86/**/*.*") 4: (ASDF:APPLY-OUTPUT-TRANSLATIONS #P"/ebs/source/upload/net/common-lisp/alexandria/package.sse2f") 5: ((METHOD ASDF:OUTPUT-FILES (:AROUND) (T T)) #<unused-arg> #S(PCL::FAST-METHOD-CALL :FUNCTION #<Function # {58CCC079}> :PV-CELL (# . #) :NEXT-METHOD-CALL #S(PCL::FAST-METHOD-CALL :FUNCTION # :PV-CELL NIL :NEXT-METHOD-CALL NIL :ARG-INFO #) :ARG-INFO (2)) #<ASDF:COMPILE-OP NIL {581EEBED}> #<ASDF:CL-SOURCE-FILE "package" {58D08F9D}>) 6: ((METHOD ASDF:OPERATION-DONE-P NIL (ASDF:OPERATION ASDF:COMPONENT)) (#(7) . #(# #)) #<unused-arg> #<ASDF:COMPILE-OP NIL {581EEBED}> #<ASDF:CL-SOURCE-FILE "package" {58D08F9D}>) 7: ((METHOD ASDF::TRAVERSE NIL (ASDF:OPERATION ASDF:COMPONENT)) (#(3 5 NIL NIL) . #(# # # # #)) #<unused-arg> #<ASDF:COMPILE-OP NIL {581EEBED}> #<ASDF:CL-SOURCE-FILE "package" {58D08F9D}>) 8: ((METHOD ASDF::TRAVERSE NIL (ASDF:OPERATION ASDF:COMPONENT)) (#(3 5 9 10) . #(# # # # #)) #<unused-arg> #<ASDF:COMPILE-OP NIL {581EEBED}> #<ASDF:SYSTEM "alexandria" {58C85305}>) 9: ((LABELS ASDF::DO-ONE-DEP) ASDF:COMPILE-OP "alexandria" NIL) 10: ((LABELS ASDF::DO-DEP) ASDF:COMPILE-OP ("alexandria")) 11: ((METHOD ASDF::TRAVERSE NIL (ASDF:OPERATION ASDF:COMPONENT)) (#(3 5 9 10) . #(# # # # #)) #<unused-arg> #<ASDF:LOAD-OP NIL {58AF290D}> #<ASDF:SYSTEM "alexandria" {58C85305}>) 12: (ASDF:OPERATE ASDF:LOAD-OP :ALEXANDRIA) 13: (INTERACTIVE-EVAL (ASDF:LOAD-SYSTEM :ALEXANDRIA)) 14: (LISP::%TOP-LEVEL) 15: ((LABELS LISP::RESTART-LISP SAVE-LISP)) 0]

2 3

[asdf-devel] ASDF2: Please, no unwrapped calls to "truename".
by Jean-Claude Beaudoin 05 Apr '10

05 Apr '10

My very first attempt at using ASDF version 1.661 on my system ended-up in the debugger on a file-error signaled by function truename. The offending call came from function asdf::component-parent-pathname that contains a call to truename unwrapped in any condition handler although truename is required to signal errors. This seems to be the only such instance in asdf.lisp, all the other truename calls being wrapped inside a ignore-errors. I fixed the situation by modifying asdf::component-parent-pathname this way: (defun component-parent-pathname (component) (aif (component-parent component) (component-pathname it) (or (ignore-errors (truename *default-pathname-defaults*)) *default-pathname-defaults*))) BTW, I would like to mention that in my CL system *default-pathname-defaults* is not a replacement for the process current working directory (which can be properly queried only through a call to getcwd). Thinking otherwise seems to me to be pretty clearly an unportable assumption unsupported by the ANSI CL standard. That ASDF seems to make that assumption troubles me seriously. In fact I cannot figure out how I would explain to a naive beginning lisper on MS-Windows that *default-pathname-defaults* is an equivalent of the current directory when there is a potential of up to 26 simultaneous current directories on his OS (one per drive letter). On the top of that the current working director(y|ies) is/are a process-level attribute and *default-pathname-defaults* being a special variable is most likely to have a thread-local binding at one point or an other, in some CL systems this could even be part of the initial thread bindings. How one reconciles these many independently evolving thread bindings with the single process-level attribute sure beat me! Cheers, Jean-Claude Beaudoin

3 4

[asdf-devel] Patches for ABCL against asdf-1.641
by Mark Evenson 05 Apr '10

05 Apr '10

Attached are patches in 'abcl-asdf.patch' to get asdf-1.641 to work against ABCL for the run-tests.sh. Unfortunately, you will need to build [ABCL from trunk][1] using at least svn r12550, because I had to patch ABCL to work with ASDF. And you'll need to apply the 'abcl-translate-pathname.patch'. [1]: svn://common-lisp.net/project/armedbear/svn/trunk/abcl Notes: 1. The fugliness of the conditional around ASDF:GET-UID works around a bug in the ABCL compiler. We are in the process of [analyzing the error][2]. [2]: http://trac.common-lisp.net/armedbear/ticket/89 2. There is some undiagnosed problem in translating the binary location for the new ASDF2 "~/.cache" conventions, as ABCL seems to "collapses" everything into a single directory. Somehow, the default *output-translations* have a "/**/**/*.*" where there should be "/**/*.*". I'm not sure if this is a problem in "abcl-translate-pathname.patch", which is why I didn't apply this to ABCL trunk. 3. ASDF's run-tests.sh seems to ignore the "flags" setting, which seems to be broken on the ASDF side. 4. The changes for ASDF look quite interesting. However I would advocate that you should make ASDF2 work *exactly* like ASDF1 out of the box, meaning you shouldn't do any of the subsumed ASDF-BINARY-LOCATIONS stuff without being asked to by configuration. -- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."

6 26

[asdf-devel] Just pushed the system-source-directory patch, which I thought had been pushed some time ago
by Robert Goldman 04 Apr '10

04 Apr '10

Faré, will you please verify that I didn't muck up your package-wrangling patch? thanks --- sorry about the confusion. best, r

1 0

[asdf-devel] ASDF 2 issues
by Faré 01 Apr '10

01 Apr '10

There are many things to clarify about logical pathnames and the use thereof or not, but I don't think this should block a release of ASDF 2. Should it? And so I'd like to declare it an ASDF 2.1 or ASDF 3 issue, and invite you to focus on blocking issues for ASDF 2. (Of course, if someone has a working, portable, solution that makes everyone happy, I'll gladly merge and commit it.) I think the asdf-bundle extension is more important, because underlying it is the structure of how we deliver extensions, and how we play well with ECL, especially as it might entail some slight backwards incompatibility for ECL that I'd rather not happen *after* we release ASDF 2. Can we focus on getting ASDF 2 out before we add new features? [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Superstition brings bad luck. — Paul Carvel On 31 March 2010 14:00, Juan Jose Garcia-Ripoll <juanjose.garciaripoll(a)googlemail.com> wrote: > On Wed, Mar 31, 2010 at 4:39 PM, james anderson <james.anderson(a)setf.de> > wrote: >> >> i simply don't understand your requirements. on one hand both with and >> without asdf, but then it is objected, that a mechanism which requires asdf >> does not work without it. please explain. > > What I do not understand is what YOU do not understand. > > In an standalone system there is no ASDF. That is all. > > I want to be able to take an ASDF system describing my program and its > dependencies on other libraries and compile it and produce a standalone > executable. That program will not have ASDF inside, it will not have the > notion of a central registry it will have nothing, but it still has to work. > > Please understand that the program I am talking about is not a dumped lisp > image that contains everything needed to build the system and even some > extra stuff that was there (database of ASDF systems and other trash). The > programs I produce are just a set of compiled files, one per ASDF system, > linked together with a core library that is the Common Lisp environment ECL > offers. > > When the program starts, the linked files are "loaded" one after another, as > if LOAD was used but actually much faster. But I insist again ASDF is not > part of the things that is loaded simply because there is no need. > > The only thing that one might need ASDF for is to locate the data associated > to the program, but if we provide logical pathname translations (one host > per library), then a small piece of code can be inserted at the beginning of > the program which locates the data and sets up the appropriate translations. > >> if the requirement is that exact same exact formulation be used with and >> without asdf, then >> a) the additional system initialization argument is excluded >> b) the second option in my earlier message provides this path. > > > I only find "b" acceptable because I still do not understand how "a" is > going to work without a central registry and a lot of complicated logic that > has to propagate to those distributed binaries. > > Having "b", which I recall was one logical host name per system, one may > simply prepend to the compiled set of files a tiny bit of lisp that looks as > follows > > (defvar *program-data-location* > #+windows ;; Data and binaries live together > (program-location) > #+unix ;; Data is installed somewhere standard > "/usr/share/program/" > #+osx ;; Assume this is an application bundle > (merge-pathnames "Content/Resources/" (program-location))) > > (defun set-host-translation (root extra) > ...) > > (set-host-translation "cl-ppcre" "cl-ppcre/") > (set-host-translation "cl-unicode" cl-unicode/") > > With this simple, automatically generated code I can pack the given systems > and let them work anywhere, even though ASDF is not present. > > The same or a similar procedure can be used to link many systems into a > single distributable binary fasl + resources, install it in a standard > location that can be found by certain lisps and prepend a simple logic for > finding out its data... Without using ASDF in the file itself. > > There is even a third model in which installation paths are known > beforehand. Say that a package maintainer from Debian has the role of > generating binaries for 10 libraries out there and they have complex > inter-dependencies. Libraries can be compiled in the machine of the > maintainer and then installed by ASDF in their standard locations. ASDF can, > in the process, hard-code the pathname translations when installing the > files. > > For instance, if we produce binaries for cl-ppcre-1.0.1 and cl-unicode-2.0.1 > and the latter depends on the former, and we know that they are going to > live in > /usr/lib/common-lisp/cl-ppcre-1.0.1 > /usr/lib/common-lisp/cl-unicode-2.0.1 > when "asdf-build" compiles the files and "asdf-installs" them we may get > > ;; Stripped down version of a system definition > ;; which only contains the precompiled files > ;; and which can be searched and used by ASDF > /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.asd > /usr/lib/common-lisp/cl-unicode-1.0.1/cl-unicode.asd > > ;; The binaries themselves > /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre-1.0.1.fas > /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode-1.0.1.fas > > ;; Proxy binaries that can be loaded using "require" > ;; even if ASDF is not present. These files define the logical > ;; pathname translations based on hardcoded locations. > /user/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.fas > /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.fas > > ;; Data used by the library > /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.dat > > ;; Symbolic link to the last version > /usr/lib/common-lisp/cl-ppcre -> cl-ppcre-1.0.1 > > The FASL without version numbers files can explicitly load the required > versions of the libraries because they know the hardcoded paths of their > files. In doing so they can also set up the appropriate pathname > translations that are used by the other files. > > I have listed just two examples with different paradigms on how to build and > distribute software, and which go beyond current use in which the user has > to set up a collection of registries, in your case you even define pathname > translations and have magic to make ASDF understand them, sources are always > present, things are recompiled... > > But there are other possibilities. ASDF can be used to produce programs and > libraries that can be distributed, ASDF can itself be used without people > noticing it (see the automatically generated FASL files), it can integrate > with other build processes (via some simple scripts asdf-build, asdf-install > which I may offer to write), we can achieve better system descriptions, and > a lot of things that make people's life easier because they can forget that > they use ASDF at all -- just like when writing C I am free to forget that I > am using "make" at some point. > > Juanjo

2 2

[asdf-devel] The future of ASDF (was logical pathnames)
by Juan Jose Garcia-Ripoll 31 Mar '10

31 Mar '10

This email arises from a discussion at the ECL mailing list about how ASDF needs an urgent replacement and that is should be just discontinued and replaced with something else. Autoconf, extended REQUIREs and other stuff was suggested. I do not agree. I think ASDF can be fixed and made easier to use. Before explaining my view, please let me disgress a bit. I use autoconf a lot. There is no way around it if one works in Unix environments with moderately complicated software [1,2]. The need of autoconf is motivated and justified because of certain reasons 1. All build environments in Unix work the same: all C/C++ compilers are similar in the way they process their input, arguments they take and what they produce: object files. 2. Components are found in rather standard locations (/usr/include, /usr/lib, ...) and Autoconf provides information about that. 3. Components may or may not be present. Autoconf helps us determining that. 4. We normally have a simple build process with a list of files to be compiled and linked together. Dependencies are also simple forming an open graph, without loops. Automake takes this graph and produces makefiles which build the system 5. Autoconf installs your software via the generated files. We know where to place them. If the user chooses a nonstandard location he will be punished but it has a simple solution (set an environment variable). What about lisp? 1. Each lisp behaves differently. ECL is not at all like SBCL. Currently this is solved by imposing a rather outmodish way of building the software based on compile-file and load. No notion of standalone binary files, no way of producing executables, no way of shipping things. 2. Each lisp has a different notion of how to locate "required" components. Some have search paths (SBCL), some do not (ECL). There is not a standard for installation. 3. There is an additional problem in the lisp world: we do not use the shared library loader! In order to locate components we may either rely on crappy things like REQUIRE or rely on an external agent that seeks those components prior to loading our software. ASDF behaves in that sense as a shared library loader that takes into account version information! 4. System definitions provide the description of our system and how pieces bind together. This has always been like that, even before ASDF, with mk:defsystem. It is a simple, mostly declarative description. What we do with it is up to the software that manipulates it, just like automake may evolve at some point and behave differently. 5. Installation is again a problem because there is not a well defined way to look for software. Furthermore, since current system descriptions do not provide information about "resources" that the software may need, such as bitmaps, icons, databases, etc, we have no information about installing them and instead things are left in place. On comparing both lists one must reach the conclusion that the situation is bad. Indeed. All of the problem revolves around two things * We need some way to build our system taking into account all dependencies * There is no standard way to produce standalone libraries and install them in a way that just works. ASDF tries to solve BOTH problems, that is BUILDING and LOADING/EXECUTING. The problem is that development has over the time forgotten the distinction between both and one finds amazing things such as users that place some of the execution logic (creating packages, classes, etc) IN THE SYSTEM DEFINITION! However, while we are in a position of say, disadvantage, w.r.t the Unix community at large, I believe we have a good starting point to get things right. My vision is to *progressively* decouple the building and loading components in ASDF. I do not want to push anything down anybody's throat, but even less take all the existing database of knowledge about all the libraries that build and work with ASDF and presenting myself as an illuminated that has the solution to everything and that this solution has to be something orthogonal to ASDF. How can we achieve these things and make everybody's life simpler? I am not going to produce the roadmap, but what I present below can be done progressively, and examples for that are the bits I have been contributing to the asdf-devel list. * First of all understand that we have two different targets here. One is people building standalone software (programs) and the other one is people building libraries. In both cases the system definitions we have are good enough to describe them and their dependencies. Let's call the component in ASDF that handles these systems ASDF-system. This includes classes and the parsing logic. This is very tiny and should be easy to bootstrap. * Understand how different lisps locate software. Realize that not all lisps can do that cleverly and that this is one of the reasons for ASDF to exist. Produce a minimalistic ASDF-search that reuses existing logic or implements new one for lisps that miss that part. * Realize that we need a better way of loading software. I do not care if that guy is improving REQUIRE, he is building a new component that people will have to install or ship with their lisps and autoconfed scripts are not going to solve it. * As a solution to the previous point realize that system definitions already have enough information for an ASDF-loader facility that builds on ASDF-search and ASDF-system. Even for standalone systems it is possible to create stripped down *.asd files with version information and dependencies in them. This is already present in the patch I sent to the list and ECL can also make it: it is the binary-op operation I introduced. The result is a file (defsystem :my-system :depends-on (:cl-ppcre) :components ((:compiled-file "my-component"))) which contains just the minimal information for our "shared library" loader to work. I personally find this better (and potentially more extensible) than Unix version numbers in file names (libecl.so.10.2.1 ?!?) * ASDF-loader can be a very transparent component and it may even be possible to ship it together with _every_ library you have around, just like libtool is shipped with every library. As a compatibility layer over REQUIRE it should be possible to produce three files for each system - the system definition my-system.asd - a simple-minded loader my-system.fas that sets up paths, requires asdf-loader and loads the last file which is... - the component of the library itself, my-component.fas * We still need a building facility, say ASDF-builder, that uses a system definition to compile everything. This would be the COMPILE-OP we have right now, glorified and improved, but it would be augmented with facilities to pack a library in a single file, building standalone programs, etc. * The interesting part about this decoupling is that standalone programs do not need any of it. I can just use ASDF-builder as a glorified makefile and produce a program that just works by concatenating or linking pieces of precompiled software and inserting a simple logic at the beginning of the program that sets up the environment that the components expect. No need for ASDF-loader nor ASDF-search nor ASDF-system. Since I already removed all dependency on ASDF from the executable code itself by having logical pathnames, things just work. Compare this with a hand-built software where I abuse REQUIRE here and there. Say you are a developer of a library, MY-LIBRARY, and want the user to just build it and install it. There are various alternatives to achieve this. One is to continue as now. The user loads ASDF and from the prompt she is able to compile, build and install the software with existing operations. But from the point of view of the user things can be improved a lot by means of scripting. * Assume that ASDF is preinstalled by the user. This needs not be done in complicated fashion. ASDF itself can be "autoconf'ed" and taught to find out the existing lisps, install itself, and create a set of scripts that the application is going to use: "asdf-build" and "asdf-install". * Alternatively you may ship a version of ASDF with your library, hidden in an asdf directory just like libtool hides itself in the m4 or config directory of a Unix library. The scripts asdf-build and asdf-install are then placed on the top of your library source tree, where the *.asd definitions live. Now the user only has to do something like ./asdf-build --lisps=sbcl,ecl,acl ./asdf-install These two scripts will loop over lisps loading them with ASDF, compiling and installing stuff, reusing the knowledge this library has about the systems themselves. If the lisp environment does not have ASDF installed, a copy may be installed together with the library or we may use the logic explained above by which each library carries a copy of ASDF-loader just in case there is none. All this can be coded on top of the existing ASDF, better factoring the code, and with a bits of magic here and there. In short * Except for standalone software, we a lisp loader that -- Takes into account dependencies -- Searches for the software at standard installation paths * ASDF may coexist with software built using other tools. * ASDF may build software for various distribution purposes. * ASDF can hide itself from the user as much as we want. * ASDF can produce software that does not rely on a configured ASDF to load * We can write libraries that do not care much whether ASDF is there or not * There is always going to be some amount of configuration on the side of the user -- Either using scripts and assuming default installation paths -- Either specifying search paths SBCL_SEARCH_PATH=... * System definitions are useful, dependencies scattered over your sources are not. Juanjo [1] ECL itself [2] Libraries I have to ship to clusters all around the world (AIX, Linux, SGI...) See signature below. -- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com

3 7