OK, so ASDF 1.711 is officially our release candidate. Consider the
git master frozen for anything but bug fixes and documentation. Please
use branches for any other development.
My next task is to find someone responsible for every implementation to
1- test it on each supported OS
2- actually commit it downstream
Below is the message I want to send to each implementation's
vendor support mailing-list.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
The naturalistic fallacy: "if it's natural, it's good."
The anti-naturalistic fallacy: "if it's natural, it's bad."
The a-naturalistic fallacy: "nature has no relationship to good and bad."
------>8------>8------>8------>8------>8------>8------>8------>8------>8------
Dear <vendor developers>,
could you test the current release candidate of ASDF 2 (version 1.711)
with your implementation, send us fixes for any glaring bug, and
otherwise include it in any alpha-testing release channel for users
of your implementation? I think it's mostly ready, and I'd like to
have it widely tested before actual release.
ASDF 2 is an evolution of ASDF that tries to be backwards compatible
with previous releases and their many extensions, while at the same time
improving the user experience.
ASDF 2 is in pre-release mode. Hopefully we'll release it in May 2010
without any code change (except maybe for some tweaks to enhance support
for some Lisp implementations on some operating systems), and
with slightly enhanced documentation (notably based on your feedback).
I'm using it ITA; recent versions are already included in ECL and ABCL;
several hackers have tried building plenty of packages with it.
While it's far from perfect and only a relatively small evolution,
I believe it's already notably better enough than the previous generation
of ASDF that I'd like to get it out to the world, and doesn't have any
egregious regression from ASDF 1.
What has changed since ASDF 1 is documented at the following page:
http://common-lisp.net/project/asdf/asdf/FAQ.html
Improvements include:
* portably handle naming of files wrt directory, type, host/device, etc.
* a builtin user-configurable output translation mechanism,
superseding asdf-binary-locations and previous hacks in
common-lisp-controller and cl-launch.
* a user-configurable source registry system for finding systems
(the old *central-registry* is still available)
* improved portability to many implementations.
* improved support for Windows (though configuration is still lacking
implementation-dependent support except on LispWorks)
* many bug fixes
* the test suite while still largely incomplete covers more cases
and is more robust across implementations.
* notable performance fixes for large systems with thousands of components
* API improvements: load-system, system-relative-pathname, etc.
* better documentation
* a sensible versioning system for ASDF 2 itself, that can be now be
upgraded in a running system. This allows in the future to more easily
decouple the releases of ASDF and those of your implementation.
* better support various extensions as used by ECL and ABCL,
and also by POIU, ASDF-DEPENDENCY-GROVEL.
Main pitfalls include:
* Output translations is enabled by default. This may surprise some users,
most of them in pleasant way (we hope), a few of them in an unpleasant way.
It is trivial to disable output translations.
* Some systems in the large have been known not to play well with
asdf output translations. They were easy to fix.
It is also easy to disable output translations or override its configuration.
* ASDF 2 output translations do not work with ASDF-Binary-Locations. They
replace A-B-L, and include a compatibility mode to emulate your previous
A-B-L configuration. But you shouldn't load ABL on top of ASDF 2.
* A notable performance bug on SBCL: (directory "**/*.asd") can make things
slower than desired when initially searching for asd files (depending on
your configuration).
* On Windows, when not using cygwin, only LispWorks has proper defaults
for configuration pathnames. Other implementations make do. Patches welcome.
Windows support is largely under-tested, too.
As to how to best include ASDF in your implementation,
here is an excerpt from the FAQ section of our manual:
12.3.2 “I'm a Common Lisp implementation vendor.
When and how should I upgrade ASDF?”
Starting with current candidate releases of ASDF 2, it should always
be a good time to upgrade to a recent version of ASDF. You may consult
with the maintainer for which specific version they recommend, but the
latest RELEASE should be correct. We trust you to thoroughly test it
with your implementation before you release it. If there are any
issues with the current release, it's a bug that you should report
upstream and that we will fix ASAP.
As to how to include ASDF, we recommend the following:
* If ASDF isn't installed yet, then (require :asdf) should load the
version of ASDF that is bundled with your system. You may have it load
some other version configured by the user, if you allow such
configuration.
* If your system provides a mechanism to hook into CL:REQUIRE, then it
would be nice to add ASDF to this hook the same way that SBCL, CCL and
ABCL do it.
* You may, like SBCL, have ASDF be implicitly used to require systems
that are bundled with your Lisp distribution. If you do have a few
magic systems that come with your implementation in a precompiled way
such that one should only use the binary version that goes with your
distribution, like SBCL does, then you should add them in the
beginning of wrapping-source-registry.
* If you have magic systems as above, like SBCL does, then we
explicitly ask you to NOT distribute asdf.asd as part of those magic
systems. You should still include the file asdf.lisp in your source
distribution and precompile it in your binary distribution, but
asdf.asd if included at all, should be secluded from the magic
systems, in a separate file hierarchy, or you may otherwise rename the
system and its file to e.g. asdf-ecl and asdf-ecl.asd, or sb-asdf and
sb-asdf.asd. Indeed, if you made asdf.asd a magic system, then users
would no longer be able to upgrade ASDF using ASDF itself to some
version of their preference that they maintain independently from your
Lisp distribution.
* If you do not have any such magic systems, or have other non-magic
systems that you want to bundle with your implementation, then you may
add them to the default-source-registry, and you are welcome to
include asdf.asd amongst them.
* Please send us upstream any patches you make to ASDF itself, so we
can merge them back in for the benefit of your users when they upgrade
to the upstream version.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
Merely having an open mind is nothing; the object of opening the mind,
as of opening the mouth, is to shut it again on something solid.
— G.K. Chesterton
I'd like to freeze the main branch of ASDF tomorrow for anything but
bug fixes and documentation updates.
Is there anything any of you want to commit before the freeze? Or as
an exception to the freeze.
I'm thinking in particular of the stuff in asdf-ecl. Juanjo, do you
think it should be merged with ASDF itself? Especially in the light of
it being semi-useful on other platforms where fasls can be
concatenated? It might make things nicer if there is only one file
needed for ASDF.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
Be humble, for you are made of earth. Be noble, for you are made of stars.
— Serbian proverb
> (asdf::ensure-source-registry)
One and only one of :INHERIT-CONFIGURATION or
:IGNORE-INHERITED-CONFIGURATION is required
I had obviously forgotten one of those keywords.
(:source-registry
(:tree "/Users/jjgarcia/src/lisp/"))
Why does not the missing keyword get interpreted as :IGNORE-...?
Juanjo
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
Dear all,
I think time has come to declare ASDF 2.0 in beta-test, and focus on
releasing it before we get more features into it.
If there are any bug fixes or improvements you think should definitely
be in 2.0, please open a bug in launchpad and tag them as such
(milestone: version 2), or otherwise make it prominent in your message
on this mailing-list.
I'd like to declare 2.0 frozen by the end of April 2010, and released
by the end of May 2010.
PS: after 2.0 is out, I'd like to resign from ASDF maintainership, to
focus on XCVB (help welcome on that, too). Is anyone volunteering to
replace me on ASDF?
Juanjo and janderson both have had very good ideas, although in both
cases, I think that there's a gap from the good idea to something that
can be released as part of the build tool that everyone will use.
Maybe we should start alternative branches on the git tree?
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
Being a graduate student is like becoming all of the Seven Dwarves.
In the beginning you're Dopey and Bashful. In the middle, you are
usually sick (Sneezy), tired (Sleepy), and irritable (Grumpy). But
at the end, they call you Doc, and then you're Happy.
— Ronald T. Azuma, http://www.cs.unc.edu/~azuma/hitch4.html
It is a two-lines change that allows a defsystem form depend on a package
that provides additional functionality, such as new system classes, new
system options, new methods, etc.
:ASDF-DEPENDENCIES ({system-name}*)
fee.asd:
(defsystem :fee
:components ((:file "fee")))
foo.asd:
(defsystem :foo
:tests ((:system :faa))
:asdf-dependencies (:fee)
:components ((:file "foo")))
> (load "asdf")
;;; Loading #P"/Users/jjgarcia/devel/asdf/asdf.fas"
;;; Loading #P"/Users/jjgarcia/lib/ecl-10.3.1/CMP.fas"
;;; Loading #P"/Users/jjgarcia/lib/ecl-10.3.1/sysfun.lsp"
#P"/Users/jjgarcia/devel/asdf/asdf.fas"
> (asdf:load-system :foo)
; loading system definition from /Users/jjgarcia/devel/asdf/foo.asd into
; #<ASDF0 package>
;;; Loading "/Users/jjgarcia/devel/asdf/foo.asd"
; loading system definition from /Users/jjgarcia/devel/asdf/fee.asd into
; #<ASDF1 package>
;;; Loading "/Users/jjgarcia/devel/asdf/fee.asd"
; registering #<SYSTEM :FEE 11882808> as FEE
;;; Loading
"/Users/jjgarcia/.cache/common-lisp/ecl-10.3.1-darwin-x86/Users/jjgarcia/devel/asdf/fee.fas"
;;;
;;; Loaded prerequisite FEE
;;;
; registering #<SYSTEM :FOO 12258736> as FOO
;;; Loading
"/Users/jjgarcia/.cache/common-lisp/ecl-10.3.1-darwin-x86/Users/jjgarcia/devel/asdf/foo.fas"
#<ASDF:LOAD-OP NIL 9826760>
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
I know there are bug reports in launchpad, and that this has been expelled
from 2.0 as of not urgent fix, but seems the latest changes have broken it
beyond fix.
I was playing with the following system (yes, I know the syntax is wrong),
to make :feature at some point more useful but a simple experiment reveals
that the structure dependency is being pushed into the list returned by
COMPONENT-DEPENDS-ON and that this list causes an error in
COMPONENT-SELF-DEPENDENCIES.
(defsystem :foo
:components
((:file "foo1" :depends-on ((:feature :foo-method1)))
(:file "foo2" :depends-on ((:feature (not :foo-method1))))))
* (asdf:load-system :foo)
[...]
; file: /Users/jjgarcia/devel/asdf-decl/examples/feature/../../asdf.lisp
(CERROR "Continue nonetheless."
"Congratulations, you're the first ever user of FEATURE
dependencies! Please contact the asdf-devel mailing-list.")
0] (continue)
debugger invoked on a TYPE-ERROR:
The value (:FEATURE :FOO-METHOD1)
is not of type
(OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING SYMBOL CHARACTER).
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(STRING= "foo1" (:FEATURE :FOO-METHOD1))[:EXTERNAL]
0]
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
The attached patch adds a new option to DEFSYSTEM. This option is
:TESTS test-list
test-list = (:system system-name) | function-name
system-name = keyword | string
function-name = symbol | string
The function name is a string designator that denotes a function with no
arguments to be called. This string will be read using the standard I/O
syntax only when the test is executed. This ensures that the mechanism works
once the appropriate dependencies are loaded.
The system name denotes a system that will be loaded when the TEST-OP
operation is invoked.
The patch can be improved with some tests done when executing DEFSYTEM
instead of what it does now, which is to signal an error once the test is
performed.
The usual example follows.
foo.asd:
(defsystem :foo
:tests ((:system "faa"))
:components ((:file "foo")))
faa.asd:
(defsystem :faa
:tests ("faa::run-tests")
:components ((:file "faa")))
foo.lisp:
(defpackage "FOO")
(in-package "FOO")
(defun my-cos (a)
(cos a))
faa.lisp:
(defpackage "FAA")
(in-package "FAA")
(defun run-tests ()
(format t "~&;;;~%;;; MY-COS signals errors: ~A~%;;;"
(handler-case (progn (foo::my-cos 'a) 'no)
(error (c) 'yes))))
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
Right now we have a situation in which load-source-op is a second class
citizen, rarely used, and for which nobody provides explicit dependencies.
More precisely
* Some compilers out there are just efficient compiling and loading source
files. The operation is the same.
* LOAD-OP can be viewed as a slightly more efficient version of an operation
that loads sources.
* Some users out there do not like compile-before-load procedure.
* ASDF makes no promise that load-source-op will behave similarly as
load-op.
* Users define methods for load-op and not for load-source-op.
* In practice the dependencies *must* be the same, for the philosophy of CL
is that compiled and not compiled files behave similarly.
* The only differences should be enforced to lay just in the compilation
process.
I attach a patch which merges both operations, leaving a trace of
compatibility.
* There is only one class, LOAD-OP.
* The class has the promise that it will use the same dependencies when
loading sources and binaries.
* When imposing that one does compile the source before, the class issues an
additional dependency on a compile-op, which may pull additional
dependencies itself.
* A global flag, *compile-before-loading* determines the overall behavior
* A new function ASDF:LOAD-SOURCE-SYSTEM forces no compilation
* The function LOAD-SYSTEM gets a flag, :compile that sets
*compile-before-loading*
* With these changes it is easy to get a sorted list of the components that
make up a program. Furthermore, this list is guaranteed to work because we
are using the same operation that we will use to load the files.
(pprint
(loop with asdf::*compile-before-loading* = nil
for (op . component)
in (asdf::traverse (make-instance 'load-op)
(find-system "chunga"))
collect (list (type-of op) (type-of component)
(asdf::component-name component))))
* With this mechanism I can reimplement the fasl concatenation mechanism
*without* using classes at all. Same goes for ECL's extensions.
The patch is still not fully tested, but I submit it for discussion, so that
ugly corners can be polished.
Juanjo
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
Who uses it?
Out of 291 packages, the only uses seem to be
- Defining TEST-OP operations (greate majority)
- In a few tests, it solves a wrong ordering of files
- In two, it deactivates operations by using FEATURE (obscure way to do it)
The "paradigmatic" example of McClim cited here
http://common-lisp.net/project/asdf/asdf.html#Common-attributes-of-componen…
does not have a single occurrence of this word.
The question is thus, is it really needed?
Juanjo
--
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com