Tobias C. Rittweiler wrote: Re: [asdf-devel] Is this necessary in this form? Re: ASDF 1.501
Isn't asdf-output-locations somewhat like a CL pendant to `make install' if you squint your eyes?
Shouldn't that be an integral part of ASDF?
Exactly yes and absolutely no. Not necessarily in that order.
My issue is that there is a great similarity; this similarity is driven by a need inherent in the use of libraries; but ASDF (due to history) has certain things absolutely backwards.
There are many users that a build system must support. - developers do incremental compiles - maintainer creates a release (collect sources, build docs, etc. into a self-contained archive) - maintainer checks that the release actually builds, cleans, installs, etc. - user or system integrator configures the release for local needs - the release is compiled (often into a separate directory or on a separate computer) - the release is tested (before install) - the release is installed (often to a read-only space, as root, or into a temporary directory that will be archived for distribution) - the release is loaded (often without sources) - the release is uninstalled
ASDF needs a coherent story for the person doing each of these jobs. Instead, the current model lumps most everything into a single "load-op", and there is no principled way to perform or configure each of the above phases separately.
My biggest concern with the recent changes is that they build a big scaffold that institutionalizes a design in need of fundamental changes. Some parts of .asd files may need incompatible changes; but I think most uses are completely upgradeable. However, I do not think the recent ASDF proposals will be.
I would like the ASDF architecture to look something like - bootstrap a few (self-contained) core utils using a special install sequence (simple COMPILE, test, and install scripts) - ASDF uses LOAD to pull these in; as each loads, more functionality becomes available (aka SBCL boostrap). - All other packages can now be built with the full ASDF. - (optional) ASDF may upgrade itself without the bootstrap sequence.
ASDF features I would like:
- Standard way to pass configuration options to ASDF for building a specific system. For example, ASDF should let me configure which compression backend cl-pdf uses, and ASDF should write these choices to file for me. This should be a replayable config file that can be modified for reconfiguration, or read in for compilation and install. I should be able to have several builds at once, each with a different set of choices.
- Standard options for each system. For example, a popular one might allow ASDF to autorecompile fasls when the sources change. Another would be an install prefix. Another is to modify the installed system name (e.g. add version info; asdf-1.5 would load the ASDF package, version 1.5). Another to enable the system config cache (below).
- Generic configuration cache, independent of use. In other words, a simple configuration format where I can set key/value pairs. Some of which may happen to control ASDF source locations, others may control ASDF output locations, yet others may be defined by end-user ASDF files. This cache should have three levels, one for the sysadmin, one for the user, and one for inside the system configuration area (usually the source tree). Compare the autoconf site configuration mechanism. http://www.gnu.org/software/autoconf/manual/html_node/Site-Configuration.htm...
- ASDF compile/install spits out a system.fasl. Semantically, loading this causes the entire library to be loaded. On some CL's, it might simply be the concatenation of the compiled source files; on others, it might LOAD them all in order.
The above is a long-enough list for today. I think it covers most of the bases. These things aren't problems I'm creating; they are lessons I have gleaned from GNU autotools and other build systems. Whenever ASDF takes a shortcut, we have issues.
For example, "" # We still need to find a way to precompile libraries # Because asdf is now taking care of placing the fasl file in the correct place, we cannot remove fasls for a specific implementation or library, so we nuke them all. Not optimal. "" - http://pvaneynd.livejournal.com/133491.html
These are part of the standard autotools procedure; they represent a couple use cases from the first list above.
Later, Daniel
P.S. Here's a draft response I penned for an earlier post Subject: Re: [asdf-devel] ASDF 1.501
Next, comes a similar revamp of ASDF-BINARY-LOCATIONS configuration. Or maybe a wholesale replacement of ABL with something that's simpler and configured in a way similar to source-registry? What do YOU think?
What do I really think? Its great that ASDF is getting some TLC, but the whole "register the sources" thing is fundamentally flawed.
From the end-user's perspective, the whole purpose of a "defsystem" (henceforth
library loader) is to get one or more packages into the running lisp.
Dear Daniel,
My issue is that there is a great similarity; this similarity is driven by a need inherent in the use of libraries; but ASDF (due to history) has certain things absolutely backwards.
If you want to fix Lisp build tools in a way that is incompatible with ASDF, then ASDF isn't it. Try XCVB or roll your own. ASDF will remain backwards compatible. Happily, _deepfire's build farm now provides a way to test ASDF backwards compatibility in practice, so we can make incompatible changes to things nobody uses and actually test that they don't break stuff.
From experience I can tell you that precious little of what you
propose will be implemented by anyone but yourself.
If I have a piece of advice, it would be: if you can find a way to write a CL plugin to OMake or Eclipse or GNU Make or autotools, or whatever, instead of recreating a whole build system, you win. I can tell you, recreating a whole build system sucks. And a very lonely job.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] I'm only a stupid AI, but my creator is a real genius!
I must second Faré's response. We are free to modify ASDF, I believe, and to improve it, but with backward compatibility as a boundary condition.
I encourage you to take these ideas --- which seem like good ones --- and invest them either in tool support or into a true ASDF /successor/, such as XCVB.
Alternatively, if you can achieve some of them in the ASDF framework (e.g., improve test-op or add a release-op), more power to you!
Best, Robert
On Wed, 3 Feb 2010, Faré wrote:
From experience I can tell you that precious little of what you propose will be implemented by anyone but yourself.
I understand that. Hence the "dream". But some parts of this are low-hanging fruit. In particular, I strongly recommend a generic config facility rather than the ad-hoc system we seem to be moving towards.
If I have a piece of advice, it would be: if you can find a way to write a CL plugin to OMake or Eclipse or GNU Make or autotools, or whatever, instead of recreating a whole build system, you win. I can tell you, recreating a whole build system sucks. And a very lonely job.
Trust me, we do not want to tie ASDF-N to any of the tools you mention. Autotools is a best-of-breed due to its years of refinement; but it is based on portable shell scripting (ick!). Portable CL is much better. In general, all these build tools are also tailored to other languages; so there is no real way to make a first-class lisp solution.
As for the suckiness of creating a build system, few guess how well I understand.
- Daniel
P.S. Currently higher on my priority list: getting ABLE running on mswinxp.
-
Daniel Herring -
Faré -
Robert Goldman