Dear Vsevolod,
Approximately after the release of ASDF 2 I've got interested in creating yet another system distribution management system on top of ASDF, and that made me investigate ASDF internals. Through that investigation, I've understood 2 things:
- ASDF has support for much more, than is well known and in ordinary use by
most Common Lisp programmers :)
- still it's support for versioning (which is essential to distribution
management) is lacking due to: [...]
Yes, ASDF is lacking in terms of support for versioning. And I admit I am not convinced that extending that support is a good thing. Unless you're going to reimplement and maintain something that's better than existing toolchains, then there's no point at all in the exercise: Lisp users are much better off using existing tools to manage versions than roll their own half-assed mock up. Dpkg or RPM or Nix are very well suited for managing the mix-and-match of many different versions of packages. Git or SVN are very well suited for managing the exact match of package combinations.
I'm not saying that distribution management is useless, or that you shouldn't be coding anything, but rather that there are a lot of interesting problems in the world, and a lot of things that could help the Lisp community, and that it's not obvious that yet another distribution management system is the one most worth of the *huge* cost of doing it right. — And I speak from the experience of trying to turn a simple one-sentence idea of an improvement upon ASDF into software (XCVB).
If I may suggest a simpler, cheaper and more useful strategy for helping with the version management of Lisp software, it would be to automatically produce Debian, RPM or Nix packages from ASDF specifications, and let those tools do the rest of the integration.
Attached is the proposed patch to ASDF, that, in my view, solves the aforementioned versioning problems. It does the following:
- Unifies internal version representation format in the form of a list of
integers: (for example, ASDF 2.106 will have an internal version of '(2 106 0), while ASDF 2.005 — '(2 5 0)).
Why only three integers? While you're at it, why not accept the whole range of Debian or RPM version comparisons?
A final note about versioning: I needed to use pre-reading of ASD files in order to know their versions without arising version conflicts. This works, except for the fact, that READ-EVAL is turned off in the process, so such version specifiers as:
- :version #.*hunchentoot-version*
- :version #.(with-open-file (vers (merge-pathnames "version.lisp-expr"
*load-truename*)) (read vers)) do not produce expected results and are treated as empty versions. There will be a need to make a note about that to library developers.
If we ever go this way, it might be better to extend ASDF to accept reading versions from file, either by READing an object, or by READ-LINEing it.
Besides, an incompatible change is introduced to FIND-SYSTEM. The ERROR-P optional argument is removed, so plain NIL is unconditionally returned, when system is not found, and VERSION and VERSION-P optional arguments are added instead.
This sounds very bad to me. What is the use case? How does that affect the semantics of find-system? Can there now be many instances of a system in the source-registry? Does find-system now have to grovel for *all* these instances, load the .asd files to determine the version and pick the most recent one by default? That's C R A Z Y.
Note that the current interface of find-system seems to be copied from that of find-class.
The way I see things is that whoever sets up the source-registry thereby specifies which versions of each piece of software is going to be used, at which point version management can be taken completely out of ASDF and into the hands of whoever specifies said source-registry.
If you want to check that what is configured is correct, you can check versions. And e.g. POIU, XCVB or ASDF-DEPENDENCY-GROVEL do have such consistency checks. But that's as much as you need.
I have supplied a number of unit tests to the functions, that I have added. This is done with a very simple unit testing library of my own (a microframework so to say) MUTEST, that can be found at: http://github.com/vseloved/mutest/.
I'd rather ASDF has only one test framework, if possible one with few dependencies if at all. The current one kind of sucks but works so far. I'm not against using a different one, but someone should then do the conversion of existing tests.
I have also done functional testing of the changes with the systems, that I have got installed already (around 70 libraries), and have verified, that there are no problems. At the same time I would not claim, that there will not be any problems with some system definitions, that use obscure ASDF features (although, I have acquainted myself with ASDF to the point, that I quite well understand most of the features as well as internals now). So I ask anyone interested in this set of changes to thoroughly test them.
I think that's a good strategy, and one many people have been following lately. Maybe there's a way to automate and share these things further.
Finally, the patch can be applied by simply loading ASDF-VERSION.LISP on top of the loaded ASDF 2.106.
I hope, this work proves useful and is integrated into ASDF. Anyway I'm willing to answer any questions and/or improve the implementation, if needed.
I admit this is not on top of my priority list. I'll try to review your code in detail later this week. Ideally, a new ASDF maintainer would step forward and replace me.
Thanks for your efforts,
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Most economic fallacies derive... from the tendency to assume that there is a fixed pie, that one party can gain only at the expense of another. — Milton Friedman