Dear James,
before one can declare asdf 2.0 to be feature complete, one should have a specification of its intended function and then gauge the implementation in those terms.
I humbly disagree. The standards that I have set to myself for ASDF 2.0 are as follows:
1- improve the ASDF interface * provide portable ways to do things that could not be done portably * have no regressions on what was previously portable * have a clear better way for anything that used to work widely in practice but was non-portable
2- improve the ASDF implementation * fix bugs, clean up internals * performance: use hash-tables instead of linear search assoc lists * make all existing tests pass everywhere, add a few tests.
3- make ASDF versioned and upgradable * make it easy to detect if you're using a recent-enough ASDF * provide a clear version 2.0 for CL implementations to provide * decouple _future_ release cycles of ASDF and CL implementations
4- make ASDF user-configurable * allow for configuration through user-controlled files separate from developer-controlled build scripts * provide early input validation for user-controlled files * still allow developers to easily override when needed, and support previous ways of configuring ASDF.
5- improve the documentation * update the documentation without introducing any regression * document enough so a newbie can survive * cleanup the manual, update the website
EXPLICITLY NOT INCLUDED amongst my goals for ASDF 2.0, are the following goals that a future maintainer may or may not include as his goal for a future release:
* make the ASDF interface fully declarative. * make useful but backwards-incompatible changes. * support the distribution of common extensions (contrib, etc.) * have the documentation cover 100% of the interface. * have tests cover 100% of code. * keep ASDF under any specific code size.
while the users' manual does explain aspects of the user and/or programmer interface, in addition to the descriptions of the interface, there should be something which describes the requirements which are being beta-tested - both to assess completeness and to inform future maintenance.
This is too abstract for me. If there are specific elements you think should be documented, please list them concretely.
One way you could do it is to send a patch of a first draft of a modification of the manual that you require, with holes where you want me to fill information.
the additions to the code base over the past few months were devoted to two areas:
- configuration
- output translations
i neglect, for the moment, the configuration component of the asdf evolution and concentrate on questions related to output translations. even though the configuration mechanism contributes settings which affected the latter, they are in sufficiently independent, that an isolated analysis is reasonable.
Well, most of the translation code *IS* the configuration system. The output-files :around method itself is small, and its helpers are relatively simple, except for the ugly directorize-pathname-host-device required for portability (that none of the previous solutions provided).
in lieu of a specification, several asdf-devel messages on the topic of output translations[1--5] discuss requirements. the asdf-binary- location users' guide includes a "what it does" passage[6]. the asdf- binary-locations-devel archives[7] include no additional discussions. to follow those discussions, an output translation mechanism should make it possible to
- place (output) files under a single root
- place (output) files in a tree specific to (runtime x version x
project x file-type)
- include process information (eg, pid or user id) in the location
- specify elements of the mapping programmatically
- specify elements of the mapping by configuration
- combine independent specifications
- express location maps both globally and in connection with specific
sub-systems (cl-controller, cl-launch, etc)
- support simultaneous, independent mapping for binary output and
ancillary data files
- disable translations for a given path
- disable translations completely
in response to these requirements, "asdf binary locations" was developed, first as an extension to, then integrated with asdf, and further developed into a general output translation facility. while the evolution brings definite benefits, in the process, between version 1.362, when a-b-l was incorporated into asdf core, and the current +/- 1.7, the source size has increased from 56K to 132K, the function count has grown from less than one hundred to more than two hundred, and the call count has almost tripled.
Yes. Configuration support has doubled the size of ASDF, and added features, cleanups and robustification have also increased its size. I don't think any of this stuff can be done without.
the call graphs[8] depict the situation. each node in the graph corresponds to a function in the asdf implementation, each edge to a call. the green nodes were present in 1.362, the yellow nodes in 1.502, and red nodes in 1.7. filled nodes have remained, while empty nodes indicate functions which are no longer present. the increase in code mass and complexity is evident. since, as expressed, the requirements could be met by logical pathname translations, the situation causes concern. there is nothing in the functional requirements, as such, which dictates an implementation of this complexity.
I don't think the implementation is particularly complex. It is larger because it does more things.
Make things as simple as possible BUT NO SIMPLER.
Show me which function you'd like to see gotten rid of, and I'll explain you why it is necessary - or get rid of it.
there are, however, non-functional concerns which appear to have informed the decision: it has been held, that
- logical pathnames are not portable;
- the facility must operate in terms of physical pathnames as real-
world lisp applications require file names which do not conform to logical pathname syntax constraints.
Yes.
in the course of testing the output location mapping itself, it was demonstrated[9] that the non-portability claims were unfounded. an experiment based on 1.502 demonstrated[10] that a mechanism which separates location translation from truename processing can provide the required functionality with much less code in terms of logical pathnames despite non-conformant physical pathnames. the logical pathname based location maps were then re-implemented in 1.702 and further experiments demonstrated that - with changes to the scripts to account for functions which would be removed, all regression tests pass, and that with appropriate initialization, a multi-runtime build is supported for the normal abcl/acl/ccl/clisp/cmucl/ecl/lw/sbcl complement. the following host definition, for example, produces a centralized per-runtime translation tree for output files.
(let ((root (make-pathname :directory (butlast (pathname-directory *build-init-pathname*) 2) :name nil :type nil :defaults *build-init- pathname*)) (host "SOURCES")) (setf (logical-pathname-translations host) '(("**;*.*" "**;*.*"))) (let ((lbin (pathname-type (compile-file-pathname (make- pathname :host host :name "TEST" :type "LISP")))) (bin (pathname-type (compile-file-pathname "test.lisp")))) (setf (logical-pathname-translations host) `((,(make-pathname :host host :directory '(:absolute :wild- inferiors) :name :wild :type lbin :version :wild) ,(make-pathname :directory `(,@(pathname-directory root) "bin" ,(asdf::implementation-identifier) :wild-inferiors) :name :wild :type bin :version :wild)) (,(make-pathname :host host :directory '(:absolute :wild- inferiors) :name :wild :type :wild :version :wild) ,(make-pathname :directory `(,@(pathname-directory root) :wild-inferiors) :name :wild :type :wild :version :wild)))))
(setq asdf::*translations-root* (make-pathname :host host :directory '(:absolute) :name nil :type nil)))
given which, the question occurs, whether the output locations implementation deserves further thought before it advances to beta status?
Less code that what? How is the end-user meant to deal with configuration? The horribly complex logical pathname translation above is not what I would like to have to explain to users in the manual.
If you want people to adopt logical pathnames -- get the spec fixed, and convince implementers (notably SBCL implementers) to adopt your fixed spec.
And even then you'll have a hard time fixing things that users can now trivially do with the following translation:
(:root ("C:/my/cl/cache" :implementation))
This will work and make his file in \remote-host\myshare\cl\cl-foo\foo-V2.000\bar_baz.lisp be compiled as obvious from the spec into C:\my\cl\cache\sbcl-1.0.38-windows-x86\remote-host\myshare\cl\cl-foo\foo-V2.000\bar_baz.fasl He won't have to understand the innards of any magic logical pathname host mapping, and he won't have to write or decode such mapping tha may map it to wherever some weirdly named file TEST:CL-FOO;FOO-V2-000-GENERATED-12345;BAR-BAZ-GENERATED-45678.FASL may point.
[1] : http://common-lisp.net/pipermail/asdf-devel/2010-March/000943.html [2] : http://tinyurl.com/yyv2o4w [3] : http://common-lisp.net/pipermail/asdf-devel/2010-February/ 000691.html [4] : http://common-lisp.net/pipermail/asdf-devel/2010-February/ 000699.html [5] : http://common-lisp.net/pipermail/asdf-devel/2010-February/ 000702.html [6] : http://common-lisp.net/project/asdf-binary-locations/user- guide.html [7] : http://common-lisp.net/pipermail/asdf-binary-locations-devel/ [8] : http://reports.setf.de/asdf-versions-twopi.svg, http://reports.setf.de/asdf-versions-twopi.pdf, http://reports.setf.de/asdf-versions-dot.svg, http://reports.setf.de/asdf-versions-dot.pdf [9] : http://tinyurl.com/y7jmzee : the test cases are present in dual form, one for a physical pathname and once for a logical pathname [10] : http://common-lisp.net/pipermail/asdf-devel/2010-April/ 001351.html
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] License Agreement: By reading this message, you agree to run around the room which you are currently in, flapping your arms, and squawking like a chicken.