On 2010-04-21, at 04:00 , Faré wrote:
Dear James,
before one can declare asdf 2.0 to be feature complete, one should have a specification of its intended function and then gauge the implementation in those terms.
I humbly disagree. The standards that I have set to myself for ASDF 2.0 are as follows:
[ ... which list of goals should well appear in release notes, but does not address the concern raised in this thread, except in the exclusion, that ... ]
EXPLICITLY NOT INCLUDED amongst my goals for ASDF 2.0, are the following goals that a future maintainer may or may not include as his goal for a future release:
- [ ... ]
- keep ASDF under any specific code size.
were one to be in a position to consider responsibility for the future maintenance of the code, that item would appear quite as far down on the list.
while the users' manual does explain aspects of the user and/or programmer interface, in addition to the descriptions of the interface, there should be something which describes the requirements which are being beta-tested - both to assess completeness and to inform future maintenance.
This is too abstract for me. If there are specific elements you think should be documented, please list them concretely.
i will help you.
there is a section in the asdf manual called "Controlling where ASDF saves compiled files". that section describes how a user can configure asdf to translate output files.
there should be two additional sections in a manual, perhaps a developers' manual, called "output translation requirements" and "output translation implementation" which describe - independent of how to use the facility, what it does and how it is implemented. a "requirements" section should describe at least - what kinds of output translations are required. it should do this in terms of abstract characteristics, specific use cases, and enumerative examples; - the operating systems and the project forms to be supported by asdf with attention to the required performance; - what trade-offs are apparent in supporting which functional features, which options were elected, and why; - an abstract model for the entities which the implementation is expected to represent, support, manipulate.
an "implementation" section should provide an overview of the one hundred twenty functions and twenty parameters which have been introduced in since version 1.362, with attention to how their relation to the functional requirements. section "8.6" in the asdf manual reads as if it might have been intended as notes for a section of this sort.
if this is still too abstract, any one of the readily available descriptions of "functional requirements", eg, here[1], might may help you to understand their purpose and construction.
One way you could do it is to send a patch of a first draft of a modification of the manual that you require, with holes where you want me to fill information.
the additions to the code base over the past few months were devoted to two areas:
- configuration
- output translations
i neglect, for the moment, the configuration component of the asdf evolution and concentrate on questions related to output translations. even though the configuration mechanism contributes settings which affected the latter, they are in sufficiently independent, that an isolated analysis is reasonable.
Well, most of the translation code *IS* the configuration system. The output-files :around method itself is small, and its helpers are relatively simple, except for the ugly directorize-pathname-host- device required for portability (that none of the previous solutions provided).
in lieu of a specification, several asdf-devel messages on the topic of output translations[1--5] discuss requirements. the asdf-binary- location users' guide includes a "what it does" passage[6]. the asdf- binary-locations-devel archives[7] include no additional discussions. to follow those discussions, an output translation mechanism should make it possible to
- place (output) files under a single root
- place (output) files in a tree specific to (runtime x version x
project x file-type)
- include process information (eg, pid or user id) in the location
- specify elements of the mapping programmatically
- specify elements of the mapping by configuration
- combine independent specifications
- express location maps both globally and in connection with specific
sub-systems (cl-controller, cl-launch, etc)
- support simultaneous, independent mapping for binary output and
ancillary data files
- disable translations for a given path
- disable translations completely
is this list complete? if not, it would help if you would complete it. it or something equivalent should appear in a "requirements" section.
in response to these requirements, "asdf binary locations" was developed, first as an extension to, then integrated with asdf, and further developed into a general output translation facility. while the evolution brings definite benefits, in the process, between version 1.362, when a-b-l was incorporated into asdf core, and the current +/- 1.7, the source size has increased from 56K to 132K, the function count has grown from less than one hundred to more than two hundred, and the call count has almost tripled.
Yes. Configuration support has doubled the size of ASDF, and added features, cleanups and robustification have also increased its size. I don't think any of this stuff can be done without.
the call graphs[8] depict the situation. each node in the graph corresponds to a function in the asdf implementation, each edge to a call. the green nodes were present in 1.362, the yellow nodes in 1.502, and red nodes in 1.7. filled nodes have remained, while empty nodes indicate functions which are no longer present. the increase in code mass and complexity is evident. since, as expressed, the requirements could be met by logical pathname translations, the situation causes concern. there is nothing in the functional requirements, as such, which dictates an implementation of this complexity.
I don't think the implementation is particularly complex. It is larger because it does more things.
Make things as simple as possible BUT NO SIMPLER.
Show me which function you'd like to see gotten rid of, and I'll explain you why it is necessary - or get rid of it.
the patch which was enclosed in the earlier message[2] demonstrated an implementation which modifies the three functions - component- pathname, input-files, and output-files, extends the function which constructs the component absolute pathname, and adds a function to map between logical and physical pathname components. with respective calls from the other five functions. without having deconstructed the asdf 1.7* implementation to distinguish which functions relate to which functional components, on first order, five is five percent one hundred and twenty and an even smaller percentage of two hundred.
there are, however, non-functional concerns which appear to have informed the decision: it has been held, that
- logical pathnames are not portable;
- the facility must operate in terms of physical pathnames as real-
world lisp applications require file names which do not conform to logical pathname syntax constraints.
Yes.
in the course of testing the output location mapping itself, it was demonstrated[9] that the non-portability claims were unfounded. an experiment based on 1.502 demonstrated[10] that a mechanism which separates location translation from truename processing can provide the required functionality with much less code in terms of logical pathnames despite non-conformant physical pathnames. the logical pathname based location maps were then re-implemented in 1.702 and further experiments demonstrated that - with changes to the scripts to account for functions which would be removed, all regression tests pass, and that with appropriate initialization, a multi-runtime build is supported for the normal abcl/acl/ccl/clisp/cmucl/ecl/lw/sbcl complement. the following host definition, for example, produces a centralized per-runtime translation tree for output files.
(let ((root (make-pathname :directory (butlast (pathname-directory *build-init-pathname*) 2) :name nil :type nil :defaults *build-init- pathname*)) (host "SOURCES")) (setf (logical-pathname-translations host) '(("**;*.*" "**;*.*"))) (let ((lbin (pathname-type (compile-file-pathname (make- pathname :host host :name "TEST" :type "LISP")))) (bin (pathname-type (compile-file-pathname "test.lisp")))) (setf (logical-pathname-translations host) `((,(make-pathname :host host :directory '(:absolute :wild- inferiors) :name :wild :type lbin :version :wild) ,(make-pathname :directory `(,@(pathname-directory root) "bin" ,(asdf::implementation-identifier) :wild-inferiors) :name :wild :type bin :version :wild)) (,(make-pathname :host host :directory '(:absolute :wild- inferiors) :name :wild :type :wild :version :wild) ,(make-pathname :directory `(,@(pathname-directory root) :wild-inferiors) :name :wild :type :wild :version :wild))) ))
(setq asdf::*translations-root* (make-pathname :host host :directory '(:absolute) :name nil :type nil)))
given which, the question occurs, whether the output locations implementation deserves further thought before it advances to beta status?
Less code that what? How is the end-user meant to deal with configuration? The horribly complex logical pathname translation above is not what I would like to have to explain to users in the manual.
the irony in a situation, where a doubled code size and tripled call count are insignificant while a two-clause logical host definition is incomprehensible, is appreciable, but not significant. the matter at issue is not whether you can read logical host specifications, but whether the attendant translation mechanism would simplify asdf implementation.
If you want people to adopt logical pathnames -- get the spec fixed, and convince implementers (notably SBCL implementers) to adopt your fixed spec.
we can discuss the general utility of logical pathnames at length, at your leisure. elsewhere. my enquiry concerns whether logical pathname translations would serve as a more compact implementation mechanism for asdf output translations than the present method.
And even then you'll have a hard time fixing things that users can now trivially do with the following translation:
(:root ("C:/my/cl/cache" :implementation))
This will work and make his file in \remote-host\myshare\cl\cl-foo\foo-V2.000\bar_baz.lisp be compiled as obvious from the spec into C:\my\cl\cache\sbcl-1.0.38-windows-x86\remote-host\myshare\cl\cl- foo\foo-V2.000\bar_baz.fasl
this would appear to be the equivalent of a translation on the order of
`("**;*.*.* ,(concatenate 'string "C:/my/cl/cache/" (asdf::implementation-identifier) "/**/*.*.*"))
but i may have misunderstood the requirement.
--- [1] : http://en.wikipedia.org/wiki/Functional_requirement [2] : http://common-lisp.net/pipermail/asdf-devel/2010-April/001351.html