On 6/12/11 Jun 12 -4:34 AM, james anderson wrote:
good morning, faré, gentlemen;
On 2011-06-12, at 02:25 , Faré wrote:
TL;DR: ASDF currently doesn't play so well with logical pathnames. I'm seeking advice on how to make things better — or on whether it's worth the trouble.
Dear all,
[...]
POTENTIAL SOLUTIONS
[...]
Finally, I could do nothing, and assume that people who go through the pain of setting up logical pathnames before they run ASDF can just as well setup the *central-registry* and have LPNs be recognized in the good old way. If you use the source-registry, your LPNs will be squashed, and nothing will break in the code since code works fine with PPNs, just like most everyone use it, but your debug information will be stored according to the PPN, not the LPN. Meh.
less might well be better.
a bit over twenty years ago, when i had sent a library to john mallory to be included among cl-http contributions, he pointed out to me, that, as my apparent intent was to implement portable systems for others' use, it was not a good idea to have used mixed case pathnames. i trusted him, renamed several dozen files and directories, and have adhered to that restriction ever since. over the years, i have seen nothing to contract his advice. furthermore, over these years, as my requirements for lisp system management tools have remained limited to those related to building those aspects of larger systems which comprise just lisp source code, i have observed that portability and maintainability are improved by adopting the logical pathname naming restrictions for system description components. that is, in distinction to experience reported elsewhere, it has been possible to use asdf system declarations which contain just logical pathname components with all lisp implementation which purported to support logical pathnames. the pathname regression test which i implemented during the "asdf 2" testing process demonstrated this to have been true for all of the variations of runtime and pathname expression which were known at that time.
This is an interesting bit of explanation and somewhat dovetails with my experience. One of the problems I have had with logical pathnames is working on projects where lisp code is /not/ in systems "compris[ing] just lisp source code." Larger systems on which I have worked often contain large hunks of Java, C++ or scripting language code. Sometimes this software has been written across multiple companies. It has been difficult to limit those systems' use of pathnames; in practice it's not worth the trouble. It's not helpful when trying to get people to tolerate your use of CL to have to tell them that you want them to give up TheirCamelCase and_underscores. Another thing that has led me to give up on logical pathnames is that it has forced me to debug multiple colleagues' lisp configurations. I have found it less work to simply put in our source code repository load files that use *load-truename* to populate asdf:*central-registry*. This even hides ASDF configuration, and minimizes time lost in debugging others' configurations. These experiences may differ from much of this audience's experience, which may be more related to individual hackers working on their own software, or working on open source systems in a distributed way.
my particular build process - even prior to "asdf 2", involves variations on the two alternatives discussed in faré's original message [cf. https://github.com/lisp/de.setf.utility/blob/master/asdf/hierarchical-names....]:
- it searches for system definition files recursively from the set of root directories specified in *central-registry*. for my own systems, a method to interpret system names as hierarchic paths speeds the process. this means that the *central-registry* is sufficient.
I use a variant of this, a portable "asd-finder" system that does a special, portable CL version of 'find' that is tailored to find .asd files. The asd-finder is used to populate asdf:*central-registry*. Using the asd-finder is similar to using Faré's :tree directive, except it does not rely on knowing the user's absolute install pathname: you can invoke the asd-finder using *load-truename*. My desire to replicate this kind of configuration (the asd-finder used in a load file) is what led me to introduce the :here directive into Faré's DSL for configuring input file search.
- it takes the "hard way out", as that serves maintainability. where possible, an additional step in the system initialization protocol derives and asserts a logical pathname root for the system, which (at least at last check) yields logical source pathnames for all system components.
I used to do this myself, but gave up on it. If you are working relative to some root directory found through *load-truename* (or a similar mechanism), introducing the logical pathname didn't seem like a big win to me. If my systems need to find some files, it's just as easy to store a copy of the directory portion of *load-truename* in a global variable, and use merge-pathnames on the global variable. In practice, I have found this more reliable than wrestling with logical pathnames (e.g., there's no need to worry that someone has installed the working directory in some location that will break logical pathnames). (merge-pathnames "data-file.csv" *my-system-root*) doesn't seem a lot harder than "my-system-root:data-file.csv" YMMV.
this approach does limit the mechanisms available to include external libraries in my own builds. that is ok. for example, while i may use something like quicklisp to obtain source code, i have seen the difficulties which even advanced dependency-based build systems, such as ruby's, introduce into production environments. it is a nightmare to which i do not intend to submit. i prefer, that production systems always include self-curated source code. in this case, it is a minor issue to normalize system names.
My experience lines up with James's here. If a project I work on needs an external library, we blow that into our "lisp-utils" repository, and incorporate that. No more problems with API-breaking mods to libraries. Also, in my experience, CL libraries very often have corner cases which break them. After a bunch of times downloading a library using ASDF-install, finding I had to patch it, and /then/ having to blow it into a source code repository, I find that it's better to just cut to the chase and push all library dependencies into the repository. Again, YMMV. If you want to set up a web server with hunchentoot, for example, you may find that library has been used enough that it can be collected with quicklisp or asdf-install and simply used. OTOH, if you are using, say, and XML-wrangling library, a DAG-manipulation library, etc., the chances are good that you will end up using a piece of its functionality that has not been adequately tested and debugged. The CL community doesn't have enough eyeballs in it yet for its utility libraries to be bulletproof, in my experience. Best regards, R