One of the common problems that ASDF users faced when introducing the new binary location system is finding out where things reside.
Another problem we will eventually find is locating the resources associated to an ASDF system which has been replaced by a precompiled version.
In general it would be desirable for a system to be able to locate its resources (additional files, icons, bitmaps, etc) using logical pathnames.
For that purpose I have coded a small patch that adds a new keyword, :logical-host, to DEFSYSTEM. The keyword defines a logical pathname translation from "**;*.*.*" to "path/to/asdf/or/pathname/argument/**/*.*"
This way we may have
(defsystem :cl-unicode :logical-host "cl-unicode" ...)
and in the sources load the databases such as "cl-unicode:uppercase-table.dat"
https://bugs.launchpad.net/asdf/+bug/551222
On 2010-03-29, at 23:00 , Juan Jose Garcia-Ripoll wrote:
One of the common problems that ASDF users faced when introducing the new binary location system is finding out where things reside.
Another problem we will eventually find is locating the resources associated to an ASDF system which has been replaced by a precompiled version.
In general it would be desirable for a system to be able to locate its resources (additional files, icons, bitmaps, etc) using logical pathnames.
if the "pure" system definition proposal finds adoption, then something of this order will be necessary. in which case, the option should support a full logical host mapping.
if purity is not an issue, a logical pathname value for the pathname argument already allows for subsequent remapping, but that does require that the host be defined in addition to the system. if a non-root location suffices, it is also possible to support remapping by placing logical roots in the registry list.
For that purpose I have coded a small patch that adds a new keyword, :logical-host, to DEFSYSTEM. The keyword defines a logical pathname translation from "**;*.*.*" to "path/to/asdf/or/pathname/ argument/**/*.*"
This way we may have
(defsystem :cl-unicode :logical-host "cl-unicode" ...)
and in the sources load the databases such as "cl-unicode:uppercase- table.dat"
https://bugs.launchpad.net/asdf/+bug/551222
-- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com _______________________________________________ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
One site per system looks like it will quickly pollute the host namespace. What about we instead use a single logical host with subdirectories?
#P"COMMON-LISP:SYSTEMS;CL-PPCRE;CL-PPCRE.ASD"
The problem I have with this approach with logical pathnames at all is that I don't understand whether or not mixed case and other characters are allowed at all. They clearly aren't portably printable if they're allowed, but I suppose that's an OK limitation. In any case, if logical pathnames with names including mixed cases, dots, dashes and underscores are not allowed at all, or there's at least one supported implementation that will bork, I think it's a non-starter.
On the other hand, maybe it's time to have make an amendment to the CL standard that will make LOGICAL-PATHNAMEs usable at long last.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Faith, n: That quality which enables us to believe what we know to be untrue.
On 2010-03-30, at 01:42 , Faré wrote:
One site per system looks like it will quickly pollute the host namespace.
It depends on the use-case, as that determines the granularity one needs when re-mapping. While one could arrange the host mapping to each system independently, that would require machinery in addition to the standard logical pathname operators. I tend to work with a host for all libraries and one each for applications.
What about we instead use a single logical host with subdirectories?
#P"COMMON-LISP:SYSTEMS;CL-PPCRE;CL-PPCRE.ASD"
In my case, that works out to
#P"LIBRARY:de;weitz;cl-ppcre;cl-ppcre.asd"
which is easy to accomplish when all sources are in one tree.
The problem I have with this approach with logical pathnames at all is that I don't understand whether or not mixed case and other characters are allowed at all.
I do not recall that the pathname tests revealed any inconsistency among implementations which, in itself, would argue against their wider use.
They clearly aren't portably printable if they're allowed, but I suppose that's an OK limitation. In any case, if logical pathnames with names including mixed cases, dots, dashes and underscores are not allowed at all, or there's at least one supported implementation that will bork, I think it's a non-starter.
dashes are allowed. underscores and dots are not handled consistently as they are not valid word constituents.
the syntax restriction does mean that a project with non-conformant file names could not use such a feature were it to be added, but nothing in the suggestion implied that the argument would be required.
On the other hand, maybe it's time to have make an amendment to the CL standard that will make LOGICAL-PATHNAMEs usable at long last.
Some considerations:
* In order to achieve a declarative syntax in ASDF, the system that we built it with should have the least knowledge about ASDF. In other words, it should work just the same if we bulid it with any other system.
* Following this reasoning, logical pathnames are a useful choice because we do not force the application to invoke any functions on the system being built, for instance. Using special-purpose variables such as *asdf-system-location* are also not an option.
* A complementary action that ASDF might take is to load the libraries with a different value of *default-pathname-defaults*, but this may have nasty effects if the library is expected to work in the directory where the user is going to invoke it.
* The user of the library should be given full choice over the logical pathnames he / she wants to use, within margings. If we begin imposing ASDF-specific choices such as COMMON-LISP:ASDF;CL-PCRE;*.*.* then we are doomed.
* Logical hostname pollution is not such a big issue, as developers tend to have a good sense of measure in choosing their hostnames, prefixing them with their actual hostnames (de.weitz.cl-ppcre: ...) or using other uniform conventions. We can recommend their use and even perform some sanity check: whether two systems use the same logical hostname, for instance.
* Logical hostname lookup is cheap, log(#hostnames) if using sorted vectors or log(1) for hashes. Logical pathname translation is NOT CHEAP, it takes a time which is linear in the number of translations.
* Including logical pathnames in defsystem instead of just the name of the host is going to be problematic. I know at least one implementation (SBCL) that complains when we try to build a logical pathname with a host name that has no logical translations. It is the fish eating its own tail: we can not define the logical pathname in defsystem and thus we can not define the translation and thus...
* We do not need to fix logical pathnames. We just have to offer them as a portable solution for system distribution. Developers will have to care about the way they name their files and learning how logical pathnames work.
Juanjo
On 3/30/10 Mar 30 -5:00 AM, Juan Jose Garcia-Ripoll wrote:
Some considerations:
- In order to achieve a declarative syntax in ASDF, the system that we
built it with should have the least knowledge about ASDF. In other words, it should work just the same if we bulid it with any other system.
- Following this reasoning, logical pathnames are a useful choice
because we do not force the application to invoke any functions on the system being built, for instance. Using special-purpose variables such as *asdf-system-location* are also not an option.
- A complementary action that ASDF might take is to load the libraries
with a different value of *default-pathname-defaults*, but this may have nasty effects if the library is expected to work in the directory where the user is going to invoke it.
- The user of the library should be given full choice over the logical
pathnames he / she wants to use, within margings. If we begin imposing ASDF-specific choices such as COMMON-LISP:ASDF;CL-PCRE;*.*.* then we are doomed.
- Logical hostname pollution is not such a big issue, as developers tend
to have a good sense of measure in choosing their hostnames, prefixing them with their actual hostnames (de.weitz.cl-ppcre: ...) or using other uniform conventions. We can recommend their use and even perform some sanity check: whether two systems use the same logical hostname, for instance.
- Logical hostname lookup is cheap, log(#hostnames) if using sorted
vectors or log(1) for hashes. Logical pathname translation is NOT CHEAP, it takes a time which is linear in the number of translations.
- Including logical pathnames in defsystem instead of just the name of
the host is going to be problematic. I know at least one implementation (SBCL) that complains when we try to build a logical pathname with a host name that has no logical translations. It is the fish eating its own tail: we can not define the logical pathname in defsystem and thus we can not define the translation and thus...
- We do not need to fix logical pathnames. We just have to offer them as
a portable solution for system distribution. Developers will have to care about the way they name their files and learning how logical pathnames work.
Juanjo's argument seems persuasive, since it does not require us to wait until the Messiah comes, bringing improved logical pathnames, but allows us to live with the logical pathnames that we have today.
Question: should we raise a style warning if the user supplies a logical pathname that does not comply with the ANSI spec? I would prefer that we do that.
Question: are we going to create a logical pathname translation for just the system sources? Or should we create also something like
CL-PPCRE;FASLS;*.*.*
in addition? This seems a little tricky, since it requires that we hook into the output name rewriting logic, but probably is The Right Thing.
Best, Robert
On Tue, Mar 30, 2010 at 4:25 PM, Robert Goldman rpgoldman@sift.info wrote:
Question: should we raise a style warning if the user supplies a logical pathname that does not comply with the ANSI spec? I would prefer that we do that.
The first question is whether we are going to provide a logical hostname or whether instead we will allow the user to provide a full logical pathname translation. That is
:logical-host "CL-PPCRE"
versus
:logical-path "CL-PPCRE:MY-DESIRED;SET;OF;VIRTUAL;DIRECTORIES;*.*.*"
The latter is trickier and proner to break. If we use the former we can provide two sets of translations
CL-PPCRE:FASL;*.*.* -> whatever binary directory CL-PPCRE:**;*.*.* -> source directory
So I would stay with that.
Question: are we going to create a logical pathname translation for just the system sources? Or should we create also something like
CL-PPCRE;FASLS;*.*.*
in addition? This seems a little tricky, since it requires that we hook into the output name rewriting logic, but probably is The Right Thing.
I agree, but again this can be done in a two-step process. First convince people that the logical hostname works and only then move to providing binary translations -- if that is ever needed, which might not be the case.
Juanjo
On 3/30/10 Mar 30 -9:29 AM, Juan Jose Garcia-Ripoll wrote:
On Tue, Mar 30, 2010 at 4:25 PM, Robert Goldman <rpgoldman@sift.info mailto:rpgoldman@sift.info> wrote:
Question: should we raise a style warning if the user supplies a logical pathname that does not comply with the ANSI spec? I would prefer that we do that.
The first question is whether we are going to provide a logical hostname or whether instead we will allow the user to provide a full logical pathname translation. That is
:logical-host "CL-PPCRE"
versus
:logical-path "CL-PPCRE:MY-DESIRED;SET;OF;VIRTUAL;DIRECTORIES;*.*.*"
The latter is trickier and proner to break. If we use the former we can provide two sets of translations
I agree. In particular, I have vague memories of differences between ACL and SBCL on how to handle the *.*.* versus *.*, but this is lost in my neural network.
CL-PPCRE:FASL;*.*.* -> whatever binary directory CL-PPCRE:**;*.*.* -> source directory
So I would stay with that.
Question: are we going to create a logical pathname translation for just the system sources? Or should we create also something like CL-PPCRE;FASLS;*.*.* in addition? This seems a little tricky, since it requires that we hook into the output name rewriting logic, but probably is The Right Thing.
I agree, but again this can be done in a two-step process. First convince people that the logical hostname works and only then move to providing binary translations -- if that is ever needed, which might not be the case.
Minor suggestion: for extensibility, uniformity, etc., would a variant where we automagically create
CL-PPCRE:SRC; and CL-PPCRE:FASL;
be an acceptable choice?
I like this for orthogonality, and because I've always been a bit unhappy using a logical pathname with just a device --- too reminiscent of Windows' C:Foo!
Also, it seems like a cleaner path to allowing the user to extend the logical pathnames for the system's host.
Not a big deal; just thought I would offer this as an early-stage suggestion.
Best, r
On 2010-03-30, at 16:42 , Robert Goldman wrote:
On 3/30/10 Mar 30 -9:29 AM, Juan Jose Garcia-Ripoll wrote:
On Tue, Mar 30, 2010 at 4:25 PM, Robert Goldman <rpgoldman@sift.info mailto:rpgoldman@sift.info> wrote:
Question: should we raise a style warning if the user supplies a logical pathname that does not comply with the ANSI spec? I
would prefer that we do that.
The first question is whether we are going to provide a logical hostname or whether instead we will allow the user to provide a full logical pathname translation. That is
:logical-host "CL-PPCRE"
versus
:logical-path "CL-PPCRE:MY-DESIRED;SET;OF;VIRTUAL;DIRECTORIES;*.*.*"
The latter is trickier and proner to break. If we use the former we can provide two sets of translations
it may be prone to break, but that just means one must pay attention.
it would be ok if, given just the host name, asdf were to assert translations which are the equivalent of the current binary mappings. if the argument is a translation specification, there is no reason not to believe it and apply it as given.
I agree. In particular, I have vague memories of differences between ACL and SBCL on how to handle the *.*.* versus *.*, but this is lost in my neural network.
my experience is that it depends entirely on whether the runtime supports versions at all. if one intended a logical host translation spec to behave "the same" on a runtime which supports file versions as on one which does not, one has to express the respective mappings correctly.
On 3/30/10 Mar 30 -11:58 AM, james anderson wrote:
On 2010-03-30, at 16:42 , Robert Goldman wrote:
On 3/30/10 Mar 30 -9:29 AM, Juan Jose Garcia-Ripoll wrote:
On Tue, Mar 30, 2010 at 4:25 PM, Robert Goldman <rpgoldman@sift.info mailto:rpgoldman@sift.info> wrote:
Question: should we raise a style warning if the user supplies a logical pathname that does not comply with the ANSI spec? I
would prefer that we do that.
The first question is whether we are going to provide a logical hostname or whether instead we will allow the user to provide a full logical pathname translation. That is
:logical-host "CL-PPCRE"
versus
:logical-path "CL-PPCRE:MY-DESIRED;SET;OF;VIRTUAL;DIRECTORIES;*.*.*"
The latter is trickier and proner to break. If we use the former we can provide two sets of translations
it may be prone to break, but that just means one must pay attention.
it would be ok if, given just the host name, asdf were to assert translations which are the equivalent of the current binary mappings. if the argument is a translation specification, there is no reason not to believe it and apply it as given.
I agree. In particular, I have vague memories of differences between ACL and SBCL on how to handle the *.*.* versus *.*, but this is lost in my neural network.
my experience is that it depends entirely on whether the runtime supports versions at all. if one intended a logical host translation spec to behave "the same" on a runtime which supports file versions as on one which does not, one has to express the respective mappings correctly.
I believe that is correct. I think that on at least one of these platforms, it is customary to rewrite
"**;*.*.*" to "**/*.*"
to ensure that the version field is thrown away.
best, r
On 2010-03-30, at 19:13 , Robert Goldman wrote:
[...] my experience is that it depends entirely on whether the runtime supports versions at all. if one intended a logical host translation spec to behave "the same" on a runtime which supports file versions as on one which does not, one has to express the respective mappings correctly.
I believe that is correct. I think that on at least one of these platforms, it is customary to rewrite
"**;*.*.*" to "**/*.*"
to ensure that the version field is thrown away.
my experience to date has been that it was more reliable to duplicate the rules. it would be nice to be in a position to test it exhaustively. soon.
On 2010-03-30, at 16:25 , Robert Goldman wrote:
On 3/30/10 Mar 30 -5:00 AM, Juan Jose Garcia-Ripoll wrote:
[...]
Question: are we going to create a logical pathname translation for just the system sources? Or should we create also something like
CL-PPCRE;FASLS;*.*.*
if asdf decides to befried logical pathnames, it should allow the system to define its own mapping.
in addition? This seems a little tricky, since it requires that we hook into the output name rewriting logic, but probably is The Right Thing.
i had understood that the name rewriting logic is disabled for logical pathnames. which is as it should be.
On 3/30/10 Mar 30 -11:52 AM, james anderson wrote:
On 2010-03-30, at 16:25 , Robert Goldman wrote:
On 3/30/10 Mar 30 -5:00 AM, Juan Jose Garcia-Ripoll wrote:
[...]
Question: are we going to create a logical pathname translation for just the system sources? Or should we create also something like
CL-PPCRE;FASLS;*.*.*
if asdf decides to befried logical pathnames, it should allow the system to define its own mapping.
in addition? This seems a little tricky, since it requires that we hook into the output name rewriting logic, but probably is The Right Thing.
i had understood that the name rewriting logic is disabled for logical pathnames. which is as it should be.
Clarification: the name-rewriting logic would still be disabled for logical pathnames. What I was suggesting was that
<SYSTEM-NAME>:FASL;
should be a logical pathname that would point to the location where <SYSTEM-NAME>'s (direct) fasls would be written by Faré's name rewriting.
I.e., this would be a way for the system to find its own fasls reliably, no matter what the output name rewriting does.
Is that more clear?
thanks, r
On 2010-03-30, at 19:12 , Robert Goldman wrote:
On 3/30/10 Mar 30 -11:52 AM, james anderson wrote:
On 2010-03-30, at 16:25 , Robert Goldman wrote:
On 3/30/10 Mar 30 -5:00 AM, Juan Jose Garcia-Ripoll wrote:
[...]
Question: are we going to create a logical pathname translation for just the system sources? Or should we create also something like
CL-PPCRE;FASLS;*.*.*
if asdf decides to befried logical pathnames, it should allow the system to define its own mapping.
in addition? This seems a little tricky, since it requires that we hook into the output name rewriting logic, but probably is The Right Thing.
i had understood that the name rewriting logic is disabled for logical pathnames. which is as it should be.
Clarification: the name-rewriting logic would still be disabled for logical pathnames. What I was suggesting was that
<SYSTEM-NAME>:FASL;
should be a logical pathname that would point to the location where <SYSTEM-NAME>'s (direct) fasls would be written by Faré's name rewriting.
I.e., this would be a way for the system to find its own fasls reliably, no matter what the output name rewriting does.
Is that more clear?
yes and no. if one wants to map binary files differently that source files, the present (last i looked) asdf behavior was adequate. the last i looked, my binaries were at the specified locations. my experience[1] is that it works better if the pattern matches on the file type. given the proper mapping one neither needs nor wants any asdf internal mapping.
--- [1] : http://github.com/lisp/de.setf.utility/blob/master/ pathnames.lisp#L119
On 3/30/10 Mar 30 -12:32 PM, james anderson wrote:
On 2010-03-30, at 19:12 , Robert Goldman wrote:
On 3/30/10 Mar 30 -11:52 AM, james anderson wrote:
On 2010-03-30, at 16:25 , Robert Goldman wrote:
On 3/30/10 Mar 30 -5:00 AM, Juan Jose Garcia-Ripoll wrote:
[...]
Question: are we going to create a logical pathname translation for just the system sources? Or should we create also something like
CL-PPCRE;FASLS;*.*.*
if asdf decides to befried logical pathnames, it should allow the system to define its own mapping.
in addition? This seems a little tricky, since it requires that we hook into the output name rewriting logic, but probably is The Right Thing.
i had understood that the name rewriting logic is disabled for logical pathnames. which is as it should be.
Clarification: the name-rewriting logic would still be disabled for logical pathnames. What I was suggesting was that
<SYSTEM-NAME>:FASL;
should be a logical pathname that would point to the location where <SYSTEM-NAME>'s (direct) fasls would be written by Faré's name rewriting.
I.e., this would be a way for the system to find its own fasls reliably, no matter what the output name rewriting does.
Is that more clear?
yes and no. if one wants to map binary files differently that source files, the present (last i looked) asdf behavior was adequate. the last i looked, my binaries were at the specified locations. my experience[1] is that it works better if the pattern matches on the file type. given the proper mapping one neither needs nor wants any asdf internal mapping.
I believe what Juanjo wants here is an ability to have the LOADING of the ASDF file generate some logical pathnames. So that after I asdf-load my system FOO, then code /inside/ foo could use a logical pathname like
"FOO:SRC;DATAFILES;DATA1.DAT"
with the guarantee that ASDF has set up the logical pathname translations for "FOO:SRC;"
best, r
I believe what Juanjo wants here is an ability to have the LOADING of the ASDF file generate some logical pathnames. So that after I asdf-load my system FOO, then code /inside/ foo could use a logical pathname like
"FOO:SRC;DATAFILES;DATA1.DAT"
with the guarantee that ASDF has set up the logical pathname translations for "FOO:SRC;"
this may be true. it is part of what he has said. another part is, that the systems should work the same if built without asdf. there were two passages:
On 2010-03-30, at 12:00 , Juan Jose Garcia-Ripoll wrote:
Some considerations:
- In order to achieve a declarative syntax in ASDF, the system that
we built it with should have the least knowledge about ASDF. In other words, it should work just the same if we bulid it with any other system.
On 2010-03-30, at 16:17 , Juan Jose Garcia-Ripoll wrote:
On Tue, Mar 30, 2010 at 4:06 PM, Robert Goldman rpgoldman@sift.info wrote: FWIW, Faré provided a couple of new API functions to give ASDF users this function, notably
SYSTEM-RELATIVE-PATHNAME
and
SYSTEM-SOURCE-DIRECTORY
I insist on the following: in order to achieve purity in ASDF files we want to keep as far away as possible from using any kind of API in the compiled code.
The best system build system is one which is invisible to the code that it builds.
Exposing functions, imposing the dependency of the user code on ASDF API as well as imposing that the library code queries ASDF for its own system is a very bad design.
We want system definitions to be descriptive, not programatic, and we want the user code to be able to exist in an ASDF-free environment, as standalone systems. The existing situation does not allow this.
it would be nice to have concrete use cases. lacking them, these lists of goals require that there be some mechanism independent of asdf to effect the logical host definitions. which does not convince of a need to add the mechanism to asdf itself.
On 3/30/10 Mar 30 -1:06 PM, james anderson wrote:
...
We want system definitions to be descriptive, not programatic, and we want the user code to be able to exist in an ASDF-free environment, as standalone systems. The existing situation does not allow this.
it would be nice to have concrete use cases. lacking them, these lists of goals require that there be some mechanism independent of asdf to effect the logical host definitions. which does not convince of a need to add the mechanism to asdf itself.
I think I know roughly what Juanjo means here. In particular:
1. I don't like to have my systems use the ASDF API internally. E.g., I will set up variables with pathnames, or use logical pathnames in my ASDF system definition files, so that my actual code doesn't have to use something like asdf:system-definition-pathname.
2. I have worked with people who don't use ASDF. If I observe strictures like the ones I lay out in point 1, then those people can write a simple load file that somehow initializes the logical pathnames and loads the code (how to do that is /their/ problem!) and then they can use my code just as I do. If I used calls like asdf:system-definition-pathname, that would not be possible.
So I think Juanjo's objectives here (or at least my interpretation of his objectives!) are reasonable.
[For the record, while I agree with Juanjo's overall statement of principle ("We want system definitions to be descriptive, not programmatic") I don't believe that this will be fully achievable in ASDF 2.]
Best, R
On Tue, Mar 30, 2010 at 8:28 PM, Robert Goldman rpgoldman@sift.info wrote:
On 3/30/10 Mar 30 -1:06 PM, james anderson wrote:
...
We want system definitions to be descriptive, not programatic, and we want the user code to be able to exist in an ASDF-free environment, as standalone systems. The existing situation does not allow this.
it would be nice to have concrete use cases. lacking them, these lists of goals require that there be some mechanism independent of asdf to effect the logical host definitions. which does not convince of a need to add the mechanism to asdf itself.
I think I know roughly what Juanjo means here. In particular:
- I don't like to have my systems use the ASDF API internally. E.g.,
I will set up variables with pathnames, or use logical pathnames in my ASDF system definition files, so that my actual code doesn't have to use something like asdf:system-definition-pathname.
- I have worked with people who don't use ASDF. If I observe
strictures like the ones I lay out in point 1, then those people can write a simple load file that somehow initializes the logical pathnames and loads the code (how to do that is /their/ problem!) and then they can use my code just as I do. If I used calls like asdf:system-definition-pathname, that would not be possible.
So I think Juanjo's objectives here (or at least my interpretation of his objectives!) are reasonable.
You got it right. I would extend the argument but I have to leave. Perhaps tomorrow.
[For the record, while I agree with Juanjo's overall statement of principle ("We want system definitions to be descriptive, not programmatic") I don't believe that this will be fully achievable in ASDF 2.]
No, but we get closer :-)
On 2010-03-30, at 20:36 , Juan Jose Garcia-Ripoll wrote:
I think I know roughly what Juanjo means here. In particular:
- I don't like to have my systems use the ASDF API internally.
E.g., I will set up variables with pathnames, or use logical pathnames in my ASDF system definition files, so that my actual code doesn't have to use something like asdf:system-definition-pathname.
- I have worked with people who don't use ASDF. If I observe
strictures like the ones I lay out in point 1, then those people can write a simple load file that somehow initializes the logical pathnames and loads the code (how to do that is /their/ problem!) and then they can use my code just as I do. If I used calls like asdf:system-definition-pathname, that would not be possible.
So I think Juanjo's objectives here (or at least my interpretation of his objectives!) are reasonable.
You got it right. I would extend the argument but I have to leave. Perhaps tomorrow.
? one can do that now. without any changes to asdf and without any purity tests. so, he must be thinking of something in addition.
On 3/30/10 Mar 30 -1:48 PM, james anderson wrote:
On 2010-03-30, at 20:36 , Juan Jose Garcia-Ripoll wrote:
I think I know roughly what Juanjo means here. In particular:
- I don't like to have my systems use the ASDF API internally. E.g.,
I will set up variables with pathnames, or use logical pathnames in my ASDF system definition files, so that my actual code doesn't have to use something like asdf:system-definition-pathname.
- I have worked with people who don't use ASDF. If I observe
strictures like the ones I lay out in point 1, then those people can write a simple load file that somehow initializes the logical pathnames and loads the code (how to do that is /their/ problem!) and then they can use my code just as I do. If I used calls like asdf:system-definition-pathname, that would not be possible.
So I think Juanjo's objectives here (or at least my interpretation of his objectives!) are reasonable.
You got it right. I would extend the argument but I have to leave. Perhaps tomorrow.
? one can do that now. without any changes to asdf and without any purity tests. so, he must be thinking of something in addition.
Can you explain how to do this?
I typically have an ugly mess in my .asd files where I get *load-truename*, and then fuss with it to extract the directory pathname, etc. Juanjo's :logical-hostname ASDF extension seems very handy to me. My alternative involves a lot of repeated boilerplate code, and it's surprisingly fussy to extract that directory pathname....
best, r
On 2010-03-30, at 21:36 , Robert Goldman wrote:
On 3/30/10 Mar 30 -1:48 PM, james anderson wrote:
On 2010-03-30, at 20:36 , Juan Jose Garcia-Ripoll wrote:
I think I know roughly what Juanjo means here. In particular:
- I don't like to have my systems use the ASDF API internally.
E.g., I will set up variables with pathnames, or use logical pathnames in my ASDF system definition files, so that my actual code doesn't have to use something like asdf:system-definition-pathname.
- I have worked with people who don't use ASDF. If I observe
strictures like the ones I lay out in point 1, then those people can write a simple load file that somehow initializes the logical pathnames and loads the code (how to do that is /their/ problem!) and then they can use my code just as I do. If I used calls like asdf:system-definition-pathname, that would not be possible.
So I think Juanjo's objectives here (or at least my interpretation of his objectives!) are reasonable.
You got it right. I would extend the argument but I have to leave. Perhaps tomorrow.
? one can do that now. without any changes to asdf and without any purity tests. so, he must be thinking of something in addition.
Can you explain how to do this?
i have found two ways to do this.
a. put logical hosts on asdf's search path and unify the pathnames for found system definitions with them. this is semi-pure in that the mechanism is part of an extension[1] and hooks into an additional method for system instantiation, but, in principle, it has nothing to do with the hierarchal name mechanism. it uses the mappings cited[2] in the earlier message to implement the equivalent of binary locations in terms pathname translation patterns which match the output file type.
b. define a host anew for each system definition[3]. this uses [2] to root a host at a given location. by default that of the currently loaded file.
--- [1] : http://github.com/lisp/de.setf.utility/blob/master/asdf/ hierarchical-names.lisp#L274 (this uses on operator from [2].) [2] : http://github.com/lisp/de.setf.utility/blob/master/ pathnames.lisp#L119 [3] : http://github.com/lisp/de.setf.amqp/blob/master/amqp.asd#L36
I typically have an ugly mess in my .asd files where I get *load-truename*, and then fuss with it to extract the directory pathname, etc. Juanjo's :logical-hostname ASDF extension seems very handy to me. My alternative involves a lot of repeated boilerplate code, and it's surprisingly fussy to extract that directory pathname....
i have managed to get most of the boilerplate out of the system definitions. if you need a system-specific host, there's going to be one form or one argument somewhere. recall the requirement, that the system also build without asdf. if shared one suffices, the unification mechanism requires no extra terms in the definition.
On 3/30/10 Mar 30 -4:32 PM, james anderson wrote:
On 2010-03-30, at 21:36 , Robert Goldman wrote:
On 3/30/10 Mar 30 -1:48 PM, james anderson wrote:
On 2010-03-30, at 20:36 , Juan Jose Garcia-Ripoll wrote:
I think I know roughly what Juanjo means here. In particular:
- I don't like to have my systems use the ASDF API internally.
E.g., I will set up variables with pathnames, or use logical pathnames in my ASDF system definition files, so that my actual code doesn't have to use something like asdf:system-definition-pathname.
- I have worked with people who don't use ASDF. If I observe
strictures like the ones I lay out in point 1, then those people can write a simple load file that somehow initializes the logical pathnames and loads the code (how to do that is /their/ problem!) and then they can use my code just as I do. If I used calls like asdf:system-definition-pathname, that would not be possible.
So I think Juanjo's objectives here (or at least my interpretation of his objectives!) are reasonable.
You got it right. I would extend the argument but I have to leave. Perhaps tomorrow.
? one can do that now. without any changes to asdf and without any purity tests. so, he must be thinking of something in addition.
Can you explain how to do this?
i have found two ways to do this.
a. put logical hosts on asdf's search path and unify the pathnames for found system definitions with them. this is semi-pure in that the mechanism is part of an extension[1] and hooks into an additional method for system instantiation, but, in principle, it has nothing to do with the hierarchal name mechanism. it uses the mappings cited[2] in the earlier message to implement the equivalent of binary locations in terms pathname translation patterns which match the output file type.
I think that this is the opposite of what Juanjo is proposing. According to Juanjo, the population of the asdf:*central-registry*, and the loading of system causes logical pathname hosts to be defined. What you propose here is to define logical pathname hosts and use them to help find and load systems.
b. define a host anew for each system definition[3]. this uses [2] to root a host at a given location. by default that of the currently loaded file.
[1] : http://github.com/lisp/de.setf.utility/blob/master/asdf/ hierarchical-names.lisp#L274 (this uses on operator from [2].) [2] : http://github.com/lisp/de.setf.utility/blob/master/ pathnames.lisp#L119 [3] : http://github.com/lisp/de.setf.amqp/blob/master/amqp.asd#L36
BTW, any chance you could use tiny url or some other url shortener on these github links? At least in my email client, they get line breaks in the middle and thus broken....
This bit of code in [2] is very nice, by the way, seems to be just what Juanjo is asking for, but with a slightly different API, yes?
On 2010-03-31, at 05:31 , Robert Goldman wrote:
On 3/30/10 Mar 30 -4:32 PM, james anderson wrote:
On 2010-03-30, at 21:36 , Robert Goldman wrote:
[ ... ] Can you explain how to do this?
i have found two ways to do this.
a. put logical hosts on asdf's search path and unify the pathnames for found system definitions with them. this is semi-pure in that the mechanism is part of an extension[1] and hooks into an additional method for system instantiation, but, in principle, it has nothing to do with the hierarchal name mechanism. it uses the mappings cited[2] in the earlier message to implement the equivalent of binary locations in terms pathname translation patterns which match the output file type.
I think that this is the opposite of what Juanjo is proposing.
i do not yet understand his use cases. despite his extended descriptions. as far as i have managed, this fulfills the requirement for dual build methods - one with and one without asdf. in the asdf case, the registry entry suffices. in the other case, there is some other host definition.
According to Juanjo, the population of the asdf:*central-registry*, and the loading of system causes logical pathname hosts to be defined. What you propose here is to define logical pathname hosts and use them to help find and load systems.
b. define a host anew for each system definition[3]. this uses [2] to root a host at a given location. by default that of the currently loaded file.
this alternative satisfies the requirement directly. the definition is the same in both methods.
[1] : http://github.com/lisp/de.setf.utility/blob/master/asdf/ hierarchical-names.lisp#L274 (this uses on operator from [2].) [2] : http://github.com/lisp/de.setf.utility/blob/master/ pathnames.lisp#L119 [3] : http://github.com/lisp/de.setf.amqp/blob/master/amqp.asd#L36
BTW, any chance you could use tiny url or some other url shortener on these github links? At least in my email client, they get line breaks in the middle and thus broken....
will endeavour to.
This bit of code in [2] is very nice, by the way, seems to be just what Juanjo is asking for, but with a slightly different API, yes?
yes. you may have noted the range of implementations and the entry for (mcl 68k)? i have evidently had the fortune of more than a decade of colleagues willing to learn rules and follow them.
On Tue, Mar 30, 2010 at 11:32 PM, james anderson james.anderson@setf.dewrote:
a. put logical hosts on asdf's search path and unify the pathnames for found system definitions with them.
Sorry, but from the paragraph that you wrote I do not follow completely the logic of what it is achieved. In any case the sentence "put logical hosts on asdf's search path" implies that ASDF has to be there for the pathnames to exist, while I was advocating a mechanism that did not depend on a central registry and, most of all, that did not depend on the user setting up translations.
I hope you understand that my requisites, so to say, are flexible, and the motivation is to reach an even easier to use, more transparent way of building Lisp components where ASDF is better hidden -- much like a shared library loader, or a C linker, is hiding a lot of complexity from casual users.
Juanjo
On 2010-03-31, at 14:07 , Juan Jose Garcia-Ripoll wrote:
On Tue, Mar 30, 2010 at 11:32 PM, james anderson james.anderson@setf.de wrote: a. put logical hosts on asdf's search path and unify the pathnames for found system definitions with them.
Sorry, but from the paragraph that you wrote I do not follow completely the logic of what it is achieved.
if the system is built with asdf, the initialize-instance operator for systems can examine the system's pathname and the pathnames which constitute the list bound to asdf:*central-registry*, determine if there are any logical host definitions which subsume the respective physical pathname, and use the respective logical pathname to designate the system's definition file instead of the physical one which was used to load it.
as they say in the rad world, "it's that easy!"
i actually use a more radical approach. one which searches the entire logical host world, but that probably goes too far and can require disambiguation.
In any case the sentence "put logical hosts on asdf's search path" implies that ASDF has to be there for the pathnames to exist, while I was advocating a mechanism that did not depend on a central registry and, most of all, that did not depend on the user setting up translations.
i simply don't understand your requirements. on one hand both with and without asdf, but then it is objected, that a mechanism which requires asdf does not work without it. please explain.
if the requirement is that exact same exact formulation be used with and without asdf, then a) the additional system initialization argument is excluded b) the second option in my earlier message provides this path.
On Wed, Mar 31, 2010 at 4:39 PM, james anderson james.anderson@setf.dewrote:
i simply don't understand your requirements. on one hand both with and without asdf, but then it is objected, that a mechanism which requires asdf does not work without it. please explain.
What I do not understand is what YOU do not understand.
In an standalone system there is no ASDF. That is all.
I want to be able to take an ASDF system describing my program and its dependencies on other libraries and compile it and produce a standalone executable. That program will not have ASDF inside, it will not have the notion of a central registry it will have nothing, but it still has to work.
Please understand that the program I am talking about is not a dumped lisp image that contains everything needed to build the system and even some extra stuff that was there (database of ASDF systems and other trash). The programs I produce are just a set of compiled files, one per ASDF system, linked together with a core library that is the Common Lisp environment ECL offers.
When the program starts, the linked files are "loaded" one after another, as if LOAD was used but actually much faster. But I insist again ASDF is not part of the things that is loaded simply because there is no need.
The only thing that one might need ASDF for is to locate the data associated to the program, but if we provide logical pathname translations (one host per library), then a small piece of code can be inserted at the beginning of the program which locates the data and sets up the appropriate translations.
if the requirement is that exact same exact formulation be used with and
without asdf, then a) the additional system initialization argument is excluded b) the second option in my earlier message provides this path.
I only find "b" acceptable because I still do not understand how "a" is going to work without a central registry and a lot of complicated logic that has to propagate to those distributed binaries.
Having "b", which I recall was one logical host name per system, one may simply prepend to the compiled set of files a tiny bit of lisp that looks as follows
(defvar *program-data-location* #+windows ;; Data and binaries live together (program-location) #+unix ;; Data is installed somewhere standard "/usr/share/program/" #+osx ;; Assume this is an application bundle (merge-pathnames "Content/Resources/" (program-location)))
(defun set-host-translation (root extra) ...)
(set-host-translation "cl-ppcre" "cl-ppcre/") (set-host-translation "cl-unicode" cl-unicode/")
With this simple, automatically generated code I can pack the given systems and let them work anywhere, even though ASDF is not present.
The same or a similar procedure can be used to link many systems into a single distributable binary fasl + resources, install it in a standard location that can be found by certain lisps and prepend a simple logic for finding out its data... Without using ASDF in the file itself.
There is even a third model in which installation paths are known beforehand. Say that a package maintainer from Debian has the role of generating binaries for 10 libraries out there and they have complex inter-dependencies. Libraries can be compiled in the machine of the maintainer and then installed by ASDF in their standard locations. ASDF can, in the process, hard-code the pathname translations when installing the files.
For instance, if we produce binaries for cl-ppcre-1.0.1 and cl-unicode-2.0.1 and the latter depends on the former, and we know that they are going to live in /usr/lib/common-lisp/cl-ppcre-1.0.1 /usr/lib/common-lisp/cl-unicode-2.0.1 when "asdf-build" compiles the files and "asdf-installs" them we may get
;; Stripped down version of a system definition ;; which only contains the precompiled files ;; and which can be searched and used by ASDF /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.asd /usr/lib/common-lisp/cl-unicode-1.0.1/cl-unicode.asd
;; The binaries themselves /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre-1.0.1.fas /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode-1.0.1.fas
;; Proxy binaries that can be loaded using "require" ;; even if ASDF is not present. These files define the logical ;; pathname translations based on hardcoded locations. /user/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.fas /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.fas
;; Data used by the library /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.dat
;; Symbolic link to the last version /usr/lib/common-lisp/cl-ppcre -> cl-ppcre-1.0.1
The FASL without version numbers files can explicitly load the required versions of the libraries because they know the hardcoded paths of their files. In doing so they can also set up the appropriate pathname translations that are used by the other files.
I have listed just two examples with different paradigms on how to build and distribute software, and which go beyond current use in which the user has to set up a collection of registries, in your case you even define pathname translations and have magic to make ASDF understand them, sources are always present, things are recompiled...
But there are other possibilities. ASDF can be used to produce programs and libraries that can be distributed, ASDF can itself be used without people noticing it (see the automatically generated FASL files), it can integrate with other build processes (via some simple scripts asdf-build, asdf-install which I may offer to write), we can achieve better system descriptions, and a lot of things that make people's life easier because they can forget that they use ASDF at all -- just like when writing C I am free to forget that I am using "make" at some point.
Juanjo
On 2010-03-31, at 20:00 , Juan Jose Garcia-Ripoll wrote:
On Wed, Mar 31, 2010 at 4:39 PM, james anderson james.anderson@setf.de wrote: i simply don't understand your requirements. on one hand both with and without asdf, but then it is objected, that a mechanism which requires asdf does not work without it. please explain.
What I do not understand is what YOU do not understand.
In an standalone system there is no ASDF. That is all.
I want to be able to take an ASDF system describing my program and its dependencies on other libraries and compile it and produce a standalone executable. That program will not have ASDF inside, it will not have the notion of a central registry it will have nothing, but it still has to work.
it's going to need to take it's logical host definitions from somewhere.
either they survived, in which case, in the simplest form they are re- rooted, or in more complex forms distinct branches are mapped distinctly, or they didn't survive, and the program will either apply heuristics to locate the intended files, or will benefit from external information to accomplish that task. to these ends, it will use at least the cl operators for logical pathnames. in the case of the examples which i posted, it might in addition use the same operators from the cited library which were used to establish the host definitions in the first place.
asdf did not appear in that paragraph.
Please understand that the program I am talking about is not a dumped lisp image that contains everything needed to build the system and even some extra stuff that was there (database of ASDF systems and other trash). The programs I produce are just a set of compiled files, one per ASDF system, linked together with a core library that is the Common Lisp environment ECL offers.
that's ok.
When the program starts, the linked files are "loaded" one after another, as if LOAD was used but actually much faster. But I insist again ASDF is not part of the things that is loaded simply because there is no need.
that's ok.
The only thing that one might need ASDF for is to locate the data associated to the program, but if we provide logical pathname translations (one host per library), then a small piece of code can be inserted at the beginning of the program which locates the data and sets up the appropriate translations.
as noted above.
if the requirement is that exact same exact formulation be used with and without asdf, then a) the additional system initialization argument is excluded b) the second option in my earlier message provides this path.
I only find "b" acceptable because I still do not understand how "a" is going to work without a central registry and a lot of complicated logic that has to propagate to those distributed binaries.
asdf uses a central registry. that is a given. that is how it finds the system definitions. i reported that it is possible for asdf to use the registry list to infer the logical host which should apply to a given system definitions. without any additional specification in the system declaration. that asdf does that has no necessary bearing on what a release process or the released program does with logical pathnames. it just makes it possible for the build process to share information with the later processes.
Having "b", which I recall was one logical host name per system, one may simply prepend to the compiled set of files a tiny bit of lisp that looks as follows
(defvar *program-data-location* #+windows ;; Data and binaries live together (program-location) #+unix ;; Data is installed somewhere standard "/usr/share/program/" #+osx ;; Assume this is an application bundle (merge-pathnames "Content/Resources/" (program-location)))
(defun set-host-translation (root extra) ...)
(set-host-translation "cl-ppcre" "cl-ppcre/") (set-host-translation "cl-unicode" cl-unicode/")
this is what the code in the examples i cited does. not in exactly these terms, but the pathname world is not large. the only difference between the two approaches which i described is whether asdf uses the operators or they are used otherwise, in addition to the defsystem form.
With this simple, automatically generated code I can pack the given systems and let them work anywhere, even though ASDF is not present.
The same or a similar procedure can be used to link many systems into a single distributable binary fasl + resources, install it in a standard location that can be found by certain lisps and prepend a simple logic for finding out its data... Without using ASDF in the file itself.
There is even a third model in which installation paths are known beforehand. Say that a package maintainer from Debian has the role of generating binaries for 10 libraries out there and they have complex inter-dependencies. Libraries can be compiled in the machine of the maintainer and then installed by ASDF in their standard locations. ASDF can, in the process, hard-code the pathname translations when installing the files.
For instance, if we produce binaries for cl-ppcre-1.0.1 and cl- unicode-2.0.1 and the latter depends on the former, and we know that they are going to live in /usr/lib/common-lisp/cl-ppcre-1.0.1 /usr/lib/common-lisp/cl-unicode-2.0.1 when "asdf-build" compiles the files and "asdf-installs" them we may get
;; Stripped down version of a system definition ;; which only contains the precompiled files ;; and which can be searched and used by ASDF /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.asd /usr/lib/common-lisp/cl-unicode-1.0.1/cl-unicode.asd
;; The binaries themselves /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre-1.0.1.fas /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode-1.0.1.fas
;; Proxy binaries that can be loaded using "require" ;; even if ASDF is not present. These files define the logical ;; pathname translations based on hardcoded locations. /user/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.fas /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.fas
;; Data used by the library /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.dat
;; Symbolic link to the last version /usr/lib/common-lisp/cl-ppcre -> cl-ppcre-1.0.1
The FASL without version numbers files can explicitly load the required versions of the libraries because they know the hardcoded paths of their files. In doing so they can also set up the appropriate pathname translations that are used by the other files.
I have listed just two examples with different paradigms on how to build and distribute software, and which go beyond current use in which the user has to set up a collection of registries, in your case you even define pathname translations and have magic to make ASDF understand them, sources are always present, things are recompiled...
the first two clauses are true, although the characterization is unwarranted. the third and fourth are not.
good evening;
On 2010-03-30, at 12:00 , Juan Jose Garcia-Ripoll wrote:
Some considerations:
[...]
- Including logical pathnames in defsystem instead of just the name
of the host is going to be problematic. I know at least one implementation (SBCL) that complains when we try to build a logical pathname with a host name that has no logical translations. It is the fish eating its own tail: we can not define the logical pathname in defsystem and thus we can not define the translation and thus...
please explain further. the passage is not clear. no implementation should permit to "build a logical pathname with a host name that has no logical translations." you must intend some other use case.
there are already mechanisms which are known to work portably with asdf without requiring it to change: - a pathname argument which is a logical pathname - unifying the pathname which asdf derives from the load pathname with known logical hosts
- We do not need to fix logical pathnames. We just have to offer
them as a portable solution for system distribution. Developers will have to care about the way they name their files and learning how logical pathnames work.
Juanjo
-- Instituto de Física Fundamental, CSIC c/ Serrano, 113b, Madrid 28006 (Spain) http://tream.dreamhosters.com
On 3/29/10 Mar 29 -6:42 PM, Faré wrote:
One site per system looks like it will quickly pollute the host namespace. What about we instead use a single logical host with subdirectories?
#P"COMMON-LISP:SYSTEMS;CL-PPCRE;CL-PPCRE.ASD"
The problem I have with this approach with logical pathnames at all is that I don't understand whether or not mixed case and other characters are allowed at all.
They are not.
The syntax of logical pathname namestrings is
"word---one or more uppercase letters, digits, and hyphens."
This causes pain when working with people WhoLikeOtherProgrammingLanguages.
here's an example: [SBCL] CL-USER(1): (setf (logical-pathname-translations "MYSRC") (list (list "**;*.*.*" "/Users/rpg/src/**/*.*")))
(("**;*.*.*" "/Users/rpg/src/**/*.*")) CL-USER(2): (translate-logical-pathname "MYSRC:growlnotify;")
#P"/Users/rpg/src/growlnotify/" CL-USER(3): (translate-logical-pathname "MYSRC:rhino1_7R1;")
#P"MYSRC:rhino1_7R1;"
rpg% ls -d /Users/rpg/src/rhino1_7R1/ /Users/rpg/src/rhino1_7R1/
ACL and SBCL /mostly/ agree, except here: rpg% ls -d /Users/rpg/src/URLHandler/ /Users/rpg/src/URLHandler/
SBCL: (translate-logical-pathname "MYSRC:URLHandler;")
#P"/Users/rpg/src/urlhandler/"
and
CL-USER(6): (translate-logical-pathname "MYSRC:URLHandler;") #P"/Users/rpg/src/URLHandler/"
I don't know whether this is a problem in general --- it doesn't cause me a problem on MacOSX because it doesn't have a truly case-sensitive filesystem:
back to SBCL:
(probe-file (translate-logical-pathname "MYSRC:URLHandler;"))
#P"/Users/rpg/src/URLHandler/"
Best, R
On Tue, Mar 30, 2010 at 4:19 PM, Robert Goldman rpgoldman@sift.info wrote:
On 3/29/10 Mar 29 -6:42 PM, Faré wrote:
One site per system looks like it will quickly pollute the host namespace. What about we instead use a single logical host with subdirectories?
#P"COMMON-LISP:SYSTEMS;CL-PPCRE;CL-PPCRE.ASD"
The problem I have with this approach with logical pathnames at all is that I don't understand whether or not mixed case and other characters are allowed at all.
They are not.
The syntax of logical pathname namestrings is
"word---one or more uppercase letters, digits, and hyphens."
This causes pain when working with people WhoLikeOtherProgrammingLanguages.
No, it does not cause any problem. If you have data that has weird names, but you know where they live in your system, then instead of using ASDF to query the paths you can simply query the logical hostname and merge the resulting path with the "nonstandard" components.
So, instead of calling ASDF to locate your system, query its pathname, etc, you would do
(defvar *my-code-base-path* (translate-logical-pathname "MY-SYSTEM:"))
(defvar *my-nonstandard-component* (merge-pathname "foo/MixedCaseWith#WeirdChars.dat" *my-code-base-path*))
The point is not whether logical pathnames satisfy everybody's requirements. I repeat again that the point is having the compiled or loaded systems have as few dependencies in any kind of ASDF API as possible, so that they can live with or without ASDF being present.
If we enforce that each compiled system gets an associated logical pathname it is very easy to set up a translation table at some point when systems are merged into a standalone program, or even do some kind of relocation "magic" to bundle multiple Common Lisp systems in a single Mac OS X applications where the location of things is determined when the application is launched.
Juanjo
On 2010-03-30, at 16:25 , Juan Jose Garcia-Ripoll wrote:
On Tue, Mar 30, 2010 at 4:19 PM, Robert Goldman rpgoldman@sift.info wrote: On 3/29/10 Mar 29 -6:42 PM, Faré wrote:
One site per system looks like it will quickly pollute the host namespace. What about we instead use a single logical host with subdirectories?
#P"COMMON-LISP:SYSTEMS;CL-PPCRE;CL-PPCRE.ASD"
The problem I have with this approach with logical pathnames at
all is
that I don't understand whether or not mixed case and other
characters
are allowed at all.
They are not.
The syntax of logical pathname namestrings is
"word---one or more uppercase letters, digits, and hyphens."
This causes pain when working with people WhoLikeOtherProgrammingLanguages.
No, it does not cause any problem. If you have data that has weird names, but you know where they live in your system,
designators for auxiliary files is not the issue.
then instead of using ASDF to query the paths you can simply query the logical hostname and merge the resulting path with the "nonstandard" components.
So, instead of calling ASDF to locate your system, query its pathname, etc, you would do
(defvar *my-code-base-path* (translate-logical-pathname "MY-SYSTEM:"))
(defvar *my-nonstandard-component* (merge-pathname "foo/ MixedCaseWith#WeirdChars.dat" *my-code-base-path*))
the problem arises when asdf itself needs to designate such files. a portable mechanism to achieve that approaches reimplementing the logical pathname subsystem.
just check out the pathname tests, put some nonconformant names in the test specification and watch them fail. differently in each implementation.
On 3/29/10 Mar 29 -4:00 PM, Juan Jose Garcia-Ripoll wrote:
One of the common problems that ASDF users faced when introducing the new binary location system is finding out where things reside.
Another problem we will eventually find is locating the resources associated to an ASDF system which has been replaced by a precompiled version.
In general it would be desirable for a system to be able to locate its resources (additional files, icons, bitmaps, etc) using logical pathnames.
For that purpose I have coded a small patch that adds a new keyword, :logical-host, to DEFSYSTEM. The keyword defines a logical pathname translation from "**;*.*.*" to "path/to/asdf/or/pathname/argument/**/*.*"
This way we may have
(defsystem :cl-unicode :logical-host "cl-unicode" ...)
and in the sources load the databases such as "cl-unicode:uppercase-table.dat"
FWIW, Faré provided a couple of new API functions to give ASDF users this function, notably
SYSTEM-RELATIVE-PATHNAME
and
SYSTEM-SOURCE-DIRECTORY
For what it's worth, I try to avoid logical-pathnames in portable code, having been hoist too many times on the petard of ACL/SBCL incompatibility in handling logical pathnames (SBCL sticks to the spec, in the name of portability and standardization; ACL accepts sensible, spec-defying logical pathnames in the name of utility).
Best, r
On Tue, Mar 30, 2010 at 4:06 PM, Robert Goldman rpgoldman@sift.info wrote:
FWIW, Faré provided a couple of new API functions to give ASDF users this function, notably
SYSTEM-RELATIVE-PATHNAME
and
SYSTEM-SOURCE-DIRECTORY
I insist on the following: in order to achieve purity in ASDF files we want to keep as far away as possible from using any kind of API in the compiled code.
The best system build system is one which is invisible to the code that it builds.
Exposing functions, imposing the dependency of the user code on ASDF API as well as imposing that the library code queries ASDF for its own system is a very bad design.
We want system definitions to be descriptive, not programatic, and we want the user code to be able to exist in an ASDF-free environment, as standalone systems. The existing situation does not allow this.
One may
1) Use *compile-file/load-truename* but that leads to very ugly code.
2) Enforce that *default-pathname-defaults* is set to the system root when loading or compiling
3) Provide logical hostnames
For what it's worth, I try to avoid logical-pathnames in portable code,
having been hoist too many times on the petard of ACL/SBCL incompatibility in handling logical pathnames (SBCL sticks to the spec, in the name of portability and standardization; ACL accepts sensible, spec-defying logical pathnames in the name of utility).
I do not have payware lisps, but I have been living with SBCL and ECL's logical pathnames without further problems.
Juanjo
A new implementation was just uploaded which implements the :logical-pathname argument for defsystem. It constructs four definitions, one using Faré's favourite no-namespace-clobbering scheme and three other ones that gives a bit more freedom to the developer, separating sources from "automatically" generated files.
The code does minimal collision checks. It will basically complain when there exists a translation that matches the one we are proposing but was not created by ASDF. If the user selects the CONTINUE restart, those translations will be deleted.
(defsystem :foo :logical-pathname "FOO:BASE" :components ((:file "faa")))
(let ((x "COMMON-LISP:SYSTEMS;FOO;FAA.LISP")) (format t "~&;;; Pathname ~S~%;;; translates into ~S" x (translate-logical-pathname x))) (let ((x "FOO:BASE;SOURCE;FAA.LISP")) ;; BASE;DATA;FAA.LISP is a synonym (format t "~&;;; Pathname ~S~%;;; translates into ~S" x (translate-logical-pathname x))) (let ((x "FOO:BASE;OUTPUT-FILES;FAA.LISP")) (format t "~&;;; Pathname ~S~%;;; translates into ~S" x (translate-logical-pathname x))) (terpri)
;;; Pathname "COMMON-LISP:SYSTEMS;FOO;FAA.LISP" ;;; translates into #P"/Users/jjgarcia/devel/asdf/faa.lisp" ;;; Pathname "FOO:BASE;SOURCE;FAA.LISP" ;;; translates into #P"/Users/jjgarcia/devel/asdf/faa.lisp" ;;; Pathname "FOO:BASE;OUTPUT-FILES;FAA.LISP" ;;; translates into #P"/Users/jjgarcia/devel/asdf/faa.lisp"
The attitude here is probably that this is useless and should be totally hidden in a contribution directory
Placing it in a "contrib" section means accepting it because of "politeness" or insistence of the submitter, but effectively making this useless, because then no library will be able to depend on it, for the :logical-pathname keyword will only be available when the user loads the appropriate extension.
Juanjo