Re: Working around nonconforming names in system definitions
On Sat, Sep 14, 2024 at 11:01 AM Robert P. Goldman <rpg@sift.net> wrote:
I’ve got to push back on your rant. There’s a perfectly good reason for those two limitations you dislike: they are the same as the limitations on logical pathnames. So if you wish to be able to find an ASDF system relative to a directory designated by a logical pathname, those limitations are pretty much forced on you.
All right. Help me sort this out. I actually want to make this work. I have a directory Foo_Bar and I want to create a .asd file (in any directory which is known to ASDF) which describes the stuff within Foo_Bar. From what I have sorted out so far, that .asd file must be named something other than "Foo_Bar.asd" and the system it describes must be named something other than "Foo_Bar". The ideal outcome is that a user would say (asdf:load-system "Foo_Bar") and that the system which describes the stuff in "Foo_Bar" would get loaded. How close can I get to that, and what do I have to do, involving logical pathnames if need be, in order to get it? I am thinking I need to do something like (I emphasize "like", as I am aware that the following code doesn't work): (setf (logical-pathname-translations "abc") ("**;foobar.asd" "Foo_Bar.asd")) and that must be executed before the system is defined (defsystem "foobar" ...) and before the user calls ASDF:LOAD-SYSTEM. (asdf:load-system "Foo_Bar") Can this work, at all? As I was saying, I actually want this (or anything) to work. I don't have any point to make except to get something working. Thanks for your help. Robert
If you're using logical pathnames, you're asking for trouble (especially on SBCL). I wasted weeks of my life on logical pathnames, and STRONGLY recommend that no one should ever use them anymore under any circumstances. (Though we did use them at ITA to get location-independent debug file information on CCL.) If you're not, there should be no problem having a system called Foo_Bar, is there? Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no? —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “The worst thing about totalitarian regimes is not that they make people poor, miserable and unfree — it’s that they corrupt people’s souls, and turn everyone into a double-thinking, double-speaking liar for the sake of survival.” On Sat, Sep 14, 2024 at 4:46 PM Robert Dodier <robert.dodier@gmail.com> wrote:
On Sat, Sep 14, 2024 at 11:01 AM Robert P. Goldman <rpg@sift.net> wrote:
I’ve got to push back on your rant. There’s a perfectly good reason for those two limitations you dislike: they are the same as the limitations on logical pathnames. So if you wish to be able to find an ASDF system relative to a directory designated by a logical pathname, those limitations are pretty much forced on you.
All right. Help me sort this out. I actually want to make this work.
I have a directory Foo_Bar and I want to create a .asd file (in any directory which is known to ASDF) which describes the stuff within Foo_Bar. From what I have sorted out so far, that .asd file must be named something other than "Foo_Bar.asd" and the system it describes must be named something other than "Foo_Bar".
The ideal outcome is that a user would say
(asdf:load-system "Foo_Bar")
and that the system which describes the stuff in "Foo_Bar" would get loaded. How close can I get to that, and what do I have to do, involving logical pathnames if need be, in order to get it?
I am thinking I need to do something like (I emphasize "like", as I am aware that the following code doesn't work):
(setf (logical-pathname-translations "abc") ("**;foobar.asd" "Foo_Bar.asd"))
and that must be executed before the system is defined
(defsystem "foobar" ...)
and before the user calls ASDF:LOAD-SYSTEM.
(asdf:load-system "Foo_Bar")
Can this work, at all? As I was saying, I actually want this (or anything) to work. I don't have any point to make except to get something working. Thanks for your help.
Robert
On Sat, Sep 14, 2024 at 8:45 PM Faré <fahree@gmail.com> wrote:
If you're using logical pathnames, you're asking for trouble (especially on SBCL). I wasted weeks of my life on logical pathnames, and STRONGLY recommend that no one should ever use them anymore under any circumstances.
Just to be clear, I don't *want* to use logical pathnames, but I will if it provides a solution to the problem.
If you're not, there should be no problem having a system called Foo_Bar, is there?
Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no?
I dunno, that doesn't seem to work ... I tried this in SBCL which has ASDF 3.3.something bundled. (asdf:load-system "Foo_Bar") provokes a warning: "WARNING: System definition file #P"/home/dodier/tmp/Foo_Bar.asd" contains definition for system "foo_bar". Please only define "Foo_Bar" and secondary systems with a name starting with "Foo_Bar/" (e.g. "Foo_Bar/test") in that file. and an error: Component "Foo_Bar" not found Anyway, the documentation says (https://asdf.common-lisp.dev/asdf.html#rule_002dsystem_002ddesignator) that "Foo_Bar" is an unacceptable name since it has both an underscore and uppercase letters. I find that (defsystem baz_quux ...) in a file named "baz_quux.asd" is loaded successfully without complaint -- I'm surprised that works; I guess the no underscores rule is not actually enforced? I guess the stated rules about simple component names are not actually enforced by the DEFSYSTEM macro itself ... the do-nothing system * (asdf:defsystem "Blurf_Mumble") * (asdf:operate 'asdf:load-op *) succeeds. I guess the stuff about string case only comes into play when the file system gets involved? FWIW & all the best. Robert
Just to be clear, I don't *want* to use logical pathnames, but I will if it provides a solution to the problem.
Logical pathnames will make anything WAAAAAYYYY worse.
Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no? Aha, sorry, I meant (defsystem "Foo_Bar" ...) because symbols are downcased.
Anyway, the documentation says (https://asdf.common-lisp.dev/asdf.html#rule_002dsystem_002ddesignator) that "Foo_Bar" is an unacceptable name since it has both an underscore and uppercase letters.
I don't know about that restriction against underscores. Looks like it's from commit 25bc69c a bit before 3.3.2.14. You'll have to ask Robert Goldman, but I believe it's a restriction that is necessary if you want to be compatible with your users using logical pathnames, but that might not apply in your case.
I find that (defsystem baz_quux ...) in a file named "baz_quux.asd" is loaded successfully without complaint -- I'm surprised that works; I guess the no underscores rule is not actually enforced?
It will be enforced with a vengeance against users of logical pathnames on SBCL. Not sure anywhere else. —♯ƒ • François-René Rideau • Chief Scientist, MuKn.com/fare “Remember kiddies: you can vote your way into communism but you have to shoot your way out.”
On Sun, Sep 15, 2024 at 9:00 AM Faré <fahree@gmail.com> wrote:
Just to be clear, I don't *want* to use logical pathnames, but I will if it provides a solution to the problem.
Logical pathnames will make anything WAAAAAYYYY worse.
I am trying to avoid them. I thought perhaps that it would be necessary -- appears that's not the case; see below.
Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no? Aha, sorry, I meant (defsystem "Foo_Bar" ...) because symbols are downcased.
Yes! (defsystem "Foo_Bar" ...) in a file named Foo_Bar.asd is located successfully by (asdf:load-system "Foo_Bar"). It's not supposed to work but it does anyway! Hurray! At this point can I ask the ASDF developers to resolve the inconsistency between the code and the documentation by making the documentation match the code instead of vice versa.
Anyway, the documentation says (https://asdf.common-lisp.dev/asdf.html#rule_002dsystem_002ddesignator) that "Foo_Bar" is an unacceptable name since it has both an underscore and uppercase letters.
I don't know about that restriction against underscores. Looks like it's from commit 25bc69c a bit before 3.3.2.14.
No, that commit is only updating the documentation *say* that underscores are rejected; it doesn't actually implement the test, not even in the most recent (3.3.7.1) version. (I tried it ASDF 3.3.7.1 with SBCL and CMUCL on Linux.) I looked at asdf/parse-defsystem.lisp and it doesn't mention underscore, low_line, _, 95, or 5f (in any case variants).
It will be enforced with a vengeance against users of logical pathnames on SBCL. Not sure anywhere else.
Well, if it's only enforced when logical pathnames are in effect, then the documentation should say that. Robert
Hi Robert, In afraid that the system lookup works for you but will not work in general so the documentation must remain as is . We support users who use logical pathnames. We also support users on case insensitive and case preserving file systems. For that matter, we permit existing systems like “foo” be referred to by ‘foo, and a system named “FOO” could not be designated by ‘foo. These restrictions were not just imposed arbitrarily: they have a rationale, as I’ve tried to explain. Yours, -Robert Sent from my iPhone On Sep 15, 2024 at 15:48:11 CDT, Robert Dodier <robert.dodier@gmail.com> wrote: On Sun, Sep 15, 2024 at 9:00 AM Faré <fahree@gmail.com> wrote:
Just to be clear, I don't *want* to use logical pathnames, but I will if it provides a solution to the problem.
Logical pathnames will make anything WAAAAAYYYY worse.
I am trying to avoid them. I thought perhaps that it would be necessary -- appears that's not the case; see below.
Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no? Aha, sorry, I meant (defsystem "Foo_Bar" ...) because symbols are downcased.
Yes! (defsystem "Foo_Bar" ...) in a file named Foo_Bar.asd is located successfully by (asdf:load-system "Foo_Bar"). It's not supposed to work but it does anyway! Hurray! At this point can I ask the ASDF developers to resolve the inconsistency between the code and the documentation by making the documentation match the code instead of vice versa.
Anyway, the documentation says (https://asdf.common-lisp.dev/asdf.html#rule_002dsystem_002ddesignator) that "Foo_Bar" is an unacceptable name since it has both an underscore and uppercase letters.
I don't know about that restriction against underscores. Looks like it's from commit 25bc69c a bit before 3.3.2.14.
No, that commit is only updating the documentation *say* that underscores are rejected; it doesn't actually implement the test, not even in the most recent (3.3.7.1) version. (I tried it ASDF 3.3.7.1 with SBCL and CMUCL on Linux.) I looked at asdf/parse-defsystem.lisp and it doesn't mention underscore, low_line, _, 95, or 5f (in any case variants).
It will be enforced with a vengeance against users of logical pathnames on SBCL. Not sure anywhere else.
Well, if it's only enforced when logical pathnames are in effect, then the documentation should say that. Robert
On Sun, Sep 15, 2024 at 4:00 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
In afraid that the system lookup works for you but will not work in general so the documentation must remain as is.
No, the documentation incorrectly implies that system names containing uppercase and underscores are invalid in general, when that only applies if logical pathnames are in the picture. I hope you'll clarify that. I read the documentation and showed up here because I thought the code actually works the way it is described. What a pleasant surprise to find out it doesn't!
We support users who use logical pathnames. We also support users on case insensitive and case preserving file systems.
OK. I don't want ASDF to somehow distinguish case on a case-insensitive file system. I only want ASDF to allow for case distinction if the file system already allows it. This is, in fact, how the code actually works at present -- I'm asking you to make that clear in the documentation.
These restrictions were not just imposed arbitrarily: they have a rationale, as I’ve tried to explain.
These aren't restrictions in general, so I'm asking you to make it clear that these restrictions only apply in special cases. Robert
Hi Robert, As far as I can tell, the current behavior is buggy, because it implies that ASDF will behave differently when the user chooses to use logical pathnames versus when they don’t. That’s not a Good Thing. That’s bad. Going back to Fare’s argument that ASDF should properly separate decision making into what’s appropriate for the user of the library and the developer of the library, this seems clearly wrong: it implies that the developer of the library gets to choose to make a system that will behave wrong if the user of the library happens to configure ASDF using logical pathnames, a behavior that the ASDF docs claim is fine. Since it’s up to the user to decide how to configure the source registry, it is inappropriate for ASDF to encourage developers to publish libraries that will break if users make a configuration decision that we claim is acceptable. If I was to alter the documentation to reflect this behavior, I would be describing something that is a bug as if it was a feature, and worse, would be committing ASDF to maintain this undesirable inconsistency going forward. On Sep 15, 2024 at 20:07:55 CDT, Robert Dodier <robert.dodier@gmail.com> wrote: On Sun, Sep 15, 2024 at 4:00 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
In afraid that the system lookup works for you but will not work in general so the documentation must remain as is.
No, the documentation incorrectly implies that system names containing uppercase and underscores are invalid in general, when that only applies if logical pathnames are in the picture. I hope you'll clarify that. I read the documentation and showed up here because I thought the code actually works the way it is described. What a pleasant surprise to find out it doesn't!
We support users who use logical pathnames. We also support users on case insensitive and case preserving file systems.
OK. I don't want ASDF to somehow distinguish case on a case-insensitive file system. I only want ASDF to allow for case distinction if the file system already allows it. This is, in fact, how the code actually works at present -- I'm asking you to make that clear in the documentation.
These restrictions were not just imposed arbitrarily: they have a rationale, as I’ve tried to explain.
These aren't restrictions in general, so I'm asking you to make it clear that these restrictions only apply in special cases. Robert
On Sun, Sep 15, 2024 at 6:30 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
As far as I can tell, the current behavior is buggy, because it implies that ASDF will behave differently when the user chooses to use logical pathnames versus when they don’t. That’s not a Good Thing. That’s bad.
Well, all right. You go ahead and fix it, since it's your project and you call the shots. But I will shed a few tears for humanity, and kick myself a couple of times for bringing to your attention this non-problem not in need of a solution. How many complaints have you received from people using logical pathnames about the current behavior of ASDF? Unless something has escaped my attention, that number is exactly zero, so I can pretty confidently say that in a concrete sense, you're not going to make anyone happy.
If I was to alter the documentation to reflect this behavior, I would be describing something that is a bug as if it was a feature, and worse, would be committing ASDF to maintain this undesirable inconsistency going forward.
You could, of course, just dump the stuff about logical pathnames. I've yet to encounter anybody ever suggesting that they are a good idea, except for that one guy who was talking about how easy it is to use them solely for the purpose of being an obnoxious prat. This whole episode is a classic case of the tail wagging the dog. The very presence of logical pathnames in the Common Lisp spec is an example of it, too -- some faction pushed it in late in the game (as it suited their commercial systems that were obsolete even before ANSI Common Lisp was adopted), and now, like the mariner with his albatross, we're saddled with it for eternity. Yikes. Robert
I'm not sure why this is making you so angry. I've given you multiple ideas for how to work around the behavior you don't like, but you seem to feel it's my job to make changes to behaviors that (even if not entirely correctly implemented) were settled on after quite a bit of thought. You complain about me intending to fix something that you feel isn't broken: something that I never proposed to do. At the same time, you want me to change the spec of ASDF to explicitly endorse breaking things for those who use logical pathnames, and potentially for users on certain file systems. The existing behavior attempts to maximize predictability and portability. I certainly don't claim it's perfect, but that's not sufficient reason for me to throw portability under the bus. Portability will *always* entail sacrificing convenience and capability at some point. I'm not a fan of logical pathnames, but it's not ASDF's job to unilaterally prune them from the language or support behavior that punishes those who wish to use them. You seem to think I owe you something here, and that you are entitled to feel aggrieved that I don't give it to you. Have you looked at the ASDF issue and merge request backlog lately? I'm doing the best I can with next to no help from anyone else, and certainly more complaints than assistance. On 16 Sep 2024, at 1:16, Robert Dodier wrote:
On Sun, Sep 15, 2024 at 6:30 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
As far as I can tell, the current behavior is buggy, because it implies that ASDF will behave differently when the user chooses to use logical pathnames versus when they don’t. That’s not a Good Thing. That’s bad.
Well, all right. You go ahead and fix it, since it's your project and you call the shots. But I will shed a few tears for humanity, and kick myself a couple of times for bringing to your attention this non-problem not in need of a solution. How many complaints have you received from people using logical pathnames about the current behavior of ASDF? Unless something has escaped my attention, that number is exactly zero, so I can pretty confidently say that in a concrete sense, you're not going to make anyone happy.
If I was to alter the documentation to reflect this behavior, I would be describing something that is a bug as if it was a feature, and worse, would be committing ASDF to maintain this undesirable inconsistency going forward.
You could, of course, just dump the stuff about logical pathnames. I've yet to encounter anybody ever suggesting that they are a good idea, except for that one guy who was talking about how easy it is to use them solely for the purpose of being an obnoxious prat.
This whole episode is a classic case of the tail wagging the dog. The very presence of logical pathnames in the Common Lisp spec is an example of it, too -- some faction pushed it in late in the game (as it suited their commercial systems that were obsolete even before ANSI Common Lisp was adopted), and now, like the mariner with his albatross, we're saddled with it for eternity. Yikes.
Robert
Sorry, that was a bit over the top. I was having a bad day, I'll be more considerate. I don't actually think that you are obliged to change anything to suit me. I'll figure something out, I'm sure, I always have so far. Robert
Hi, Robert -- Thank you very much for the gracious email. Sorry not to have answered it promptly: got it just when I was leaving town for a long weekend. If I can help you fix the names in any way, feel free to ask. R On 17 Sep 2024, at 12:18, Robert Dodier wrote:
Sorry, that was a bit over the top. I was having a bad day, I'll be more considerate. I don't actually think that you are obliged to change anything to suit me.
I'll figure something out, I'm sure, I always have so far.
Robert
:rpg Going back to Fare’s argument that ASDF should properly separate decision making into what’s appropriate for the user of the library and the developer of the library, this seems clearly wrong: it implies that the developer of the library gets to choose to make a system that will behave wrong if the user of the library happens to configure ASDF using logical pathnames, a behavior that the ASDF docs claim is fine. Since it’s up to the user to decide how to configure the source registry, it is inappropriate for ASDF to encourage developers to publish libraries that will break if users make a configuration decision that we claim is acceptable.
Agreed, with a small caveat: if one is trying to publish a library usable by CLers in general, one should absolutely restrict oneself to lowercase names (or symbols, that will be downcased) containing only letters, numbers and dashes as allowed in logical-pathnames (CLHS says it's uppercase letters, and in practice, it's case-converted down in the system name, up in the logical pathname, down in the physical pathname) https://www.lispworks.com/documentation/HyperSpec/Body/19_ca.htm On the other hand, if the system remains local only and is not published, then any string accepted by your implementation should do. Probably, the nature of the constraint should be explained somewhere in the manual. —♯ƒ • François-René Rideau • Chief Scientist, MuKn.com/fare “Not all Law is created equal before Man. Some Law causes least conflict and least perverse incentives. By definition we call it Natural Law.”
Hi , Just to clarify: the restrictions arose because your users’ ASDF source configuration could use logical pathnames. I tend to agree with Faré that logical pathnames are to be avoided whenever possible, but ASDF doesn’t impose that opinion on its users. Since we don’t, we require that file names — and therefore system names — be compatible with logical pathnames. One could certainly override the way ASDF maps system names to file names, as I said earlier. But as I also said earlier, that’s probably more trouble than it’s worth. It would require understanding the ASDF internals extensively and also would make use of the libraries more complex and less predictable Thanks, -Robert Sent from my iPhone On Sep 14, 2024 at 22:44:47 CDT, Faré <fahree@gmail.com> wrote: If you're using logical pathnames, you're asking for trouble (especially on SBCL). I wasted weeks of my life on logical pathnames, and STRONGLY recommend that no one should ever use them anymore under any circumstances. (Though we did use them at ITA to get location-independent debug file information on CCL.) If you're not, there should be no problem having a system called Foo_Bar, is there? Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no? —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “The worst thing about totalitarian regimes is not that they make people poor, miserable and unfree — it’s that they corrupt people’s souls, and turn everyone into a double-thinking, double-speaking liar for the sake of survival.” On Sat, Sep 14, 2024 at 4:46 PM Robert Dodier <robert.dodier@gmail.com> wrote:
On Sat, Sep 14, 2024 at 11:01 AM Robert P. Goldman <rpg@sift.net> wrote:
I’ve got to push back on your rant. There’s a perfectly good reason for those two limitations you dislike: they are the same as the limitations on logical pathnames. So if you wish to be able to find an ASDF system relative to a directory designated by a logical pathname, those limitations are pretty much forced on you.
All right. Help me sort this out. I actually want to make this work.
I have a directory Foo_Bar and I want to create a .asd file (in any directory which is known to ASDF) which describes the stuff within Foo_Bar. From what I have sorted out so far, that .asd file must be named something other than "Foo_Bar.asd" and the system it describes must be named something other than "Foo_Bar".
The ideal outcome is that a user would say
(asdf:load-system "Foo_Bar")
and that the system which describes the stuff in "Foo_Bar" would get loaded. How close can I get to that, and what do I have to do, involving logical pathnames if need be, in order to get it?
I am thinking I need to do something like (I emphasize "like", as I am aware that the following code doesn't work):
(setf (logical-pathname-translations "abc") ("**;foobar.asd" "Foo_Bar.asd"))
and that must be executed before the system is defined
(defsystem "foobar" ...)
and before the user calls ASDF:LOAD-SYSTEM.
(asdf:load-system "Foo_Bar")
Can this work, at all? As I was saying, I actually want this (or anything) to work. I don't have any point to make except to get something working. Thanks for your help.
Robert
participants (4)
-
Faré -
Robert Dodier -
Robert Goldman -
Robert P. Goldman