*load-pathname* and *compile-file-pathname* issues
Hi I am getting old and I get confused. Here is the thing (a followup to the "wild" ASDF module question). I am writing something that will have, at run time, a variable * the-directory-where-stuff-is*. Now, I want to initialize this variable automagically, with no "outside" references. I thought I could do something like (defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*))) Alas, ASDF and other things may move the compiled file to a cache folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be). This is something that build systems like cmake achieve with the notion of "source", "install", etc folders. Any ideas, suggestions, incantations, workarounds, even at the ASDF level to get this done? Thanks -- Marco Antoniotti
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable *the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no "outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF level to get this done?
Thanks
-- Marco Antoniotti
Thanks but no. That would create a dependency from ASDF in the code. MA On Sat, Dec 13, 2025 at 1:50 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable *the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no "outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF level to get this done?
Thanks
-- Marco Antoniotti
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY REGAINS: https://regains.disco.unimib.it/
... update. I bit the bullet and kludged up something like (defparameter *stuff-dir* (and (asdf:find-system "stuff" nil) (asdf:system-source-directory "stuff")) It works for me. The asdf:find-system is probably overkill, but better safe than sorry. MA On Sat, Dec 13, 2025 at 4:16 PM Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
Thanks but no.
That would create a dependency from ASDF in the code.
MA
On Sat, Dec 13, 2025 at 1:50 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable *the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no "outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF level to get this done?
Thanks
-- Marco Antoniotti
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY REGAINS: https://regains.disco.unimib.it/
There are many issues with using ASDF at runtime: the source may have moved, been updated ou replaced with an older version, the ASDF configuration of the user can be wrong, there can be a lot of subdirectories under the trees registered in the source-registry causing a long pause when ASDF starts, etc. Then again there are just as many issues with requiring a program to install its data in e.g. data directories from XDG_DATA_DIRS and such. But if you want to try that, see the xdg- functions in UIOP. In the end, the question is whether the target audience is CL developers staying in their CL world, or regular Unix (or Windows, macOS) users expecting things under regular paths in their filesystem. —♯ƒ • François-René Rideau • Chief Scientist, MuKn.com/fare “Quality means doing it right when no one is looking.” — Henry Ford “Morality means doing the right thing when no one is looking.” — Jan Krepelka On Sat, Dec 13, 2025 at 12:40 PM Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
... update.
I bit the bullet and kludged up something like
(defparameter *stuff-dir* (and (asdf:find-system "stuff" nil) (asdf:system-source-directory "stuff"))
It works for me.
The asdf:find-system is probably overkill, but better safe than sorry.
MA
On Sat, Dec 13, 2025 at 4:16 PM Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
Thanks but no.
That would create a dependency from ASDF in the code.
MA
On Sat, Dec 13, 2025 at 1:50 PM Robert P. Goldman <rpgoldman@sift.net> wrote:
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable *the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no "outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF level to get this done?
Thanks
-- Marco Antoniotti
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
I think the machinery the ANS intended to handle this sort of thing (allowing a portable CL application to know at run time where an application was installed long after file compilation or load time) was logical pathnames. Of course, the detailed behavior of LP translation is so poorly constrained by the ANS that using LPs just squeezes the portability toothpaste from one end of the tube to the other, out through the nozzle, and onto your keyboard. Some of the incomprehensibility and system-dependent behavior of LPs could have been fixed or at least improved if LP matching and translation could have been defined in terms of user-coded functions, but we were tired and didn't think of it at the time. On Sat, Dec 13, 2025 at 10:21 AM Faré <fahree@gmail.com> wrote:
There are many issues with using ASDF at runtime: the source may have moved, been updated ou replaced with an older version, the ASDF configuration of the user can be wrong, there can be a lot of subdirectories under the trees registered in the source-registry causing a long pause when ASDF starts, etc.
Then again there are just as many issues with requiring a program to install its data in e.g. data directories from XDG_DATA_DIRS and such. But if you want to try that, see the xdg- functions in UIOP.
In the end, the question is whether the target audience is CL developers staying in their CL world, or regular Unix (or Windows, macOS) users expecting things under regular paths in their filesystem.
—♯ƒ • François-René Rideau • Chief Scientist, MuKn.com/fare “Quality means doing it right when no one is looking.” — Henry Ford “Morality means doing the right thing when no one is looking.” — Jan Krepelka
On Sat, Dec 13, 2025 at 12:40 PM Marco Antoniotti <marco.antoniotti@unimib.it> wrote:
... update.
I bit the bullet and kludged up something like
(defparameter *stuff-dir* (and (asdf:find-system "stuff" nil) (asdf:system-source-directory "stuff"))
It works for me.
The asdf:find-system is probably overkill, but better safe than sorry.
MA
On Sat, Dec 13, 2025 at 4:16 PM Marco Antoniotti <
marco.antoniotti@unimib.it> wrote:
Thanks but no.
That would create a dependency from ASDF in the code.
MA
On Sat, Dec 13, 2025 at 1:50 PM Robert P. Goldman <rpgoldman@sift.net>
wrote:
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti <
marco.antoniotti@unimib.it> wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable
*the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no
"outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache
folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the
notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF
level to get this done?
Thanks
-- Marco Antoniotti
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
Here is what according to me might be a portable way to catch the source pathname: ``` (defmacro source-pathname () `,(or *compile-file-truename* *load-truename*)) ``` Then at the top of your file that you want to know what was the source directory: ``` (defparameter *project-dir* (make-pathname :name nil :type nil :defaults (source-pathname))) ``` This source-pathname is a standard function or macro defined for Gendl in its so-called "glisp" portability layer. I use that actual macro for SBCL because I couldn't find a built in thing to do it. Most of the other CLs have a built-in way e.g. Allegro's excl:*source-pathname* or CCL's ccl:*loading-file-source-file*. I think the macro should be portable though so maybe I should just be using it across the board. Dave Cooper
On Sat, Dec 13, 2025 at 4:16 PM Marco Antoniotti < mailto:marco.antoniotti@unimib.it > wrote:
Thanks but no.
That would create a dependency from ASDF in the code.
MA
On Sat, Dec 13, 2025 at 1:50 PM Robert P. Goldman < mailto:rpgoldman@sift.net > wrote:
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti < mailto:marco.antoniotti@unimib.it > wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable *the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no "outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF level to get this done?
Thanks
-- Marco Antoniotti
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
Thanks Dave Let me understand how this works.... If the file gets compiled (whatever invokes compile-file) the macro effectively freezes *compile-file-truename* in the code. When I load the compiled file, that pathname is there, even if the compiled file now resides elsewhere. If the file just gets loaded as source, then that's it and we do not have issues. Did I get it right? Nice. MA On Sun, Dec 14, 2025 at 4:13 AM David Cooper <david.cooper@genworks.com> wrote:
Here is what according to me might be a portable way to catch the source pathname:
``` (defmacro source-pathname () `,(or *compile-file-truename* *load-truename*)) ```
Then at the top of your file that you want to know what was the source directory:
``` (defparameter *project-dir* (make-pathname :name nil :type nil :defaults (source-pathname)))
```
This source-pathname is a standard function or macro defined for Gendl in its so-called "glisp" portability layer. I use that actual macro for SBCL because I couldn't find a built in thing to do it. Most of the other CLs have a built-in way e.g. Allegro's excl:*source-pathname* or CCL's ccl:*loading-file-source-file*.
I think the macro should be portable though so maybe I should just be using it across the board.
Dave Cooper
On Sat, Dec 13, 2025 at 4:16 PM Marco Antoniotti <
marco.antoniotti@unimib.it> wrote:
Thanks but no.
That would create a dependency from ASDF in the code.
MA
On Sat, Dec 13, 2025 at 1:50 PM Robert P. Goldman <rpgoldman@sift.net>
wrote:
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti <
marco.antoniotti@unimib.it> wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable
*the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no
"outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache
folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the
notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF
level to get this done?
Thanks
-- Marco Antoniotti
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY REGAINS: https://regains.disco.unimib.it/
Hi Marco, ---- On Sun, 14 Dec 2025 03:41:47 -0500 Marco Antoniotti < mailto:marco.antoniotti@unimib.it > wrote --- Thanks Dave Let me understand how this works.... If the file gets compiled (whatever invokes compile-file) the macro effectively freezes *compile-file-truename* in the code. When I load the compiled file, that pathname is there, even if the compiled file now resides elsewhere. If the file just gets loaded as source, then that's it and we do not have issues. Did I get it right? Nice. MA That is basically how I understand it is working, yes. And I think I got this originally from someone on #commonlisp IRC libera.chat when I was asking about this exact kind of stuff over there. A search of the logs for that channel might reveal who produced that nugget. On Sun, Dec 14, 2025 at 4:13 AM David Cooper < mailto:david.cooper@genworks.com > wrote: -- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY REGAINS: https://regains.disco.unimib.it/ Here is what according to me might be a portable way to catch the source pathname: ``` (defmacro source-pathname () `,(or *compile-file-truename* *load-truename*)) ``` Then at the top of your file that you want to know what was the source directory: ``` (defparameter *project-dir* (make-pathname :name nil :type nil :defaults (source-pathname))) ``` This source-pathname is a standard function or macro defined for Gendl in its so-called "glisp" portability layer. I use that actual macro for SBCL because I couldn't find a built in thing to do it. Most of the other CLs have a built-in way e.g. Allegro's excl:*source-pathname* or CCL's ccl:*loading-file-source-file*. I think the macro should be portable though so maybe I should just be using it across the board. Dave Cooper
On Sat, Dec 13, 2025 at 4:16 PM Marco Antoniotti < mailto:marco.antoniotti@unimib.it > wrote:
Thanks but no.
That would create a dependency from ASDF in the code.
MA
On Sat, Dec 13, 2025 at 1:50 PM Robert P. Goldman < mailto:rpgoldman@sift.net > wrote:
ASDF:SYSTEM-RELATIVE-PATHNAME is probably what you want Sent from my iPhone
On Dec 13, 2025, at 06:11, Marco Antoniotti < mailto:marco.antoniotti@unimib.it > wrote:
Hi
I am getting old and I get confused.
Here is the thing (a followup to the "wild" ASDF module question).
I am writing something that will have, at run time, a variable *the-directory-where-stuff-is*.
Now, I want to initialize this variable automagically, with no "outside" references. I thought I could do something like
(defvar the-directory-where-stuff-is* (merge-pathnames (make-pathname :directory '(:relative "stuff") :name nil :type nil) (make-pathname :name nil :type nil :defaults *load-pathname*)))
Alas, ASDF and other things may move the compiled file to a cache folder, which, in the end misaligns *load-pathname* w.r.t. where "stuff" actually is (and should be).
This is something that build systems like cmake achieve with the notion of "source", "install", etc folders.
Any ideas, suggestions, incantations, workarounds, even at the ASDF level to get this done?
Thanks
-- Marco Antoniotti
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
-- Marco Antoniotti, Professor, Director tel. +39 - 02 64 48 79 01 DISCo, University of Milan-Bicocca U14 2043 http://dcb.disco.unimib.it Viale Sarca 336 I-20126 Milan (MI) ITALY
REGAINS: https://regains.disco.unimib.it/
Of course, the detailed behavior of LP translation is so poorly constrained by the ANS that using LPs just squeezes the portability toothpaste from one end of the tube to the other, out through the nozzle, and onto your keyboard. You'd think it's toothpaste, but actually it's radioactive sludge. Don't touch it. Do go anywhere near it. Keep it away from kids, keep it away from adults without a hazmat suit.
Making UIOP and ASDF work with "Logical" pathnames was hell. And it won't actually "work" that well: between limitations that SBCL enforces but not others implementations, quirks and bugs of many implementations, atrocious performance (guaranteed O(N)), and impossibility for non-gurus to configure, much less debug... it's 100% worse than using a variable and uiop:subpathname and/or pathname-translations, 100% of the time. Only valid use I've ever seen was to have location-independent debug information on CCL, but that was CCL-only magic. Oh yeah, did you know that it's impossible to not have a HOST? And on some implementation, the HOST will determine the syntax and various limitations, such that you cannot, conceivably, have a portable representation of pathname in CL, even less so in presence of Logical Pathnames? Not that any of the standard functions (especially NOT merge-pathnames, as you might like to believe) is designed to deal with relative pathnames properly. It took me months to get UIOP to kind of work correctly on portably most of the time for most of the cases on most implementations... but absolutely not, never, not possible using Logical Pathnames—UIOP will do its best, but the best isn't very good. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “There are just two rules of governance in a free society: Mind your own business. Keep your hands to yourself.” — P. J. O'Rourke
Hi Faré, and thanks for remaining active on these mailing lists, I'm getting that you're spread thin these days. Could you (or anyone who feels competent to explain it) tell us again what is a Logical Pathname (LP) and how does it differ from a Pathname object in general (as printed and readable by #P"some-string-representing-a-pathname") ? i have CLtL2 as well as Franz' "Common Lisp: The Reference" here on my shelf but, true confessions time, I have rarely cracked these books and I find them somewhat intimidating just to open and read for no specific reason. If anyone's explanations of pathnames and logical pathnames can refer to specific sections or passages from those books, I might dare crack them to look at those sections. One question I have is: Are Pathnames evil in general (and we should be working with bare strings as "pathname designators")? Or is it Logical Pathnames specifically that are evil. Is a logical pathname a thing with the VMS-looking path syntax, e.g. `sys:some;logical;path;` as opposed to a Pathname object which is something like a struct with 6 slots (directory, name, type, host, device, version)? Maybe the concepts are orthogonal, i.e. a Pathname object's string #P"<some-string>" can be of "native" format or in "logical pathname" format? So we could potentially get things like #P"/opt/gendl-ccl/gdl/program/gdl-ccl" or #P"sys:gdl-ccl". And in the first case, HOST, DEVICE, VERSION would be unspecified, while in the second case the HOST is "sys:" and DEVICE and VERSION are still unspecified, or can a given HOST come along with its own default notion of DEVICE and VERSION just as it comes along with its notion of DIRECTORY, or do those other defaults come strictly from *default-pathname-defaults* ? And is the Logical Pathname format meant to be able to accomodate all six of the slots in the Pathname object -- is that the connection between Pathnames and Logical Pathnames? None of these are burning questions, or blocking my productivity as far as I know (I tend to get things working by hook or by crook and leave them alone, and knock wood, pathnames per se have not been one of my big bottlenecks lately). But I am curious to understand the systemic issues better i.e. if there are an identifiable finite list of shortcomings in the current ANSI spec that we could at least quarantine or create some kind of polyfill for it, although if I had understood Faré completely, maybe I would have understood that uiop is the closest thing we currently have to a "polyfill." Dave Cooper ---- On Mon, 15 Dec 2025 03:30:49 -0500 Faré <fahree@gmail.com> wrote ---
Of course, the detailed behavior of LP translation is so poorly constrained by the ANS that using LPs just squeezes the portability toothpaste from one end of the tube to the other, out through the nozzle, and onto your keyboard. You'd think it's toothpaste, but actually it's radioactive sludge. Don't touch it. Do go anywhere near it. Keep it away from kids, keep it away from adults without a hazmat suit.
Making UIOP and ASDF work with "Logical" pathnames was hell. And it won't actually "work" that well: between limitations that SBCL enforces but not others implementations, quirks and bugs of many implementations, atrocious performance (guaranteed O(N)), and impossibility for non-gurus to configure, much less debug... it's 100% worse than using a variable and uiop:subpathname and/or pathname-translations, 100% of the time. Only valid use I've ever seen was to have location-independent debug information on CCL, but that was CCL-only magic. Oh yeah, did you know that it's impossible to not have a HOST? And on some implementation, the HOST will determine the syntax and various limitations, such that you cannot, conceivably, have a portable representation of pathname in CL, even less so in presence of Logical Pathnames? Not that any of the standard functions (especially NOT merge-pathnames, as you might like to believe) is designed to deal with relative pathnames properly. It took me months to get UIOP to kind of work correctly on portably most of the time for most of the cases on most implementations... but absolutely not, never, not possible using Logical Pathnames—UIOP will do its best, but the best isn't very good. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “There are just two rules of governance in a free society: Mind your own business. Keep your hands to yourself.” — P. J. O'Rourke
Logical pathnames were invented for the Lisp Machine. I believe the purpose was to introduce a level of indirection between pathnames as written in code (e.g. in 'defsystem' forms) and the actual physical locations, which could be on any machine on the LAN, running any of several operating systems. Then the choice of physical location would be a matter of site configuration, rather than requiring touching source code. I recall using logical pathnames on the LispM, and finding them clunky but not unusable. I believe much of the clunkiness originated from the fact that for the first several years of LispM development, the sources were kept on the MIT-AI PDP-10 running ITS, the aptly named Incompatible Timesharing System developed at MIT for its own use. ITS had a very restrictive filename syntax: IIRC, digits and uppercase letters only, with a 6-character limit. This, as I recall, allowed a filename to fit in a single 36-bit machine word. I haven't tried to use logical pathnames much in CL, so I don't have firsthand experience like Faré's, but I don't doubt his report. Perhaps they were usable as long as there was only one implementation, but for CL, there seems to have been a combination of them being underspecified and implementors not being familiar with how they worked on the LispM. -- Scott On Mon, Dec 15, 2025, 8:32 AM David Cooper <david.cooper@genworks.com> wrote:
Hi Faré, and thanks for remaining active on these mailing lists, I'm getting that you're spread thin these days.
Could you (or anyone who feels competent to explain it) tell us again what is a Logical Pathname (LP) and how does it differ from a Pathname object in general (as printed and readable by #P"some-string-representing-a-pathname") ? i have CLtL2 as well as Franz' "Common Lisp: The Reference" here on my shelf but, true confessions time, I have rarely cracked these books and I find them somewhat intimidating just to open and read for no specific reason. If anyone's explanations of pathnames and logical pathnames can refer to specific sections or passages from those books, I might dare crack them to look at those sections.
One question I have is: Are Pathnames evil in general (and we should be working with bare strings as "pathname designators")? Or is it Logical Pathnames specifically that are evil. Is a logical pathname a thing with the VMS-looking path syntax, e.g. `sys:some;logical;path;` as opposed to a Pathname object which is something like a struct with 6 slots (directory, name, type, host, device, version)? Maybe the concepts are orthogonal, i.e. a Pathname object's string #P"<some-string>" can be of "native" format or in "logical pathname" format? So we could potentially get things like #P"/opt/gendl-ccl/gdl/program/gdl-ccl" or #P"sys:gdl-ccl". And in the first case, HOST, DEVICE, VERSION would be unspecified, while in the second case the HOST is "sys:" and DEVICE and VERSION are still unspecified, or can a given HOST come along with its own default notion of DEVICE and VERSION just as it comes along with its notion of DIRECTORY, or do those other defaults come strictly from *default-pathname-defaults* ? And is the Logical Pathname format meant to be able to accomodate all six of the slots in the Pathname object -- is that the connection between Pathnames and Logical Pathnames?
None of these are burning questions, or blocking my productivity as far as I know (I tend to get things working by hook or by crook and leave them alone, and knock wood, pathnames per se have not been one of my big bottlenecks lately). But I am curious to understand the systemic issues better i.e. if there are an identifiable finite list of shortcomings in the current ANSI spec that we could at least quarantine or create some kind of polyfill for it, although if I had understood Faré completely, maybe I would have understood that uiop is the closest thing we currently have to a "polyfill."
Dave Cooper
---- On Mon, 15 Dec 2025 03:30:49 -0500 *Faré <fahree@gmail.com <fahree@gmail.com>>* wrote ---
Of course, the detailed behavior of LP translation is so poorly constrained by the ANS that using LPs just squeezes the portability toothpaste from one end of the tube to the other, out through the nozzle, and onto your keyboard. You'd think it's toothpaste, but actually it's radioactive sludge. Don't touch it. Do go anywhere near it. Keep it away from kids, keep it away from adults without a hazmat suit.
Making UIOP and ASDF work with "Logical" pathnames was hell. And it won't actually "work" that well: between limitations that SBCL enforces but not others implementations, quirks and bugs of many implementations, atrocious performance (guaranteed O(N)), and impossibility for non-gurus to configure, much less debug... it's 100% worse than using a variable and uiop:subpathname and/or pathname-translations, 100% of the time. Only valid use I've ever seen was to have location-independent debug information on CCL, but that was CCL-only magic.
Oh yeah, did you know that it's impossible to not have a HOST? And on some implementation, the HOST will determine the syntax and various limitations, such that you cannot, conceivably, have a portable representation of pathname in CL, even less so in presence of Logical Pathnames? Not that any of the standard functions (especially NOT merge-pathnames, as you might like to believe) is designed to deal with relative pathnames properly. It took me months to get UIOP to kind of work correctly on portably most of the time for most of the cases on most implementations... but absolutely not, never, not possible using Logical Pathnames—UIOP will do its best, but the best isn't very good.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “There are just two rules of governance in a free society: Mind your own business. Keep your hands to yourself.” — P. J. O'Rourke
The real reason they don't work well is that nowadays everything more or less has a POSIX file system, and we expect more substantial and consistent filesystem access through pathnames. LPs were developed, among other things, to support an ecosystem that had wildly different filesystems. That diversity has been weeded out in the time since the language was designed, and the standard was made. So, for example, we expect directories to work consistently across implementations, but LPs were made for a world in which one could expect little of directories. On 15 Dec 2025, at 13:51, Scott L. Burson wrote:
Logical pathnames were invented for the Lisp Machine. I believe the purpose was to introduce a level of indirection between pathnames as written in code (e.g. in 'defsystem' forms) and the actual physical locations, which could be on any machine on the LAN, running any of several operating systems. Then the choice of physical location would be a matter of site configuration, rather than requiring touching source code.
I recall using logical pathnames on the LispM, and finding them clunky but not unusable. I believe much of the clunkiness originated from the fact that for the first several years of LispM development, the sources were kept on the MIT-AI PDP-10 running ITS, the aptly named Incompatible Timesharing System developed at MIT for its own use. ITS had a very restrictive filename syntax: IIRC, digits and uppercase letters only, with a 6-character limit. This, as I recall, allowed a filename to fit in a single 36-bit machine word.
I haven't tried to use logical pathnames much in CL, so I don't have firsthand experience like Faré's, but I don't doubt his report. Perhaps they were usable as long as there was only one implementation, but for CL, there seems to have been a combination of them being underspecified and implementors not being familiar with how they worked on the LispM.
-- Scott
On Mon, Dec 15, 2025, 8:32 AM David Cooper <david.cooper@genworks.com> wrote:
Hi Faré, and thanks for remaining active on these mailing lists, I'm getting that you're spread thin these days.
Could you (or anyone who feels competent to explain it) tell us again what is a Logical Pathname (LP) and how does it differ from a Pathname object in general (as printed and readable by #P"some-string-representing-a-pathname") ? i have CLtL2 as well as Franz' "Common Lisp: The Reference" here on my shelf but, true confessions time, I have rarely cracked these books and I find them somewhat intimidating just to open and read for no specific reason. If anyone's explanations of pathnames and logical pathnames can refer to specific sections or passages from those books, I might dare crack them to look at those sections.
One question I have is: Are Pathnames evil in general (and we should be working with bare strings as "pathname designators")? Or is it Logical Pathnames specifically that are evil. Is a logical pathname a thing with the VMS-looking path syntax, e.g. `sys:some;logical;path;` as opposed to a Pathname object which is something like a struct with 6 slots (directory, name, type, host, device, version)? Maybe the concepts are orthogonal, i.e. a Pathname object's string #P"<some-string>" can be of "native" format or in "logical pathname" format? So we could potentially get things like #P"/opt/gendl-ccl/gdl/program/gdl-ccl" or #P"sys:gdl-ccl". And in the first case, HOST, DEVICE, VERSION would be unspecified, while in the second case the HOST is "sys:" and DEVICE and VERSION are still unspecified, or can a given HOST come along with its own default notion of DEVICE and VERSION just as it comes along with its notion of DIRECTORY, or do those other defaults come strictly from *default-pathname-defaults* ? And is the Logical Pathname format meant to be able to accomodate all six of the slots in the Pathname object -- is that the connection between Pathnames and Logical Pathnames?
None of these are burning questions, or blocking my productivity as far as I know (I tend to get things working by hook or by crook and leave them alone, and knock wood, pathnames per se have not been one of my big bottlenecks lately). But I am curious to understand the systemic issues better i.e. if there are an identifiable finite list of shortcomings in the current ANSI spec that we could at least quarantine or create some kind of polyfill for it, although if I had understood Faré completely, maybe I would have understood that uiop is the closest thing we currently have to a "polyfill."
Dave Cooper
---- On Mon, 15 Dec 2025 03:30:49 -0500 *Faré <fahree@gmail.com <fahree@gmail.com>>* wrote ---
Of course, the detailed behavior of LP translation is so poorly constrained by the ANS that using LPs just squeezes the portability toothpaste from one end of the tube to the other, out through the nozzle, and onto your keyboard. You'd think it's toothpaste, but actually it's radioactive sludge. Don't touch it. Do go anywhere near it. Keep it away from kids, keep it away from adults without a hazmat suit.
Making UIOP and ASDF work with "Logical" pathnames was hell. And it won't actually "work" that well: between limitations that SBCL enforces but not others implementations, quirks and bugs of many implementations, atrocious performance (guaranteed O(N)), and impossibility for non-gurus to configure, much less debug... it's 100% worse than using a variable and uiop:subpathname and/or pathname-translations, 100% of the time. Only valid use I've ever seen was to have location-independent debug information on CCL, but that was CCL-only magic.
Oh yeah, did you know that it's impossible to not have a HOST? And on some implementation, the HOST will determine the syntax and various limitations, such that you cannot, conceivably, have a portable representation of pathname in CL, even less so in presence of Logical Pathnames? Not that any of the standard functions (especially NOT merge-pathnames, as you might like to believe) is designed to deal with relative pathnames properly. It took me months to get UIOP to kind of work correctly on portably most of the time for most of the cases on most implementations... but absolutely not, never, not possible using Logical Pathnames—UIOP will do its best, but the best isn't very good.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “There are just two rules of governance in a free society: Mind your own business. Keep your hands to yourself.” — P. J. O'Rourke
Robert P. Goldman Research Fellow Smart Information Flow Technologies (d/b/a SIFT, LLC) 319 N. First Ave., Suite 400 Minneapolis, MN 55401 Google Voice: (612) 326-3934 Cell: (612) 384-3454 Email: rpgoldman@SIFT.net
One thing that made Logical Pathnames usable on Lisp Machines was that big universities and research centers could afford a system administrator to set up the logical pathnames for those users who couldn't or wouldn't do system administration. Look at some Lisp "libraries" of the 1980s-1990s—they pretty much require that kind of setup. It is not reasonable to require a random unix luser to edit his ~/.sbclrc (and similarly for any other implementation) to setup tens of mappings with cryptic *.*.* wildcards, and then learn the fine semantics of logical-pathnames to be able to maintain it, just so he wants to use some random program that happened to be written in Lisp. I remember the effort of common-lisp-controller (a.k.a c-l-c), in the days of ASDF 1, to provide standardized access to Lisp software on debian (with feeble attempts to make it work beyond debian). It was hell to maintain for too small results. Moving this cruft into ASDF itself, fixing ASDF where needed, and adding a nicely-configured-with-good-defaults source-registry to ASDF was one of the great victories of ASDF2, IMNSHO. Importantly: it works without a system administrator, or deep system integration like c-l-c. But ASDF still requires a user who only compiles one program at a time into his binary cache. You can't run random Lisp scripts that compile stuff in the background without some eventual painful conflict; your scripts have to be either interpreted, precompiled into an image, or flaky. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org «A “pragmatist” is someone who admits that his own theory is worthless, and his practice baseless—yet claims he knows better than you.» On Mon, Dec 15, 2025 at 3:11 PM Scott L. Burson <Scott@sympoiesis.com> wrote:
Logical pathnames were invented for the Lisp Machine. I believe the purpose was to introduce a level of indirection between pathnames as written in code (e.g. in 'defsystem' forms) and the actual physical locations, which could be on any machine on the LAN, running any of several operating systems. Then the choice of physical location would be a matter of site configuration, rather than requiring touching source code.
I recall using logical pathnames on the LispM, and finding them clunky but not unusable. I believe much of the clunkiness originated from the fact that for the first several years of LispM development, the sources were kept on the MIT-AI PDP-10 running ITS, the aptly named Incompatible Timesharing System developed at MIT for its own use. ITS had a very restrictive filename syntax: IIRC, digits and uppercase letters only, with a 6-character limit. This, as I recall, allowed a filename to fit in a single 36-bit machine word.
I haven't tried to use logical pathnames much in CL, so I don't have firsthand experience like Faré's, but I don't doubt his report. Perhaps they were usable as long as there was only one implementation, but for CL, there seems to have been a combination of them being underspecified and implementors not being familiar with how they worked on the LispM.
-- Scott
On Mon, Dec 15, 2025, 8:32 AM David Cooper <david.cooper@genworks.com> wrote:
Hi Faré, and thanks for remaining active on these mailing lists, I'm getting that you're spread thin these days.
Could you (or anyone who feels competent to explain it) tell us again what is a Logical Pathname (LP) and how does it differ from a Pathname object in general (as printed and readable by #P"some-string-representing-a-pathname") ? i have CLtL2 as well as Franz' "Common Lisp: The Reference" here on my shelf but, true confessions time, I have rarely cracked these books and I find them somewhat intimidating just to open and read for no specific reason. If anyone's explanations of pathnames and logical pathnames can refer to specific sections or passages from those books, I might dare crack them to look at those sections.
One question I have is: Are Pathnames evil in general (and we should be working with bare strings as "pathname designators")? Or is it Logical Pathnames specifically that are evil. Is a logical pathname a thing with the VMS-looking path syntax, e.g. `sys:some;logical;path;` as opposed to a Pathname object which is something like a struct with 6 slots (directory, name, type, host, device, version)? Maybe the concepts are orthogonal, i.e. a Pathname object's string #P"<some-string>" can be of "native" format or in "logical pathname" format? So we could potentially get things like #P"/opt/gendl-ccl/gdl/program/gdl-ccl" or #P"sys:gdl-ccl". And in the first case, HOST, DEVICE, VERSION would be unspecified, while in the second case the HOST is "sys:" and DEVICE and VERSION are still unspecified, or can a given HOST come along with its own default notion of DEVICE and VERSION just as it comes along with its notion of DIRECTORY, or do those other defaults come strictly from *default-pathname-defaults* ? And is the Logical Pathname format meant to be able to accomodate all six of the slots in the Pathname object -- is that the connection between Pathnames and Logical Pathnames?
None of these are burning questions, or blocking my productivity as far as I know (I tend to get things working by hook or by crook and leave them alone, and knock wood, pathnames per se have not been one of my big bottlenecks lately). But I am curious to understand the systemic issues better i.e. if there are an identifiable finite list of shortcomings in the current ANSI spec that we could at least quarantine or create some kind of polyfill for it, although if I had understood Faré completely, maybe I would have understood that uiop is the closest thing we currently have to a "polyfill."
Dave Cooper
---- On Mon, 15 Dec 2025 03:30:49 -0500 Faré <fahree@gmail.com> wrote ---
Of course, the detailed behavior of LP translation is so poorly constrained by the ANS that using LPs just squeezes the portability toothpaste from one end of the tube to the other, out through the nozzle, and onto your keyboard. You'd think it's toothpaste, but actually it's radioactive sludge. Don't touch it. Do go anywhere near it. Keep it away from kids, keep it away from adults without a hazmat suit.
Making UIOP and ASDF work with "Logical" pathnames was hell. And it won't actually "work" that well: between limitations that SBCL enforces but not others implementations, quirks and bugs of many implementations, atrocious performance (guaranteed O(N)), and impossibility for non-gurus to configure, much less debug... it's 100% worse than using a variable and uiop:subpathname and/or pathname-translations, 100% of the time. Only valid use I've ever seen was to have location-independent debug information on CCL, but that was CCL-only magic.
Oh yeah, did you know that it's impossible to not have a HOST? And on some implementation, the HOST will determine the syntax and various limitations, such that you cannot, conceivably, have a portable representation of pathname in CL, even less so in presence of Logical Pathnames? Not that any of the standard functions (especially NOT merge-pathnames, as you might like to believe) is designed to deal with relative pathnames properly. It took me months to get UIOP to kind of work correctly on portably most of the time for most of the cases on most implementations... but absolutely not, never, not possible using Logical Pathnames—UIOP will do its best, but the best isn't very good.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org “There are just two rules of governance in a free society: Mind your own business. Keep your hands to yourself.” — P. J. O'Rourke
participants (7)
-
David Cooper -
Faré -
Marco Antoniotti -
Robert Goldman -
Robert P. Goldman
-
Scott L. Burson -
Steve Haflich