*LOAD-TRUENAME* and ASDF
I just realized that ASDF somewhat breaks *LOAD-TRUENAME*. I had some code in a DSL that has an :INCLUDE construct, and that DSL is being interpreted at load-time. The :INCLUDE directive tries to find other lisp files relative to the current file (the source file that contained the DSL :INCLUDE expression). Now, if we were not using ASDF, I would be able to find those files by merging a name with *load-truename* (and this is how things used to work). But ASDF's relocation of the fasls breaks this. Now, of course, I could change (:INCLUDE construct "construct-source.lisp") to (:INCLUDE construct #.(asdf:system-relative-pathname "foo" "construct-source.lisp")) but: 1. This is blatantly ugly, effortful, and error-prone for the programmer. 2. It is poor software engineering, because it requires the contained thing (the DSL expression) to "know" that it is being included in a very specific ASDF system. Now if we rename the ASDF system, or shuffle files, our DSL code is broken, and that's just wrong, because it makes the abstraction upside down. I somehow assumed it would be possible to go from the FASL to the source, but I don't actually see any obvious way to do this. suggestions? thanks! r
On Wed, Aug 24, 2016 at 3:09 PM, Robert Goldman <rpgoldman@sift.net> wrote:
Can UIOP:CURRENT-LISP-FILE-PATHNAME help you? —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The demands that good people make are upon themselves; Those that bad people make are upon others. — Confucius (551–479 BC)
On 8/24/16 Aug 24 -4:50 PM, Faré wrote:
[...snip...]
Can UIOP:CURRENT-LISP-FILE-PATHNAME help you?
No, I'm afraid not. It returns the same thing that *LOAD-TRUENAME* does: the pathname for the fasl file, located in the cache, instead of the source file. CURRENT-LISP-FILE-PATHNAME evaluates to (or *compile-file-pathname* *load-pathname*) which means that I still get the pathname of the relocated fasl file, instead of the pathname of the lisp source file. I don't obviously see a way to invert the file translation function, either. I'm surprised I never noticed this in all these years. It's not obvious how to fix this, since the idea of having a dynamic variable bound (which was my first thought) isn't clearly feasible, since the COMPILE-FILE and LOAD aren't in a shared function call scope, because of the plan structure. I suppose one could add something to the PERFORM method for LOAD-OP, but when you're performing a LOAD-OP, you aren't even guaranteed that a corresponding COMPILE-OP exists (i.e., the process of COMPILE-THEN-LOAD could have been overridden). Best, r
On 8/25/16 Aug 25 -1:08 PM, Robert Goldman wrote:
Ok, I *believe* that we could change this: (defun perform-lisp-load-fasl (o c) (if-let (fasl (first (input-files o c))) (load* fasl))) into something like (defun perform-lisp-load-fasl (o c) (if-let (fasl (first (input-files o c))) (let ((*lisp-source-truename* (component-pathnme c)) (load* fasl))) and then export *lisp-source-truename*. Not ideal, but I'm having trouble thinking of a good alternative. The only other thing I can think of would be to supply a function that would do: (uiop:inverted-file-redirection *load-truename*)
Correct me if I'm wrong, but the following macro should work: (defmacro lisp-file-truename () (or *compile-file-truename* *load-truename*)) Since minimal compilation requires that macros be expanded at compile-time, compiling a file with an invocation of the above macro should expand it. -Jason On 15:02 Thu 25 Aug , Robert Goldman wrote:
Yeah, you may have to call (current-lisp-pathname) in a macro and/or in an eval-when. What I generally recommend, though, is to save the pathname of the system (via system-relative-pathname) and/or the file that you process so that you can keep things relative to it. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org On Thu, Aug 25, 2016 at 7:55 PM, Jason Miller <jason@milr.com> wrote:
There is nothing innocent about assuming that compiled file are close to the source files, or that the source files are available at runtime, what more at the same location as at compile-time. Unless may if you're using Nix. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org He who says he will die for a cause will probably lie for it and may kill for it. — John McCarthy
Faré wrote:
I've avoided getting into this discussion, but I feel I need to ask: why use *load-truename* instead of *load-pathname*? *load-truename* goes through symbolic links (even though the ANS says nothing about it, this is the behavior of implementations I know of) and that is almost always the wrong thing. GNU make doesn't do it, I don't see why ASDF should do it. A build system should never itself follow symlinks, because it defeats systems that have been in place for 30+ years: linked directories of binary files linking to a single source directory. Kevin
On 09:48 Mon 29 Aug , Kevin Layer wrote:
This is because *load-pathname* is likely to be a relative pathname, and relative pathnames have their own problems. The unix way of solving it is to transform relative pathnames with "$PWD/pathname" but I don't believe there is a builtin facility for doing the equivalent in common lisp, nor did I find one in a quick look throug uiop/pathname.
On 8/30/16 Aug 30 -3:46 PM, Jason Miller wrote:
There is a method for doing this, but it's done through *default-pathname-defaults* The problem is that the Unix way relies on the current working directory, but *default-pathname-defaults* is populated in a domain-dependent way. Rant: As with non-CL software, it's amazing how many people assume that you will be running their software in the way they do, typically from the directory where the code is located. I cannot tell you how much aggregate time I have spent over the years hardening other people's software against violations of this assumption. Please folks, remember that even *I* may not know the working directory of my SLIME instance, so *you* cannot count on it!
On Tue, Aug 30, 2016 at 4:46 PM, Jason Miller <jason@milr.com> wrote:
Then you missed uiop:ensure-absolute-pathname, and/or the :ensure-absolute option to ensure-pathname (and, by extension, parse-unix-namestring). It's there because indeed ASDF 3 uses it, precisely to prevent situations like the one you mention. It's in uiop/pathname, but if you might in many cases suppose #'uiop/os:getcwd as its :default. Remarkably, uiop is now 40% larger than asdf/defsystem, and over eight times larger than the original ASDF 1.85... that's the price of being portable and robust. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org I'll start exercising as soon as I'm into shape.
On 8/31/16 Aug 31 -11:26 AM, Kevin Layer wrote:
I suppose this is safe, but I always have an unpleasant feeling about using a function call like this that depends on the value of a special variable (*default-pathname-defaults*) that I don't control. At any rate, we pretty much established that in an ASDF-loaded system, *load-pathname* is likely to be garbagy, because at load time it typically holds the pathname of the *RELOCATED* FASL file, which is of no use to anyone. best, r
On 8/31/16 Aug 31 -11:43 AM, Robert Goldman wrote:
This issue just popped up on the LispWorks users group mailing list, so I took a few minutes to write a FAQ about it, and pushed it into the manual. I have updated the manual version on the ASDF web page at common-lisp.net to hold the new version. Comments welcome.
On 8/31/16 Aug 31 -4:13 PM, Kevin Layer wrote:
No, you are right, they are *both* broken in ASDF-loaded systems. Sorry -- what I was trying to say is that I don't think it's worth arguing about the distinction between the two (in connection with ASDF-loaded systems), because each is useless. Hence my new FAQ entry... best, r
Also sprach Robert Goldman on 2016-08-24:
I've identified two problems from this.
I somehow assumed it would be possible to go from the FASL to the source, but I don't actually see any obvious way to do this.
I've been using the :STATIC-FILE component type to include the to-be-loaded file with the system. Then I just use LOAD like normal. I've been using ASDF with image dumping, so finding the source file probably wouldn't work. This isn't quite ideal because ASDF doesn't perform compilation of :STATIC-FILE. I noticed there's an experimental option :BUILD-OPERATION, but it is wholly undescribed apart from saying the default is to load the compiled file into the system. Since it's already there, we should standardize some more of it, such as to compile but not load the component. (This won't work in all situations, but for files that can be compiled it should help.)
There is no dynamic binding for the current system (or component), like *PACKAGE* and *LOAD-TRUENAME*. If ASDF calls COMPILE-FILE anyways, couldn't we bind ASDF:*SYSTEM* (the currently compiled system) around it? When the containing form is compiled using ASDF it points to the current system so you can use ASDF:FIND-COMPONENT with it. When it isn't, you fall back on *LOAD-TRUENAME* and attach the correct file type to the component name.
participants (6)
-
Faré -
Faré -
Jason Miller -
Kevin Layer -
Lucien Pullen -
Robert Goldman