- asdf-devel - mailman3.common-lisp.net

Edge case of TIMESTAMP<=
by Robert Goldman 01 Feb '25

01 Feb '25

The following message was received to the mailing list and, for some reason, discarded instead of held for moderation (if you want to submit emails to asdf-devel, please subscribe; you can always unsubscribe later.) This certainly seems plausibly to be a bug. Please post it at [GitLab](https://gitlab.common-lisp.net/asdf/asdf/-/issues) and I will have a look at it as time permits. A test case would be a real plus -- I'm much more likely to get on this if I don't also have to create the test case myself. ---- Hello, I may have discovered a bug in ASDF. In the function COMPUTE-ACTION-STAMP, first the IN-STAMPS and then the OUT-STAMPS are collected. When one of the last write dates are not available, T is returned by GET-FILE-STAMP, which is deemed infinitely old by TIMESTAMP<=. But suppose that all the IN-STAMPS and at least one of the OUT-STAMPS are not available, then LATEST-IN and EARLIEST-OUT are both T, and the wisest thing would probably be rebuild everything, but TIMESTAMP<= returns T so nothing is built. What do you think? Andrea Monaco

2 2

package-inferred-system noob-user confusion
by Russell L. Carter 20 Jan '25

20 Jan '25

Greetings, I am a common-lisp noob. I am not a programmer noob, nor a build system noob. I have carefully studied fare's asdf manual, pages 25-26 in the pdf and backwards through the reverse dependencies of the terms used, about the package-inferred-system. I have a problem that I've whittled down to a very small snippet. I'm using SBCL 2.4.10.117-507e7ae05 and its native ASDF, but also tested against ASDF repo head (ln -s into ~/common-lisp, verified in the repl). My question is why this system, located in file "./hello.asd", containing: ***************************************************************** (asdf:defsystem "hello" :class :package-inferred-system :depends-on ("hello/src/main")) ***************************************************************** paired with the system/package file "./src/main.lisp", containing: ****************************************************************** (uiop:define-package :hello/src/main ;; (:use :alexandria) ;; XXXRLC This line fails: "The variable MAIN ;; is unbound". I have no idea why. Backtrace provides no clues ;; (to me). Load the library in the repl matters not. Elide that ;; line and (asdf:load-system "hello") => T and then CL-USER> ;; (hello/src/main:main) => urg \n "urg" as expected. (:export #:main)) (in-package :hello/src/main) (defun main () (write-line "urg")) ******************************************************************* fails as described in the comment. Sure I would like the answer but the more interesting question is how could I debug this failure? I slogged through the late '90s debugging complex C++ template programming errors. Pages and pages of output. I don't mind doing it again. But I don't know where to start with this ASDF build system. Thank you for any suggestions, obviously the answer must be trivial. All the best, Russell L. Carter

5 17

Balancing ASDF Configuration for Precompiled OS and User-Compiled FASL Files
by Joseph Mingrone 02 Jan '25

02 Jan '25

Hello, I'm attempting to adjust the ASDF configuration for FreeBSD CL ports/packages to balance the use of precompiled FASL files from OS packages with the ability to compile and write code under users' home directories. Details are below. Any advice would be appreciated. The system-wide configuration [0] I'm attempting to adjust is used both when building FreeBSD CL OS packages and on target systems. Specifically, it: - sets up a central registry, - enforces a filesystem layout, - and configures output translations. With this configuration, users can run `pkg install cl-alexandria-sbcl` and two packages will be installed: - cl-alexandria, essentially the upstream repository, - cl-alexandria-sbcl, the compiled code. From their Lisp implementation, users only need to execute (asdf:load-system :alexandria) and no local compilation is required. However, if a user installs only cl-alexandria without cl-alexandria-sbcl, issues arise. Due to the output translations, ASDF attempts to write compiled code to a location where the user lacks write access. On the other hand, if we remove the output translations on target systems, all FASL files provided by the OS packages are ignored, and ASDF tries to recompile and write them under ~/.cache/common-lisp/. Is there a way to configure ASDF to recognize and accept the FASL files from the OS packages and compile and write any missing FASL files to the default untranslated location under ~/.cache/common-lisp/? Thank you, Joe [0] https://cgit.freebsd.org/ports/tree/devel/cl-asdf/bsd.cl-asdf.mk

2 7

ASDF support for Haiku (CLISP)
by Alexandru Popa 21 Nov '24

21 Nov '24

Haiku is an operation system different from Windows, Linux or *BSD. Still, in many respects it can be considered Unix-like, which is especially true when porting software to it. CLISP was recently ported to Haiku. In the *features*, CLISP on Haiku defines both :UNIX (i.e. Unix-like) and :HAIKU, and this is probably the right decision. However, after (require "asdf"), *features* are changed, :HAIKU is removed and :OS-UNIX is added. This is certainly something which needs to be fixed. The desired solution would be to keep both :UNIX, :HAIKU and to add :OS-HAIKU. Here is the analysis from CLISP development list: ``` (defun detect-os () "Detects the current operating system. Only needs be run at compile-time, except on ABCL where it might change between FASL compilation and runtime." (loop* :with o :for (feature . detect) :in '((:os-unix . os-unix-p) (:os-macosx . os-macosx-p) … (:haiku . os-haiku-p)) :when (and (or (not o) (eq feature :os-macosx)) (funcall detect)) :do (setf o feature) (pushnew feature *features*) :else :do (setf *features* (remove feature *features*)) :finally (return (or o (error "Congratulations for trying ASDF on an operating system~%~ that is neither Unix, nor Windows, nor Genera, nor even old MacOS.~%Now you port it."))))) That is somewhat brittle code that side-effects *FEATURES*. It contains a special bypass to allow :OS-MACOSX to live there beside :OS-UNIX, but there’s nothing equivalent for Haiku. Whether Haiku is considered a UNIX or not I won’t debate. I call such code /brittle/ because there’s an undocumented (non-explicitly mentioned) dependency on element order in some innocuous list: If :os-macosx were before :os-unix in the A-list, that code would not set – and rather delete – :OS-UNIX in *features*. I would have appreciated a tiny comment like :for (feature . detect) :in '(… (:os-unix . os-unix-p) (:os-macosx . os-macosx-p) ; Beware, unix must come first! ''' Alexandru Popa

2 5

Expected deliver-asd-op behavior involving slashy subsystems
by Kasper Gałkowski 21 Oct '24

21 Oct '24

Dear users and developers, I'm having issues using the bundling ops, deliver-asd-op to be specific. Apparently, since commit c213e319fd9f6dc53570f0a05034ee31a24ae1ae, the generated system definitions get mangled when deliver-asd-op is used on slashy subsystems. Instead of foo/bar, the system name becomes foo--bar. Previously, the pathname-name was set to the full complex component name (sneakily including the forward slash character). Consider this ABCL session (ASDF 3.3.7): (load "https://common-lisp.net/project/asdf/archives/asdf.lisp") (ensure-directories-exist "~/common-lisp/testsys1/") (uiop:with-output-file (o "~/common-lisp/testsys1/testsys1.asd") (print '(defsystem "testsys1" :components ((:file "test1"))) o) (print '(defsystem "testsys1/sub" :components ((:file "test2"))) o)) (uiop:with-output-file (o "~/common-lisp/testsys1/test1.lisp") (print '(defpackage test1 (:use :cl)) o) (print '(in-package test1) o) (print '(defun test1 () 42) o)) (uiop:with-output-file (o "~/common-lisp/testsys1/test2.lisp") (print '(defpackage test2 (:use :cl)) o) (print '(in-package test2) o) (print '(defun test2 () 43) o)) (asdf:clear-configuration) (asdf:operate 'asdf:deliver-asd-op "testsys1") (asdf:operate 'asdf:deliver-asd-op "testsys1/sub") (uiop:println (list (asdf:output-files 'asdf:compile-bundle-op "testsys1") (asdf:output-files 'asdf:deliver-asd-op "testsys1") (asdf:output-files 'asdf:compile-bundle-op "testsys1/sub") (asdf:output-files 'asdf:deliver-asd-op "testsys1/sub"))) The result: (("~/.cache/common-lisp/.../testsys1/testsys1--system.abcl") ("~/.cache/common-lisp/.../testsys1/testsys1.asd") ("~/.cache/common-lisp/.../testsys1/testsys1--sub--system.abcl") ("~/.cache/common-lisp/.../testsys1/testsys1--sub.asd")) Concatenating these .asd files and replacing the invalid system names becomes an epic hack. At which point it might be easier to implement a replacement asdf:operation. But it might be better to do it in ASDF itself. To keep these systems loadable, their definitions should land in the parent .asd file. But if ASDF checks whether an operation is already completed based on probe-file, it becomes a problem when multiple slashy systems are involved. It could cause early cutoff. The deliver-asd-op would have to check for the existence of a corresponding defsystem form in the file, rather than the mere existence of that file. The incoming defsystems would have to be appended. I don't know whether that should be automatic or require an explicit invoke-restart. There might be some validation required to verify that the primary system is also being delivered. (Correct?) Fasls can probably stay mangled, since they can be targeted independently, as opposed to .asd files... well, that's unless someone decides to create a (defsystem "testsys1--sub") - though it does seem unconventional. Maybe that would be an acceptable limitation. Here are the results from SBCL, which still uses uses ASDF 3.3.1: (("~/.cache/common-lisp/.../testsys1/testsys1--system.fasl") ("~/.cache/common-lisp/.../testsys1/testsys1.asd") ("~/.cache/common-lisp/.../testsys1/testsys1/sub--system.fasl") ("~/.cache/common-lisp/.../testsys1/testsys1/sub.asd")) (By the way, I'm not sure how did the testsys1/testsys1 subdirectory manage to get created without an error, since it seems that testsys/sub is the pathname-name, and I believe ensure-directory-exists works with pathname-directory.) In this case, I'm lucky that the testsys1 project doesn't contain an inner testsys1 directory containing another ASD file "sub.asd" - that would cause another conflict, but I digress. To summarize, regarding deliver-asd-op: - The slashy system names and the .asd pathname-names are being mangled - The system definitions need to stay in one file - DIY merging un-mangling them afterwards is too hairy - Let's discuss encapsulating that hairiness in ASDF itself The goal: to support binary deployment of slashy systems (non-slashy systems already work), so that the generated .asd and .fasl can be installed, without source, on client systems, all the while being ASDF-discoverable. An alternative workaround might be to create an ephemeral uber system just for delivery, one that depends-on the parent system and every desired slashy system. But that prevents fine grained updates and also creates one more system to remember.

3 3

Query about 3.3.7
by Greg Bennett 07 Oct '24

07 Oct '24

Hi from Greg Bennett who is running 3.3.7 as part of sbcl-2.4.8-linux-x64 under linux mint. I must begin by thanking Robert G for all his efforts on maintaining asdf !! In ~/.config/common-lisp/source-registry.conf.d there is the file gb-source.conf The live element of that file is (:tree "/home/gwbennett/RESERVE/mintq/") When compiling a collection of systems I see results like this: ; compiling file "/home/gwbennett/RESERVE/mintq/source/window-basics/menus/menu-sblx.lsp" (written 13 MAY 2024 05:45:54 PM): ; wrote /home/gwbennett/.cache/common-lisp/sbcl-2.4.8-linux-x64-s/home/gwbennett/RESERVE/mintq/source/window-basics/menus/menu-sblx-tmpQXR7SNXD.fasl ; compilation finished in 0:00:00.024 I wonder why the fasls are written to ...../sbcl-2.4.8-linux-x64-s/home.... rather than the path without the -s ? ^^ My ~/.config/common-lisp/asdf-output-translations.conf.d is empty. The fasls are all there in the -s directory tree and, I assume, are the files which are loaded through the load phase of (asdf:load-system "window-basics) Hints, pointers, advice all appreciated Cheers Greg

2 1

Re: Working around nonconforming names in system definitions
by Robert Dodier 27 Sep '24

27 Sep '24

On Sat, Sep 14, 2024 at 11:01 AM Robert P. Goldman <rpg(a)sift.net> wrote: > I’ve got to push back on your rant. There’s a perfectly good reason for those two limitations you dislike: they are the same as the limitations on logical pathnames. So if you wish to be able to find an ASDF system relative to a directory designated by a logical pathname, those limitations are pretty much forced on you. All right. Help me sort this out. I actually want to make this work. I have a directory Foo_Bar and I want to create a .asd file (in any directory which is known to ASDF) which describes the stuff within Foo_Bar. From what I have sorted out so far, that .asd file must be named something other than "Foo_Bar.asd" and the system it describes must be named something other than "Foo_Bar". The ideal outcome is that a user would say (asdf:load-system "Foo_Bar") and that the system which describes the stuff in "Foo_Bar" would get loaded. How close can I get to that, and what do I have to do, involving logical pathnames if need be, in order to get it? I am thinking I need to do something like (I emphasize "like", as I am aware that the following code doesn't work): (setf (logical-pathname-translations "abc") ("**;foobar.asd" "Foo_Bar.asd")) and that must be executed before the system is defined (defsystem "foobar" ...) and before the user calls ASDF:LOAD-SYSTEM. (asdf:load-system "Foo_Bar") Can this work, at all? As I was saying, I actually want this (or anything) to work. I don't have any point to make except to get something working. Thanks for your help. Robert

4 13

Re: Working around nonconforming names in system definitions
by Robert Dodier 16 Sep '24

16 Sep '24

On Sun, Sep 15, 2024 at 7:29 PM phoebe Goldman <phoebe(a)goldman-tribe.org> wrote: > if you're newly adding system definitions, it seems like you must be at liberty to choose whatever names you want for those systems, right? Yes, and I might end up there, but I'd rather keep the name of the system the same as the name of the project, because it's going to cause unnecessary confusion to try to ensure that users refer to the "right" name. Maybe I can encapsulate some heuristics so that nonconforming names get mapped into the corresponding conforming name or names, so that users can refer to the original, nonconforming name ... That's not foolproof, so, again, I'd rather not if I don't have to. Robert

2 1

Re: Working around nonconforming names in system definitions
by Robert P. Goldman 16 Sep '24

16 Sep '24

Hi Robert, I’m away from my desk, so can’t check, but If I recall correctly there are name canonicalization functions that downcase. I don’t know if these are only applied when a system designator is a symbol or to strings as well. The underscores would be an issue if the system name was merged with a logical pathname, if a logical pathname was part of the ASDF configuration. In that case, some lisp implementations would reject the resulting pathname as illegal. Some might be more permissive. I have had experiences with path names that work on allegro and not on SBCL. CL-JSON replaces underscores with 2 hyphens, perhaps you could do something similar and double letters to indicate caps. You could provide some translation of system names to file names like that if you wanted to permit non standard system names. Of course, you would have to ensure that anyone using one of these systems would also have loaded the extension that found it properly. Honestly, this seems a lot more trouble than it’s worth: the name restrictions aren’t that onerous. On Sep 14, 2024 at 23:50:11 CDT, Robert Dodier <robert.dodier(a)gmail.com> wrote: On Sat, Sep 14, 2024 at 8:45 PM Faré <fahree(a)gmail.com> wrote: > If you're using logical pathnames, you're asking for trouble > (especially on SBCL). I wasted weeks of my life on logical pathnames, > and STRONGLY recommend that no one should ever use them anymore under > any circumstances. Just to be clear, I don't *want* to use logical pathnames, but I will if it provides a solution to the problem. > If you're not, there should be no problem having a system called > Foo_Bar, is there? > > Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no? I dunno, that doesn't seem to work ... I tried this in SBCL which has ASDF 3.3.something bundled. (asdf:load-system "Foo_Bar") provokes a warning: "WARNING: System definition file #P"/home/dodier/tmp/Foo_Bar.asd" contains definition for system "foo_bar". Please only define "Foo_Bar" and secondary systems with a name starting with "Foo_Bar/" (e.g. "Foo_Bar/test") in that file. and an error: Component "Foo_Bar" not found Anyway, the documentation says (https://asdf.common-lisp.dev/asdf.html#rule_002dsystem_002ddesignator) that "Foo_Bar" is an unacceptable name since it has both an underscore and uppercase letters. I find that (defsystem baz_quux ...) in a file named "baz_quux.asd" is loaded successfully without complaint -- I'm surprised that works; I guess the no underscores rule is not actually enforced? I guess the stated rules about simple component names are not actually enforced by the DEFSYSTEM macro itself ... the do-nothing system * (asdf:defsystem "Blurf_Mumble") * (asdf:operate 'asdf:load-op *) succeeds. I guess the stuff about string case only comes into play when the file system gets involved? FWIW & all the best. Robert

2 1

Working around nonconforming names in system definitions
by Robert Dodier 14 Sep '24

14 Sep '24

I am interested in automatically generating ASDF system definitions for projects which lack them. These projects have names, both the project name and the names of files or directories within them, which might not conform to ASDF's requirements for names. In particular, project names containing uppercase letters and underscores.These are other peoples' projects, so I'm not at liberty to rename them to conform to ASDF. RANT. The limitations that ASDF puts on names are just bananas. I have to wonder who thought it would be a good idea to force people to work around them. I know this has been discussed before, without any resolution, so I won't pursue it any further. There's no need to explain about logical pathnames, I know already. Thanks for understanding. /RANT For a project which has a conforming name, say foobar, I can create foobar.asd and say that the name of the system is "foobar". That's great. Let's say the name of the project is Foo_Bar. What are some options for naming the .asd system and the system which it contains, ideally so that when a user tries (asdf:load-system "Foo_Bar") they get the appropriate system? Perhaps that's impossible; if so I'd just like to know. I've tried some different things and haven't come up with anything that seems to work. A web search finds a few aging and/or obsolete references, and in any event, nothing specifically directed towards solving this problem. Thanks for any insights. Robert Dodier

2 1