Dear users and developers,
I'm having issues using the bundling ops, deliver-asd-op to be specific.
Apparently, since commit c213e319fd9f6dc53570f0a05034ee31a24ae1ae, the generated system definitions get mangled when deliver-asd-op is used on slashy subsystems. Instead of foo/bar, the system name becomes foo--bar. Previously, the pathname-name was set to the full complex component name (sneakily including the forward slash character).
Consider this ABCL session (ASDF 3.3.7):
(load "https://common-lisp.net/project/asdf/archives/asdf.lisp")
(ensure-directories-exist "~/common-lisp/testsys1/")
(uiop:with-output-file (o "~/common-lisp/testsys1/testsys1.asd") (print '(defsystem "testsys1" :components ((:file "test1"))) o) (print '(defsystem "testsys1/sub" :components ((:file "test2"))) o))
(uiop:with-output-file (o "~/common-lisp/testsys1/test1.lisp") (print '(defpackage test1 (:use :cl)) o) (print '(in-package test1) o) (print '(defun test1 () 42) o))
(uiop:with-output-file (o "~/common-lisp/testsys1/test2.lisp") (print '(defpackage test2 (:use :cl)) o) (print '(in-package test2) o) (print '(defun test2 () 43) o))
(asdf:clear-configuration) (asdf:operate 'asdf:deliver-asd-op "testsys1") (asdf:operate 'asdf:deliver-asd-op "testsys1/sub")
(uiop:println (list (asdf:output-files 'asdf:compile-bundle-op "testsys1") (asdf:output-files 'asdf:deliver-asd-op "testsys1") (asdf:output-files 'asdf:compile-bundle-op "testsys1/sub") (asdf:output-files 'asdf:deliver-asd-op "testsys1/sub")))
The result:
(("~/.cache/common-lisp/.../testsys1/testsys1--system.abcl") ("~/.cache/common-lisp/.../testsys1/testsys1.asd") ("~/.cache/common-lisp/.../testsys1/testsys1--sub--system.abcl") ("~/.cache/common-lisp/.../testsys1/testsys1--sub.asd"))
Concatenating these .asd files and replacing the invalid system names becomes an epic hack. At which point it might be easier to implement a replacement asdf:operation. But it might be better to do it in ASDF itself.
To keep these systems loadable, their definitions should land in the parent .asd file. But if ASDF checks whether an operation is already completed based on probe-file, it becomes a problem when multiple slashy systems are involved. It could cause early cutoff. The deliver-asd-op would have to check for the existence of a corresponding defsystem form in the file, rather than the mere existence of that file.
The incoming defsystems would have to be appended. I don't know whether that should be automatic or require an explicit invoke-restart. There might be some validation required to verify that the primary system is also being delivered. (Correct?)
Fasls can probably stay mangled, since they can be targeted independently, as opposed to .asd files... well, that's unless someone decides to create a (defsystem "testsys1--sub") - though it does seem unconventional. Maybe that would be an acceptable limitation.
Here are the results from SBCL, which still uses uses ASDF 3.3.1:
(("~/.cache/common-lisp/.../testsys1/testsys1--system.fasl") ("~/.cache/common-lisp/.../testsys1/testsys1.asd") ("~/.cache/common-lisp/.../testsys1/testsys1/sub--system.fasl") ("~/.cache/common-lisp/.../testsys1/testsys1/sub.asd"))
(By the way, I'm not sure how did the testsys1/testsys1 subdirectory manage to get created without an error, since it seems that testsys/sub is the pathname-name, and I believe ensure-directory-exists works with pathname-directory.)
In this case, I'm lucky that the testsys1 project doesn't contain an inner testsys1 directory containing another ASD file "sub.asd" - that would cause another conflict, but I digress.
To summarize, regarding deliver-asd-op:
- The slashy system names and the .asd pathname-names are being mangled - The system definitions need to stay in one file - DIY merging un-mangling them afterwards is too hairy - Let's discuss encapsulating that hairiness in ASDF itself
The goal: to support binary deployment of slashy systems (non-slashy systems already work), so that the generated .asd and .fasl can be installed, without source, on client systems, all the while being ASDF-discoverable.
An alternative workaround might be to create an ephemeral uber system just for delivery, one that depends-on the parent system and every desired slashy system. But that prevents fine grained updates and also creates one more system to remember.