asdf-devel

asdf-devel@common-lisp.net

1 participants
1378 discussions

Balancing ASDF Configuration for Precompiled OS and User-Compiled FASL Files
by Joseph Mingrone 01 Dec '24

01 Dec '24

Hello, I'm attempting to adjust the ASDF configuration for FreeBSD CL ports/packages to balance the use of precompiled FASL files from OS packages with the ability to compile and write code under users' home directories. Details are below. Any advice would be appreciated. The system-wide configuration [0] I'm attempting to adjust is used both when building FreeBSD CL OS packages and on target systems. Specifically, it: - sets up a central registry, - enforces a filesystem layout, - and configures output translations. With this configuration, users can run `pkg install cl-alexandria-sbcl` and two packages will be installed: - cl-alexandria, essentially the upstream repository, - cl-alexandria-sbcl, the compiled code. From their Lisp implementation, users only need to execute (asdf:load-system :alexandria) and no local compilation is required. However, if a user installs only cl-alexandria without cl-alexandria-sbcl, issues arise. Due to the output translations, ASDF attempts to write compiled code to a location where the user lacks write access. On the other hand, if we remove the output translations on target systems, all FASL files provided by the OS packages are ignored, and ASDF tries to recompile and write them under ~/.cache/common-lisp/. Is there a way to configure ASDF to recognize and accept the FASL files from the OS packages and compile and write any missing FASL files to the default untranslated location under ~/.cache/common-lisp/? Thank you, Joe [0] https://cgit.freebsd.org/ports/tree/devel/cl-asdf/bsd.cl-asdf.mk

2 5

ASDF support for Haiku (CLISP)
by Alexandru Popa 21 Nov '24

21 Nov '24

Haiku is an operation system different from Windows, Linux or *BSD. Still, in many respects it can be considered Unix-like, which is especially true when porting software to it. CLISP was recently ported to Haiku. In the *features*, CLISP on Haiku defines both :UNIX (i.e. Unix-like) and :HAIKU, and this is probably the right decision. However, after (require "asdf"), *features* are changed, :HAIKU is removed and :OS-UNIX is added. This is certainly something which needs to be fixed. The desired solution would be to keep both :UNIX, :HAIKU and to add :OS-HAIKU. Here is the analysis from CLISP development list: ``` (defun detect-os () "Detects the current operating system. Only needs be run at compile-time, except on ABCL where it might change between FASL compilation and runtime." (loop* :with o :for (feature . detect) :in '((:os-unix . os-unix-p) (:os-macosx . os-macosx-p) … (:haiku . os-haiku-p)) :when (and (or (not o) (eq feature :os-macosx)) (funcall detect)) :do (setf o feature) (pushnew feature *features*) :else :do (setf *features* (remove feature *features*)) :finally (return (or o (error "Congratulations for trying ASDF on an operating system~%~ that is neither Unix, nor Windows, nor Genera, nor even old MacOS.~%Now you port it."))))) That is somewhat brittle code that side-effects *FEATURES*. It contains a special bypass to allow :OS-MACOSX to live there beside :OS-UNIX, but there’s nothing equivalent for Haiku. Whether Haiku is considered a UNIX or not I won’t debate. I call such code /brittle/ because there’s an undocumented (non-explicitly mentioned) dependency on element order in some innocuous list: If :os-macosx were before :os-unix in the A-list, that code would not set – and rather delete – :OS-UNIX in *features*. I would have appreciated a tiny comment like :for (feature . detect) :in '(… (:os-unix . os-unix-p) (:os-macosx . os-macosx-p) ; Beware, unix must come first! ''' Alexandru Popa

2 5

Expected deliver-asd-op behavior involving slashy subsystems
by Kasper Gałkowski 21 Oct '24

21 Oct '24

Dear users and developers, I'm having issues using the bundling ops, deliver-asd-op to be specific. Apparently, since commit c213e319fd9f6dc53570f0a05034ee31a24ae1ae, the generated system definitions get mangled when deliver-asd-op is used on slashy subsystems. Instead of foo/bar, the system name becomes foo--bar. Previously, the pathname-name was set to the full complex component name (sneakily including the forward slash character). Consider this ABCL session (ASDF 3.3.7): (load "https://common-lisp.net/project/asdf/archives/asdf.lisp") (ensure-directories-exist "~/common-lisp/testsys1/") (uiop:with-output-file (o "~/common-lisp/testsys1/testsys1.asd") (print '(defsystem "testsys1" :components ((:file "test1"))) o) (print '(defsystem "testsys1/sub" :components ((:file "test2"))) o)) (uiop:with-output-file (o "~/common-lisp/testsys1/test1.lisp") (print '(defpackage test1 (:use :cl)) o) (print '(in-package test1) o) (print '(defun test1 () 42) o)) (uiop:with-output-file (o "~/common-lisp/testsys1/test2.lisp") (print '(defpackage test2 (:use :cl)) o) (print '(in-package test2) o) (print '(defun test2 () 43) o)) (asdf:clear-configuration) (asdf:operate 'asdf:deliver-asd-op "testsys1") (asdf:operate 'asdf:deliver-asd-op "testsys1/sub") (uiop:println (list (asdf:output-files 'asdf:compile-bundle-op "testsys1") (asdf:output-files 'asdf:deliver-asd-op "testsys1") (asdf:output-files 'asdf:compile-bundle-op "testsys1/sub") (asdf:output-files 'asdf:deliver-asd-op "testsys1/sub"))) The result: (("~/.cache/common-lisp/.../testsys1/testsys1--system.abcl") ("~/.cache/common-lisp/.../testsys1/testsys1.asd") ("~/.cache/common-lisp/.../testsys1/testsys1--sub--system.abcl") ("~/.cache/common-lisp/.../testsys1/testsys1--sub.asd")) Concatenating these .asd files and replacing the invalid system names becomes an epic hack. At which point it might be easier to implement a replacement asdf:operation. But it might be better to do it in ASDF itself. To keep these systems loadable, their definitions should land in the parent .asd file. But if ASDF checks whether an operation is already completed based on probe-file, it becomes a problem when multiple slashy systems are involved. It could cause early cutoff. The deliver-asd-op would have to check for the existence of a corresponding defsystem form in the file, rather than the mere existence of that file. The incoming defsystems would have to be appended. I don't know whether that should be automatic or require an explicit invoke-restart. There might be some validation required to verify that the primary system is also being delivered. (Correct?) Fasls can probably stay mangled, since they can be targeted independently, as opposed to .asd files... well, that's unless someone decides to create a (defsystem "testsys1--sub") - though it does seem unconventional. Maybe that would be an acceptable limitation. Here are the results from SBCL, which still uses uses ASDF 3.3.1: (("~/.cache/common-lisp/.../testsys1/testsys1--system.fasl") ("~/.cache/common-lisp/.../testsys1/testsys1.asd") ("~/.cache/common-lisp/.../testsys1/testsys1/sub--system.fasl") ("~/.cache/common-lisp/.../testsys1/testsys1/sub.asd")) (By the way, I'm not sure how did the testsys1/testsys1 subdirectory manage to get created without an error, since it seems that testsys/sub is the pathname-name, and I believe ensure-directory-exists works with pathname-directory.) In this case, I'm lucky that the testsys1 project doesn't contain an inner testsys1 directory containing another ASD file "sub.asd" - that would cause another conflict, but I digress. To summarize, regarding deliver-asd-op: - The slashy system names and the .asd pathname-names are being mangled - The system definitions need to stay in one file - DIY merging un-mangling them afterwards is too hairy - Let's discuss encapsulating that hairiness in ASDF itself The goal: to support binary deployment of slashy systems (non-slashy systems already work), so that the generated .asd and .fasl can be installed, without source, on client systems, all the while being ASDF-discoverable. An alternative workaround might be to create an ephemeral uber system just for delivery, one that depends-on the parent system and every desired slashy system. But that prevents fine grained updates and also creates one more system to remember.

3 3

Query about 3.3.7
by Greg Bennett 07 Oct '24

07 Oct '24

Hi from Greg Bennett who is running 3.3.7 as part of sbcl-2.4.8-linux-x64 under linux mint. I must begin by thanking Robert G for all his efforts on maintaining asdf !! In ~/.config/common-lisp/source-registry.conf.d there is the file gb-source.conf The live element of that file is (:tree "/home/gwbennett/RESERVE/mintq/") When compiling a collection of systems I see results like this: ; compiling file "/home/gwbennett/RESERVE/mintq/source/window-basics/menus/menu-sblx.lsp" (written 13 MAY 2024 05:45:54 PM): ; wrote /home/gwbennett/.cache/common-lisp/sbcl-2.4.8-linux-x64-s/home/gwbennett/RESERVE/mintq/source/window-basics/menus/menu-sblx-tmpQXR7SNXD.fasl ; compilation finished in 0:00:00.024 I wonder why the fasls are written to ...../sbcl-2.4.8-linux-x64-s/home.... rather than the path without the -s ? ^^ My ~/.config/common-lisp/asdf-output-translations.conf.d is empty. The fasls are all there in the -s directory tree and, I assume, are the files which are loaded through the load phase of (asdf:load-system "window-basics) Hints, pointers, advice all appreciated Cheers Greg

2 1

Re: Working around nonconforming names in system definitions
by Robert Dodier 27 Sep '24

27 Sep '24

On Sat, Sep 14, 2024 at 11:01 AM Robert P. Goldman <rpg(a)sift.net> wrote: > I’ve got to push back on your rant. There’s a perfectly good reason for those two limitations you dislike: they are the same as the limitations on logical pathnames. So if you wish to be able to find an ASDF system relative to a directory designated by a logical pathname, those limitations are pretty much forced on you. All right. Help me sort this out. I actually want to make this work. I have a directory Foo_Bar and I want to create a .asd file (in any directory which is known to ASDF) which describes the stuff within Foo_Bar. From what I have sorted out so far, that .asd file must be named something other than "Foo_Bar.asd" and the system it describes must be named something other than "Foo_Bar". The ideal outcome is that a user would say (asdf:load-system "Foo_Bar") and that the system which describes the stuff in "Foo_Bar" would get loaded. How close can I get to that, and what do I have to do, involving logical pathnames if need be, in order to get it? I am thinking I need to do something like (I emphasize "like", as I am aware that the following code doesn't work): (setf (logical-pathname-translations "abc") ("**;foobar.asd" "Foo_Bar.asd")) and that must be executed before the system is defined (defsystem "foobar" ...) and before the user calls ASDF:LOAD-SYSTEM. (asdf:load-system "Foo_Bar") Can this work, at all? As I was saying, I actually want this (or anything) to work. I don't have any point to make except to get something working. Thanks for your help. Robert

4 13

Re: Working around nonconforming names in system definitions
by Robert Dodier 16 Sep '24

16 Sep '24

On Sun, Sep 15, 2024 at 7:29 PM phoebe Goldman <phoebe(a)goldman-tribe.org> wrote: > if you're newly adding system definitions, it seems like you must be at liberty to choose whatever names you want for those systems, right? Yes, and I might end up there, but I'd rather keep the name of the system the same as the name of the project, because it's going to cause unnecessary confusion to try to ensure that users refer to the "right" name. Maybe I can encapsulate some heuristics so that nonconforming names get mapped into the corresponding conforming name or names, so that users can refer to the original, nonconforming name ... That's not foolproof, so, again, I'd rather not if I don't have to. Robert

2 1

Re: Working around nonconforming names in system definitions
by Robert P. Goldman 16 Sep '24

16 Sep '24

Hi Robert, I’m away from my desk, so can’t check, but If I recall correctly there are name canonicalization functions that downcase. I don’t know if these are only applied when a system designator is a symbol or to strings as well. The underscores would be an issue if the system name was merged with a logical pathname, if a logical pathname was part of the ASDF configuration. In that case, some lisp implementations would reject the resulting pathname as illegal. Some might be more permissive. I have had experiences with path names that work on allegro and not on SBCL. CL-JSON replaces underscores with 2 hyphens, perhaps you could do something similar and double letters to indicate caps. You could provide some translation of system names to file names like that if you wanted to permit non standard system names. Of course, you would have to ensure that anyone using one of these systems would also have loaded the extension that found it properly. Honestly, this seems a lot more trouble than it’s worth: the name restrictions aren’t that onerous. On Sep 14, 2024 at 23:50:11 CDT, Robert Dodier <robert.dodier(a)gmail.com> wrote: On Sat, Sep 14, 2024 at 8:45 PM Faré <fahree(a)gmail.com> wrote: > If you're using logical pathnames, you're asking for trouble > (especially on SBCL). I wasted weeks of my life on logical pathnames, > and STRONGLY recommend that no one should ever use them anymore under > any circumstances. Just to be clear, I don't *want* to use logical pathnames, but I will if it provides a solution to the problem. > If you're not, there should be no problem having a system called > Foo_Bar, is there? > > Just make sure you use a (defsystem |Foo_Bar| ...) with the same case, no? I dunno, that doesn't seem to work ... I tried this in SBCL which has ASDF 3.3.something bundled. (asdf:load-system "Foo_Bar") provokes a warning: "WARNING: System definition file #P"/home/dodier/tmp/Foo_Bar.asd" contains definition for system "foo_bar". Please only define "Foo_Bar" and secondary systems with a name starting with "Foo_Bar/" (e.g. "Foo_Bar/test") in that file. and an error: Component "Foo_Bar" not found Anyway, the documentation says (https://asdf.common-lisp.dev/asdf.html#rule_002dsystem_002ddesignator) that "Foo_Bar" is an unacceptable name since it has both an underscore and uppercase letters. I find that (defsystem baz_quux ...) in a file named "baz_quux.asd" is loaded successfully without complaint -- I'm surprised that works; I guess the no underscores rule is not actually enforced? I guess the stated rules about simple component names are not actually enforced by the DEFSYSTEM macro itself ... the do-nothing system * (asdf:defsystem "Blurf_Mumble") * (asdf:operate 'asdf:load-op *) succeeds. I guess the stuff about string case only comes into play when the file system gets involved? FWIW & all the best. Robert

2 1

Working around nonconforming names in system definitions
by Robert Dodier 14 Sep '24

14 Sep '24

I am interested in automatically generating ASDF system definitions for projects which lack them. These projects have names, both the project name and the names of files or directories within them, which might not conform to ASDF's requirements for names. In particular, project names containing uppercase letters and underscores.These are other peoples' projects, so I'm not at liberty to rename them to conform to ASDF. RANT. The limitations that ASDF puts on names are just bananas. I have to wonder who thought it would be a good idea to force people to work around them. I know this has been discussed before, without any resolution, so I won't pursue it any further. There's no need to explain about logical pathnames, I know already. Thanks for understanding. /RANT For a project which has a conforming name, say foobar, I can create foobar.asd and say that the name of the system is "foobar". That's great. Let's say the name of the project is Foo_Bar. What are some options for naming the .asd system and the system which it contains, ideally so that when a user tries (asdf:load-system "Foo_Bar") they get the appropriate system? Perhaps that's impossible; if so I'd just like to know. I've tried some different things and haven't come up with anything that seems to work. A web search finds a few aging and/or obsolete references, and in any event, nothing specifically directed towards solving this problem. Thanks for any insights. Robert Dodier

2 1

Return value of UIOP:CALL-WITH-CURRENT-DIRECTORY
by Kevin Zheng 02 Sep '24

02 Sep '24

Hi asdf developers, I recently was writing some code with UIOP:CALL-WITH-CURRENT-DIRECTORY and came across something slightly unexpected. In particular, running CALL-WITH-CURRENT-DIRECTORY or WITH-CURRENT-DIRECTORY forms evaluates to NIL, whereas one may expect that the form evaluates to the return value of THUNK or the macro body, respectively. In my program, I'm trying to do something like: (uiop:with-current-directory (dir) (uiop:run-program (list "some" "program") :output :string)) Which doesn't quite do what I want it to do. I believe this has to do with the definition, which ends in a trailing (chdir dir) form to get out of the directory which unfortunately returns NIL instead of the value from evaluating THUNK. So my question is: would it be acceptable if a future version of UIOP changes call-with-current-directory to return the value of evaluating THUNK? Thanks, Kevin

3 4

Running a binary on a server, ASDF wants to access the common-lisp/ directory and fails
by vindarel 29 Aug '24

29 Aug '24

Hi everyone, I built a binary, I run it on my server but I get an ASDF error: |;; Error while trying to load definition for system asdf from pathname ;; /home/vince/common-lisp/asdf/asdf.asd: ;; couldn't load #P"/home/vince/common-lisp/asdf/asdf.asd": file does not exist. | and my binary exits with an error code. ASDF looks for itself on the ~/common-lisp/ directory, which doesn't exist on my VPS, as expected. Do I have a way to tell ASDF to not do this, so that my binary can run? Actually my script doesn't exit right away, it does something, then exits. So, something in my program or in my dependencies are asking this operation to ASDF, when probably it shouldn't. I'm trying to figure it out. However if there was an easy way to gently ask ASDF not to fail, that would be a bliss. Thanks in advance! More details… --- I tried one thing. First, let's show you my .asd declaration. (in-package :asdf-user) (defsystem "observatoire" :version "0" :depends-on (:ciel :secret-values :progressons ) :components ((:module "src" :components ((:file "packages") (:file "common") (:file "extractor") (:file "stock") (:file "ventes") (:file "observatoire")))) :build-operation "program-op" :build-pathname "observatoire" :entry-point "observatoire::main" ) I built the binary with asdf:make. I tried adding these lines at the end of the .asd: #+asdf (asdf:clear-source-registry) #+asdf (defun asdf:upgrade-asdf () nil) This did not help, now I get: There is no applicable method for the generic function #<STANDARD-GENERIC-FUNCTION QL-DIST:INSTALLEDP (3)> when called with arguments (NIL) I had the same issue before, which I bypassed by using Deploy and a hook, containing those two lines: (deploy:define-hook (:deploy asdf) (directory) (declare (ignorable directory)) #+asdf (asdf:clear-source-registry) #+asdf (defun asdf:upgrade-asdf () NIL)) but unsurprisingly it doesn't help either. --- I see that ASDF is signaling a load-system-definition-error condition. Without knowing the root cause, I don't think handling it would help. I'll try to avoid the error: ;; ASDF, find-system.lisp #'(lambda (condition) (if (uiop:symbol-call :deploy :deployed-p) ;; added (warn "asdf: ignoring load-system-definition-error~&") ;; added (error 'load-system-definition-error :name (coerce-name s) :pathname pathname :condition condition))) Here, we use Deploy's (deploy:deployed-p) that tells us when the binary is running. It could be another flag. I'll try and see… Thanks for any pointers, Best regards, Vincent @vindarel SBCL 2.1.5 ASDF "3.3.4.15" on my development machine

1 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

asdf-devel