asdf-devel

asdf-devel@common-lisp.net

1 participants
1378 discussions

Closed the last of the issues on the 3.3.6 milestone
by Robert Goldman 02 Aug '22

02 Aug '22

So I am definitely moving towards a release real soon now. So please test the new ASDF as you can, and report any issues you see.

1 0

Trying once more to get 3.3.6 out the door
by Robert Goldman 27 Jul '22

27 Jul '22

I have poked all the merge requests that were tagged for 3.3.6 If these are close to working, I will get them merged, otherwise I will drop them from the milestone, with an eye to getting 3.3.6 out soon. If you get a chance to test your systems against the current version 3.3.5.10 looking for any new problems, I would be very grateful. The regression tests can take us only so far.

1 0

[RFC] subsystems vs. modules
by Didier Verna 18 May '22

18 May '22

Hi there, until now, I had a tendency to split a library into several systems (more or less without thinking much) as soon as such or such feature was conditional (e.g. depending on CFFI availability, thread support, etc.). Now I realize that in many cases, a conditionally included module or component would suffice, and so I'm starting to think that maybe I should do that, unless it makes sense for the conditional part to be loaded sort of standalone, independently from the rest, in which case a subsystem is more appropriate. So my question is: WDYT, is there a general recommendation about this? Thanks. -- Resistance is futile. You will be jazzimilated. Lisp, Jazz, Aïkido: http://www.didierverna.info

4 4

Re: Define a Simple Echo-Op
by zacque 06 May '22

06 May '22

Hi phoebe, > [[PGP Signed Part:Undecided]] > I'll still get the warnings (even with one new warning: "No dependency > propagating scheme specified..."): > > Perhaps it was imprecise of me to say that defining a method on COMPONENT-DEPENDS-ON will prevent the deprecation warning. > Causing the default method to be invoked on an OPERATION that is not a direction-OPERATION is deprecated; defining a more > specific method which does CALL-NEXT-METHOD still invokes the deprecated behavior. > > What I meant was subclassing ASDF:OPERATION only > > You can subclass OPERATION without a direction-OPERATION, but you have to completely override COMPONENT-DEPENDS-ON > with your own behavior, never invoking the built-in method. I strongly recommend against doing this. > > So, all operations are by default downward and sideway unless they are > subclass of (OR DOWNWARD-OPERATION UPWARD-OPERATION SIDEWAY-OPERATION > SELFWARD-OPERATION NON-PROPAGATING-OPERATION) > > Er... I'm not comfortable with saying that. In ASDF 2, all operations behaved in a way that is now called being downward and > sideway. ASDF 3 treats operations which don't specify their dependency relationship in a way compatible with ASDF 2 to give > maintainers a chance to update old extensions. The beautiful dream is that, one day, the compatibility behavior will be replaced > with a hard error. Please do not treat the compatibility behavior as a part of ASDF 3's interface. > >From our exchange, I think its safe to summarise into this table: "Table of ASDF operation classes to subclass and the corresponding methods to define on your custom class." | | OPERATION | <direction>-OPERATION | *-op | |----------------------+------------+--------------------------+------| | perform | X | X | X | | component-depends-on | X (note 1) | | | | input-files | X | (Depending on your need) | | | output-files | X | (Depending on your need) | | Note 1: You have to completely override COMPONENT-DEPENDS-ON for your custom class without invoking (CALL-NEXT-METHOD). As a new ASDF extension writer, I think having a table like this helps a lot to easily getting started. > Is it a linear chain or a "network chain" of > operations? > > The latter. When you define a subclass of SELFWARD-OPERATION, you specify what other operation(s) it is selfward to by > overriding the :INITFORM of the slot SELFWARD-OPERATION. The value you provide may be either a single operation designator, > or a list of operation designators. For example, (LOAD-OP COMPONENT) depends on both (COMPILE-OP COMPONENT) and > (PREPARE-OP COMPONENT), so its SELFWARD-OPERATION slot is defined: > > (selfward-operation :initform '(prepare-op compile-op) :allocation :class) > > can be done in parallel, > > But how can I > express the operation dependencies such that ASDF knows PRINT-OP depends > on ECHO-OP? > > (defclass print-op (asdf:selfward-operation) > ((asdf:selfward-operation :initform 'echo-op :allocation :class))) > Thanks! I have tried it out and it works. Still need some time to understand its behaviour though. > ASDF as it exists now is not capable of running anything in parallel. Faré has an experimental project to do this, called POIU, but I > can't speak for its status. Until very recently, SBCL was incapable of compiling files in parallel, and still today, lots of code does > not-thread-safe stuff at load-time, so I'm not convinced it's practically possible to make ASDF parallel via shared-memory threaded > concurrency. I believe POIU compiles each component in a separate process, then loads them sequentially. I see, that's good to know. At least it's an implementation limitation and not a design limitation I guess. -- Regards, zacque

2 1

Re: Define a Simple Echo-Op
by zacque 28 Apr '22

28 Apr '22

Hi phoebe, Thanks for your thoughtful reply! > A few things. > > 1. ECHO-OP should not be selfward. SELFWARD-OPERATION is for (OPERATION COMPONENT) pairs which depend on > (DIFFERENT-OPERATION COMPONENT) for the same COMPONENT. For example, LOAD-OP is selfward with respect to > COMPILE-OP, because in order to perform (load-op FILE), you must first perform (compile-op FILE). Your ECHO-OP has no such > dependency. In this case, I believe you want ECHO-OP to be downward and sideways, meaning that (ECHO-OP MODULE) > depends on (ECHO-OP MODULE-COMPONENT) for each of the children MODULE-COMPONENTs of the MODULE, and that > (ECHO-OP SOURCE-FILE) depends on (ECHO-OP EARLIER-SOURCE-FILE) for each of the EARLIER-SOURCE-FILEs in > SOURCE-FILE's :DEPENDS-ON list. This way, when you call (OPERATE 'ECHO-OP (FIND-SYSTEM "whatever")), ASDF will do a > depth-first dependency-order traversal of your system. After playing a little with downward and sideway, I can observe that downward-operation provides a depth-search/"postorder" traversal for the current system; sideway-operation provides a dependency-first traversal for dependent systems. * Note: Use :force all with OPERATE to force reloading all dependent systems. E.g. (asdf:operate 'echo-op:echo-op :echo-op-test :force :all) * Question: Why can't I subclass ASDF:OPERATION? This is unexpected since ASDF:OPERATION is the base class for all operations. My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op) (defclass echo-op (asdf:operation) ()) (defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" op c (asdf:input-files op c))) ------------------------------------------------- Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. WARNING: DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON :FOR-OPERATION #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API. Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> has input files: Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> has input files: Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> has input files: #<ECHO-OP:ECHO-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {1015FB0BC3}> ------------------------------------------------- > 2. Your COMPONENT-DEPENDS-ON method is wrong. No pair of (OPERATION COMPONENT) should ever depend on the same > (OPERATION COMPONENT). What you're saying is, "in order to perform (ECHO-OP FILE), you must first perform (ECHO-OP > FILE)." > > 3. For operations that subclass one or more of DOWNWARD- UPWARD- SIDEWAY- SELFWARD- or > NON-PROPOGATING-OPERATION, you don't need to define a COMPONENT-DEPENDS-ON method. Why is my COMPONENT-DEPENDS-ON method wrong? It's not obvious that one doesn't need to define a COMPONENT-DEPENDS-ON method for >=1 *-OPERATION. Is it stated anywhere? I'm following the manual: "If the action of performing the operation on a component has dependencies, you must define a method on component-depends-on." and following the examples in cffi-grovel: https://github.com/cffi/cffi/blob/3c76afe7ba03ce015e0df99ac9ddcd61320a44a4/… That said, my code works fine by removing my COMPONENT-DEPENDS-ON method. And I can see what you meant by (ECHO-OP FILE) depending on itself by printing it out: My code: ----------- file: echo-op.lisp ------------------- (in-package #:echo-op) (defclass echo-op (asdf:sideway-operation asdf:downward-operation) ()) (defmethod asdf:perform ((op echo-op) c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c))) ------------------------------------------------- Output: ----------------- REPL -------------------------- CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> depends on (#<ECHO-OP >) Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> depends on (#<ECHO-OP >) (#<ECHO-OP > #<CL-SOURCE-FILE "echo-op-test" "package"> #<CL-SOURCE-FILE "echo-op-test" "example"> #<CL-SOURCE-FILE "echo-op-test" "main">) (DEFINE-OP echo-op-test) #<ECHO-OP:ECHO-OP > #<ASDF/PLAN:SEQUENTIAL-PLAN {10162397C3}> ------------------------------------------------- But still I don't get why echo-op depends on itself. I didn't specify it anyway. Or it that the default behaviour for all *-operation classes? > 4. Most (OPERATION COMPONENT) pairs have very uninteresting sets of input files. (COMPILE-OP CL-SOURCE-FILE) has one > input file, the .lisp source file. (LOAD-OP CL-SOURCE-FILE) has one input file, the .fasl compiled file. (ECHO-OP CL-SOURCE-FILE) > will have no input files at all, unless you define a method on INPUT-FILES to list them. Why do I get nothing for my input-files? I'm expecting it to print out the pathname to (:FILE "foo") object, very much like the input files into COMPILE-OP. Again, I thought ASDF is smart enough to infer it for me? To quote the manual: "A method for this function is often not needed, since ASDF has a pretty clever default input-files mechanism." Is there a way to obtain the pathname to (:FILE "foo") from the CL-SOURCE-FILE object like what COMPILE-OP does? > I think the following definition of ECHO-OP might be enlightening to you: > > (uiop:define-package :echo-op > (:use :cl) > (:export #:echo-op)) > (in-package :echo-op) > > (defclass echo-op (asdf:sideway-operation asdf:downward-operation) ()) > > (defun print-input-files (op c) > (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" > op c (asdf:input-files op c))) > > (defun print-dependencies (op c) > (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" > op c (asdf:component-depends-on op c))) > > (defmethod asdf:perform ((op echo-op) c) > (flet ((do-operations (thunk) > (funcall thunk op c) > (funcall thunk (asdf:make-operation 'asdf:compile-op) c) > (funcall thunk (asdf:make-operation 'asdf:load-op) c))) > (format t "~&~%Input files for component ~a with a variety of operations:~%~%" c) > (do-operations #'print-input-files) > (format t "~&~%Dependencies for component ~a with a variety of operations:~%~%" c) > (do-operations #'print-dependencies))) > > Note that: > 1. The only method I have defined is on PERFORM, and it is a primary method, not an :AROUND method. ASDF already has all > the COMPONENT-DEPENDS-ON methods I need. > 2. I print the COMPONENT-DEPENDS-ON list in addition to the INPUT-FILES list. > 3. I print both the COMPONENT-DEPENDS-ON and INPUT-FILES lists for all three of ECHO-OP, COMPILE-OP and LOAD-OP. > > I recommend you load this version, try (ASDF:OPERATE 'ECHO-OP:ECHO-OP (ASDF:FIND-SYSTEM "echo-op-test") :FORCE T) and > see what output you get. Thanks! It's indeed very enlightening. And I've separated it out into few parts to play with, as shown in the code snippet I pasted above. One question: How do you know whether to define PERFORM primary method or an :AROUND method for the custom operation class? It's not obvious to me which one to choose for different use cases. Thanks! -- Regards, zacque

3 4

Re: Define a Simple Echo-Op
by Robert Goldman 27 Apr '22

27 Apr '22

On 27 Apr 2022, at 11:15, phoebe Goldman wrote: >> According to the docs, most operations inherit from self ward-operation, which would take the component-pathname as an input file (and might take others). > > I think input file sets are orthogonal to dependency direction. SELFWARD-OPERATION means that (OP COMPONENT) depends on (DIFFERENT-OP COMPONENT), like how (LOAD-OP LISP-SOURCE-FILE) depends on (COMPILE-OP LISP-SOURCE-FILE). That doesn't mean that (LOAD-OP LISP-SOURCE-FILE) takes COMPONENT-PATHNAME as an INPUT-FILE. This is what I see in the source, but I haven't had enough time to read it over carefully (or really, at all): ``` ;; An action with a selfward-operation by default gets its input-files from the output-files of ;; the actions using selfward-operations it depends on (and the same component), ;; or if there are none, on the component-pathname of the component if it's a file ;; -- and then on the results of the next-method. (defmethod input-files ((o selfward-operation) (c component)) `(,@(or (loop :for dep-o :in (ensure-list (selfward-operation o)) :append (or (output-files dep-o c) (input-files dep-o c))) (if-let ((pathname (component-pathname c))) (and (file-pathname-p pathname) (list pathname)))) ,@(call-next-method)))) ```

1 0

can't reply to asdf list Fwd: Undelivered Mail Returned to Sender
by Robert Goldman 26 Apr '22

26 Apr '22

Response from Phoebe Goldman hand-forwarded because of problems with common-lisp.net's email server: A few things. 1. ECHO-OP should not be selfward. SELFWARD-OPERATION is for (OPERATION COMPONENT) pairs which depend on (DIFFERENT-OPERATION COMPONENT) for the same COMPONENT. For example, LOAD-OP is selfward with respect to COMPILE-OP, because in order to perform (load-op FILE), you must first perform (compile-op FILE). Your ECHO-OP has no such dependency. In this case, I believe you want ECHO-OP to be downward and sideways, meaning that (ECHO-OP MODULE) depends on (ECHO-OP MODULE-COMPONENT) for each of the children MODULE-COMPONENTs of the MODULE, and that (ECHO-OP SOURCE-FILE) depends on (ECHO-OP EARLIER-SOURCE-FILE) for each of the EARLIER-SOURCE-FILEs in SOURCE-FILE's :DEPENDS-ON list. This way, when you call (OPERATE 'ECHO-OP (FIND-SYSTEM "whatever")), ASDF will do a depth-first dependency-order traversal of your system. 2. Your COMPONENT-DEPENDS-ON method is wrong. No pair of (OPERATION COMPONENT) should ever depend on the same (OPERATION COMPONENT). What you're saying is, "in order to perform (ECHO-OP FILE), you must first perform (ECHO-OP FILE)." 3. For operations that subclass one or more of DOWNWARD- UPWARD- SIDEWAY- SELFWARD- or NON-PROPOGATING-OPERATION, you don't need to define a COMPONENT-DEPENDS-ON method. 4. Most (OPERATION COMPONENT) pairs have very uninteresting sets of input files. (COMPILE-OP CL-SOURCE-FILE) has one input file, the .lisp source file. (LOAD-OP CL-SOURCE-FILE) has one input file, the .fasl compiled file. (ECHO-OP CL-SOURCE-FILE) will have no input files at all, unless you define a method on INPUT-FILES to list them. I think the following definition of ECHO-OP might be enlightening to you: ``` (uiop:define-package :echo-op (:use :cl) (:export #:echo-op)) (in-package :echo-op) (defclass echo-op (asdf:sideway-operation asdf:downward-operation) ()) (defun print-input-files (op c) (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%" op c (asdf:input-files op c))) (defun print-dependencies (op c) (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%" op c (asdf:component-depends-on op c))) (defmethod asdf:perform ((op echo-op) c) (flet ((do-operations (thunk) (funcall thunk op c) (funcall thunk (asdf:make-operation 'asdf:compile-op) c) (funcall thunk (asdf:make-operation 'asdf:load-op) c))) (format t "~&~%Input files for component ~a with a variety of operations:~%~%" c) (do-operations #'print-input-files) (format t "~&~%Dependencies for component ~a with a variety of operations:~%~%" c) (do-operations #'print-dependencies))) ``` Note that: 1. The only method I have defined is on PERFORM, and it is a primary method, not an :AROUND method. ASDF already has all the COMPONENT-DEPENDS-ON methods I need. 2. I print the COMPONENT-DEPENDS-ON list in addition to the INPUT-FILES list. 3. I print both the COMPONENT-DEPENDS-ON and INPUT-FILES lists for all three of ECHO-OP, COMPILE-OP and LOAD-OP. I recommend you load this version, try (ASDF:OPERATE 'ECHO-OP:ECHO-OP (ASDF:FIND-SYSTEM "echo-op-test") :FORCE T) and see what output you get. cheers, phoebe > On Apr 26, 2022, at 3:40 AM, zacque <technical+asdf-devel(a)zacque.tk <mailto:technical+asdf-devel@zacque.tkwrote: >>> > Hi, >>> > I'm trying to learn how ASDF object model works by defining a simple > operation "echo-op". I want it to print out pathnames of all loaded lisp > files to *standard-output*. It sounds like a simple task, but I couldn't > get it to work. Thus I'm asking for help here. >>> > To do that, I define a project: > ----------- file: echo-op.asd ------------------- > (defsystem #:echo-op > :components > ((:file "package") > (:file "echo-op"))) > ------------------------------------------------- >>> > ----------- file: package.lisp ------------------- > (defpackage #:echo-op > (:use #:cl #:asdf) > (:export > #:selfward-echo-op)) > ------------------------------------------------- >>> > ----------- file: echo-op.lisp ------------------- > (in-package #:echo-op) >>> > (defclass selfward-echo-op (selfward-operation) > ()) >>> > (defmethod asdf:component-depends-on ((op selfward-echo-op) c) > `((selfward-echo-op ,c) ,@(call-next-method))) >>> > (defmethod asdf:output-files ((op selfward-echo-op) c) > nil) >>> > (defmethod asdf:perform :around ((op selfward-echo-op) c) > (let ((input-files (asdf:input-files op c))) > (loop for file in input-files > do (format t "~s" file)))) >>> > (setf (find-class 'asdf::selfward-echo-op) (find-class 'selfward-echo-op)) > ------------------------------------------------- >>> > Then, if I run these from the REPL: > ----------- REPL ------------------- > CL-USER> (asdf:load-system :echo-op :force t) > ; compiling file > ; compilation finished in 0:00:00.004 > ; compiling file > ; wrote > ; compilation finished in 0:00:00.036 > T > CL-USER> (asdf:operate 'asdf::selfward-echo-op :echo-op-test) > ; Evaluation aborted on #<SB-PCL::NO-APPLICABLE-METHOD-ERROR {1002341273}>. > ------------------------------------------------- >>> > I got the error: > ----------- SLIME *sldb* ------------------- > The slot ASDF/ACTION:SELFWARD-OPERATION is unbound in the object > #<SELFWARD-ECHO-OP >. > [Condition of type UNBOUND-SLOT] > ------------------------------------------------- >>> > The "echo-op-test" system is simply: > ----------- file: echo-op-test.lisp ------------------- > (defsystem #:echo-op-test > :defsystem-depends-on (#:echo-op) > :components > ((:file "package") > (:file "example") > (:file "main"))) > ------------------------------------------------- > with empty files package.lisp, example.lisp, and main.lisp. >>> > Now, if I redefine selfward-echo-op to subclass load-op, I got this > error instead: > ----------- SLIME *sldb* ------------------- > Circular dependency of > #1=(#<ECHO-OP:SELFWARD-ECHO-OP > > . #<ASDF/SYSTEM:SYSTEM "echo-op-test">) > on: > (#1#) > [Condition of type ASDF/ACTION:CIRCULAR-DEPENDENCY] > ------------------------------------------------- > which I have no idea what is going on. >>> > I'm still learning about ASDF, so any help to achieve what I want to do > would be very much appreciated. >>> > Thanks! >>> > -- > Regards, > zacque >>> >> > > >

1 0

Define a Simple Echo-Op
by zacque 26 Apr '22

26 Apr '22

Hi, I'm trying to learn how ASDF object model works by defining a simple operation "echo-op". I want it to print out pathnames of all loaded lisp files to *standard-output*. It sounds like a simple task, but I couldn't get it to work. Thus I'm asking for help here. To do that, I define a project: ----------- file: echo-op.asd ------------------- (defsystem #:echo-op :components ((:file "package") (:file "echo-op"))) ------------------------------------------------- ----------- file: package.lisp ------------------- (defpackage #:echo-op (:use #:cl #:asdf) (:export #:selfward-echo-op)) ------------------------------------------------- ----------- file: echo-op.lisp ------------------- (in-package #:echo-op) (defclass selfward-echo-op (selfward-operation) ()) (defmethod asdf:component-depends-on ((op selfward-echo-op) c) `((selfward-echo-op ,c) ,@(call-next-method))) (defmethod asdf:output-files ((op selfward-echo-op) c) nil) (defmethod asdf:perform :around ((op selfward-echo-op) c) (let ((input-files (asdf:input-files op c))) (loop for file in input-files do (format t "~s" file)))) (setf (find-class 'asdf::selfward-echo-op) (find-class 'selfward-echo-op)) ------------------------------------------------- Then, if I run these from the REPL: ----------- REPL ------------------- CL-USER> (asdf:load-system :echo-op :force t) ; compiling file ; compilation finished in 0:00:00.004 ; compiling file ; wrote ; compilation finished in 0:00:00.036 T CL-USER> (asdf:operate 'asdf::selfward-echo-op :echo-op-test) ; Evaluation aborted on #<SB-PCL::NO-APPLICABLE-METHOD-ERROR {1002341273}>. ------------------------------------------------- I got the error: ----------- SLIME *sldb* ------------------- The slot ASDF/ACTION:SELFWARD-OPERATION is unbound in the object #<SELFWARD-ECHO-OP >. [Condition of type UNBOUND-SLOT] ------------------------------------------------- The "echo-op-test" system is simply: ----------- file: echo-op-test.lisp ------------------- (defsystem #:echo-op-test :defsystem-depends-on (#:echo-op) :components ((:file "package") (:file "example") (:file "main"))) ------------------------------------------------- with empty files package.lisp, example.lisp, and main.lisp. Now, if I redefine selfward-echo-op to subclass load-op, I got this error instead: ----------- SLIME *sldb* ------------------- Circular dependency of #1=(#<ECHO-OP:SELFWARD-ECHO-OP > . #<ASDF/SYSTEM:SYSTEM "echo-op-test">) on: (#1#) [Condition of type ASDF/ACTION:CIRCULAR-DEPENDENCY] ------------------------------------------------- which I have no idea what is going on. I'm still learning about ASDF, so any help to achieve what I want to do would be very much appreciated. Thanks! -- Regards, zacque

1 0

[Openmcl-devel] My asdf problem
by Paul Krueger 05 Apr '22

05 Apr '22

I am pretty much certain that I now understand what happened to cause my asdf problem. At the end of this I have a few final questions that maybe somebody can answer definitively for me. If you are curious about the details, read on. Here are some background facts: What I was seeing is a failure of asdf to locate a new system in my ~/common-lisp directory as it should. This problem only occurred in the CCL IDE, not when running CCL from the command line. There is an asdf internal variable called asdf/source-registry:*source-registry* which contains a list of all the .asd files it can find as a result of searching all of the directories where asdf thinks they might be. In my case that was my ~/common-lisp/ directory. In asdf itself. That variable is only initialized when a file is searched for. That might happen either via an explicit asdf call (e.g.(asdf::find-system <something>) or indirectly via a quickload call such as (ql::quickload <something>). Just loading asdf will not cause the *source-registry* to be initialized However, it turns out that just loading quickload WILL cause the asdf *source-registry* variable to be initialized. Once that variable is initialized, asdf will never try to initialize it again since that would waste a lot of time looking for .asd files all over again. So knowing all that, here is what happened. I installed and rebuilt the latest CCL a couple of months ago. As part of building the CCL IDE after that, both asdf and quickload were loaded into the CCL IDE image that was created (one question later is related to this) As a normal result of loading quickload, the asdf/source-registry:*source-registry* variable was initialized (another question below is related to this) When the CCL IDE image was built it included the value of that asdf *source-registry* variable, so that when the IDE was launched that variable was initialized. Only recently did I add additional asdf-defined systems to the ~/common-lisp/ directory. But of course when asdf tried to find them it couldn’t because the *source-registry* variable was effectively initialized when the IDE was built (long before I added my new systems). I can explicitly call an asdf function to reinitialize that variable and then everything is found normally. One question I have left is whether loading asdf and quickload into the CCL IDE is something that is done explicitly by the process that is now used to build the IDE or whether there is some option to not include them. Was there something that I should have done or not done when building the IDE that would have prevented this problem? Had there been nothing in my ~/common-lisp directory to find, then the value of that asdf variable would have been nil and that is enough to make asdf reinitialize it. But assuring that each time I build seems like a pain. My next question is why loading quicklisp causes it to initialize that variable? Conceivably it needs to search for .asd files that might be in its load environment and uses asdf to do so. Next I’m wondering whether there is some change to asdf code that can be made to reinitialize the *source-registry* every time the IDE is re-started. As another possibility, perhaps the CCL code for dumping the IDE can be modified in some way. In a private email Robert suggested: "If you have time to look, take a peek at asdf/uiop/image.lisp and asdf/bundle.lisp which should have code Fare wrote about dumping an image in a way that it will restart happily.” That seems like something that CCL maintainers might want to consider. While waiting for fixes or other suggestions, I will put an explicit call to reinitialize the *source-registry* in my ccl-ide-init.lisp file: (asdf/source-registry::initialize-source-registry) If anyone else has an environment similar to mine and rebuilds their own version of the IDE, I would suggest they do the same.

1 0

Forever Stable Branch
by Michał "phoe" Herda 20 Jan '22

20 Jan '22

Hey, I've poured my thoughts related to recent events into a blogpost that turned into a long-ish article. This mail is purely informational; comments via mail are welcome but not required. https://gist.github.com/phoe/7d24bdb1f2be76a02fecba8cfecbef38 Linking a Reddit thread below and attaching the original article to this mail for posterity. https://www.reddit.com/r/Common_Lisp/comments/s8r075/ BR ~phoe

1 0

← Newer
1
2
3
4
5
6
7
8
...
138
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

asdf-devel