Hi, I'm interested in analyzing dependencies among a collection of Lisp files, or analyzing dependencies stated in a .asd file, or both.
(1) Towards the goal of analyzing dependencies among Lisp files, I'm trying to load asdf-dependency-grovel, and I'm not having any luck. I obtained a tar.gz of the current version from Gitlab, from the project https://gitlab.common-lisp.net/xcvb/asdf-dependency-grovel. I found it necessary to patch tests/run-tests.sh to get anywhere (patch below) and I get the following error at this point:
------------------------------------------------ $ sh tests/run-tests.sh This is SBCL 2.1.6, an implementation of ANSI Common Lisp.
WARNING: redefining UIOP/UTILITY:PARSE-BODY in DEFUN WARNING: Deprecated recursive use of (ASDF/OPERATE:OPERATE 'ASDF/LISP-ACTION:LOAD-OP '("test-serial-system")) while visiting (ASDF-DEPENDENCY-GROVEL:DEPENDENCY-OP "test-serial" "asdf-dependency-grovel-test/serial") - please use proper dependencies instead
debugger invoked on a SIMPLE-ERROR in thread #<THREAD "main thread" RUNNING {988AB081}>: Just performed compiling #<INSTRUMENTED-CL-SOURCE-FILE "test-serial-system" "package"> but failed to mark it done ------------------------------------------------
Can anyone offer any advice about the "failed to mark it done" error? or the bit about "deprecated recursive use"?
(2) About looking at dependencies stated in a .asd file, I have a .asd file which ASDF is refusing to load because it has a circular dependency. Is there a way to get ASDF to tell me what that circular dependency is? Failing that, can someone suggest any tools to extract a directed graph from a .asd file? I suppose it wouldn't be too hard to do it myself but no need to reinvent the wheel.
I'm thinking I could find a graph analysis library to extract any cycles, if I had a directed graph in hand. I don't suppose someone has already automated looking for cycles in .asd files?
Thanks in advance for any advice, I appreciate your help.
best,
Robert Dodier
PS. Here's my patch for asdf-dependency-grovel.
$ diff -u tests/run-tests.sh-original tests/run-tests.sh --- tests/run-tests.sh-original 2022-12-13 22:39:50.174826971 -0800 +++ tests/run-tests.sh 2022-12-13 22:43:41.675974917 -0800 @@ -5,7 +5,10 @@
ADG_TEST_DIR="$(dirname $0)"
-$LISP --load ${ADG_TEST_DIR}/grovel-tests.lisp --eval '(uiop:quit (asdf-dependency-grovel-tester:test-result))' +$LISP --eval '(require (quote asdf))'\ + --eval '(push "/home/robert/by-others/asdf-dependency-grovel-master/" asdf:*central-registry*)'\ + --eval '(push "/home/robert/by-others/asdf-dependency-grovel-master/tests/" asdf:*central-registry*)'\ + --load ${ADG_TEST_DIR}/grovel-tests.lisp --eval '(uiop:quit (asdf-dependency-grovel-tester:test-result))' STATUS=$? rm -rf asdf-dependency-grovel-tmp-* #$LISP --load ${ADG_TEST_DIR}/grovel-tests.lisp --eval '(asdf-dependency-grovel-tester:check-base-deps)' --eval '(uiop:quit)'
On 15 Dec 2022, at 1:07, Robert Dodier wrote:
Hi, I'm interested in analyzing dependencies among a collection of Lisp files, or analyzing dependencies stated in a .asd file, or both.
(1) Towards the goal of analyzing dependencies among Lisp files, I'm trying to load asdf-dependency-grovel, and I'm not having any luck. I obtained a tar.gz of the current version from Gitlab, from the project https://gitlab.common-lisp.net/xcvb/asdf-dependency-grovel. I found it necessary to patch tests/run-tests.sh to get anywhere (patch below) and I get the following error at this point:
$ sh tests/run-tests.sh This is SBCL 2.1.6, an implementation of ANSI Common Lisp.
WARNING: redefining UIOP/UTILITY:PARSE-BODY in DEFUN WARNING: Deprecated recursive use of (ASDF/OPERATE:OPERATE 'ASDF/LISP-ACTION:LOAD-OP '("test-serial-system")) while visiting (ASDF-DEPENDENCY-GROVEL:DEPENDENCY-OP "test-serial" "asdf-dependency-grovel-test/serial")
- please use proper dependencies instead
debugger invoked on a SIMPLE-ERROR in thread #<THREAD "main thread" RUNNING {988AB081}>: Just performed compiling #<INSTRUMENTED-CL-SOURCE-FILE "test-serial-system" "package"> but failed to mark it done
Can anyone offer any advice about the "failed to mark it done" error? or the bit about "deprecated recursive use"?
It would help to get a backtrace here. I can't tell from this if the error is happening in ASDF or in ASDF-DEPENDENCY-GROVEL (which probably isn't maintained).
Also whose `run-tests.sh` are these? ASDF's tests or ASDF-dependency-grovel's?
(2) About looking at dependencies stated in a .asd file, I have a .asd file which ASDF is refusing to load because it has a circular dependency. Is there a way to get ASDF to tell me what that circular dependency is? Failing that, can someone suggest any tools to extract a directed graph from a .asd file? I suppose it wouldn't be too hard to do it myself but no need to reinvent the wheel.
I'm thinking I could find a graph analysis library to extract any cycles, if I had a directed graph in hand. I don't suppose someone has already automated looking for cycles in .asd files?
Thanks in advance for any advice, I appreciate your help.
Again, it would help to have a backtrace from this circular dependency to help us get started (if you can share it -- I suppose your system could be private/proprietary).
I just looked up the definition of the `circular-dependency` condition (in `asdf/action.lisp` around line 142) and it has a slot that contains the actions that create the cycle. Does that not give you enough information to isolate the problem? What are you seeing? The report function for this condition should be printing the set of actions in the loop (or at least I think so, that's some hairy `format` string there!).
I wonder if we should adopt `asdf-dependency-grovel` into the ASDF group on cl.net's GitLab? There's no guarantee that would result in active maintenance, though -- it should be obvious that I don't have a lot of cycles to spare...
R
Robert, thanks for your reply. I have made some progress.
Just performed compiling #<INSTRUMENTED-CL-SOURCE-FILE "test-serial-system" "package"> but failed to mark it done
It would help to get a backtrace here. I can't tell from this if the error is happening in ASDF or in ASDF-DEPENDENCY-GROVEL (which probably isn't maintained).
The "failed to mark it done" is coming from ASDF. The ASDF version (bundled with SBCL) is, I think, 3.2.something. I tried it with Clozure CL, which has (on my system at least) ASDF 3.1.something, and the same operation succeeds. I guess I'm not too worried about it, since I now have a workaround, but I'm curious to know what's going on here.
Also whose run-tests.sh are these? ASDF's tests or ASDF-dependency-grovel's?
For the record, it's the run-tests.sh of asdf-dependency-grovel.
Again, it would help to have a backtrace from this circular dependency to help us get started (if you can share it -- I suppose your system could be private/proprietary).
About the circular dependency, it's in Maxima's ASDF file. After puzzling over a backtrace (thanks for the hint) I figured out it's because :serial t is specified, and a module depends on something later in the .asd. I moved the offending module and that resolved the dependency error. Hurray!
I wonder if we should adopt asdf-dependency-grovel into the ASDF group on cl.net's GitLab? There's no guarantee that would result in active maintenance, though -- it should be obvious that I don't have a lot of cycles to spare...
Well, I think that sounds like a good idea. I know developing new code is unlikely but it would be great to at least make any updates necessary to keep up with new ASDF versions.
asdf-dependency-grovel seems like a really great idea -- I am hoping to use it to analyze dependencies in Maxima, which, as you may know, is an ancient and pretty large system ... If asdf-dependency-grovel didn't exist, I would have to reinvent some wheels.
best,
Robert
On 15 Dec 2022, at 14:48, Robert Dodier wrote:
Robert, thanks for your reply. I have made some progress.
Just performed compiling #<INSTRUMENTED-CL-SOURCE-FILE "test-serial-system" "package"> but failed to mark it done
It would help to get a backtrace here. I can't tell from this if the error is happening in ASDF or in ASDF-DEPENDENCY-GROVEL (which probably isn't maintained).
The "failed to mark it done" is coming from ASDF. The ASDF version (bundled with SBCL) is, I think, 3.2.something. I tried it with Clozure CL, which has (on my system at least) ASDF 3.1.something, and the same operation succeeds. I guess I'm not too worried about it, since I now have a workaround, but I'm curious to know what's going on here.
FWIW, *both* of these are obsolete. The current release is 3.3.6 which you can get from the ASDF home page (at common-lisp.net)
I don't see any reason that 3.3.6 would fix the problem you see, but we can't debug old versions of ASDF.
It should be easy to install the new version of ASDF on any lisp if you grab the monolithic file and load it in your lisp .rc file
Also whose run-tests.sh are these? ASDF's tests or ASDF-dependency-grovel's?
For the record, it's the run-tests.sh of asdf-dependency-grovel.
Again, it would help to have a backtrace from this circular dependency to help us get started (if you can share it -- I suppose your system could be private/proprietary).
About the circular dependency, it's in Maxima's ASDF file. After puzzling over a backtrace (thanks for the hint) I figured out it's because :serial t is specified, and a module depends on something later in the .asd. I moved the offending module and that resolved the dependency error. Hurray!
I wonder if we should adopt asdf-dependency-grovel into the ASDF group on cl.net's GitLab? There's no guarantee that would result in active maintenance, though -- it should be obvious that I don't have a lot of cycles to spare...
Well, I think that sounds like a good idea. I know developing new code is unlikely but it would be great to at least make any updates necessary to keep up with new ASDF versions.
asdf-dependency-grovel seems like a really great idea -- I am hoping to use it to analyze dependencies in Maxima, which, as you may know, is an ancient and pretty large system ... If asdf-dependency-grovel didn't exist, I would have to reinvent some wheels.
Forking it into ASDF would make sense because then at least we could set up GitLab CI and identify when it does or does not work with the most up-to-date ASDF.
If you think this is a good idea, and would be willing to set up a fork of asdf-dependency-grovel, please contact off-list with your cl.net account (they are easy to get if you don't have one), and I can add you to the group. Then you would be able to fork A-D-G into the ASDF group. From there I could set up the GitLab CI to run the tests automagically.
-
Robert Dodier -
Robert Goldman