Re: ASDF dependency graph or complexity score
What kind of dependency graph are you looking to make? Just the systems, or also the individual system components? And do you want a visible representation of this graph, or just some set of data structures? I assume the former, since you should be able to use the system components themselves to get all the dependency links. If it's the former, the easiest thing might be to use CL-DOT and write the necessary methods to create a graph. You might need to get ASDF to compute the plan for `LOAD-OP` as a way to ensure that all the required system definitions are loaded. With respect to test systems, I have usually found it best to have my systems, say "foo", have "foo/test" related systems, delegate the `test-op` for "foo" to "foo/test" using `in-order-to`, and then only "foo/test" needs to depend on the test library -- "foo" itself doesn't need it. HtH, R On 23 Dec 2019, at 19:08, Jay wrote:
Thanks.
I will follow up with Rob later.
Anyway, thanks for help in the past. I will liaise with Rob to figure out the best way forward.
Jay
Faré 6:52 PM (14 minutes ago) to me
I guess, I will have to step up at some point. I have a tool that's built with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above. Then, I will be able to tackle the graph thing, as you said it's no easy fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at #lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Mon, Dec 23, 2019 at 6:52 PM Faré <fahree@gmail.com> wrote:
I guess, I will have to step up at some point. I have a tool that's built with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above. Then, I will be able to tackle the graph thing, as you said it's no easy fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at #lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Fri, Nov 29, 2019 at 11:33 AM Faré <fahree@gmail.com> wrote:
I could not follow up in the past as I had to relocate between cities
(central to west coast) so that caused a lot of changes in my schedule. What is the state of the asdf contributors you mentioned last time -- the developers taking over asdf development albeit gradually.
No one has really stepped forward. It's still Robert Goldman maintaining things, while I have moved away from active maintenance and am only reviewing patches and giving advice, sometimes making small edits to documentation.
I just started working on my fairly large system again, and recently had issues when I was trying to disentangle systems to avoid loading multiple test systems that clobber global variables. A secondary goal is to reduce unnecessary dependencies ( btw I use :class :package-inferred-system in my setup).
Is there some asdf option that I can use to get the graph for a given loaded system or some score of complexity due to linkages. I have been able to improve this over the years and the system is a more stable but occasionally hangs when it has to load many packages (hundreds of files) after compiler update (using SBCL primarily)
Aha. Well, it used to be that POIU could extract a dependency graph from an ASDF system, and act on it to build; but this has bitrotten a bit with the latest changes in ASDF 3.3, and POIU is not currently very stable. If you could fix it, that would be great. But beware, it's no small undertaking: you have to really get into a lot of internals to do it right, and make non-trivial changes to UIOP, likely including subtle changes to ASDF itself.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Appropriate fear keeps you alive. Excessive fear prevents the very thing it's supposed to protect: life. — Attila Lendvai
Thanks,
Jay
I updated the POIU README with remarks based on our discussion so far. After thinking a bit more, I suspect it is not *that* hard to completely fix POIU and extract (and display) a precise ASDF dependency graph from it. But it's still some amount of serious work and a walk through the internals or ASDF and POIU. NB: I'm talking about POIU, not UIOP. Even I got confused in my previous messages at times. Oops. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The idea is not to confront bad ideas but to come up with good ideas. Otherwise, your enemies define the game and you are the loyal opposition. — Terence McKenna On Tue, Dec 24, 2019 at 12:17 PM Robert Goldman <rpgoldman@sift.info> wrote:
What kind of dependency graph are you looking to make? Just the systems, or also the individual system components?
And do you want a visible representation of this graph, or just some set of data structures?
I assume the former, since you should be able to use the system components themselves to get all the dependency links.
If it's the former, the easiest thing might be to use CL-DOT and write the necessary methods to create a graph.
You might need to get ASDF to compute the plan for LOAD-OP as a way to ensure that all the required system definitions are loaded.
With respect to test systems, I have usually found it best to have my systems, say "foo", have "foo/test" related systems, delegate the test-op for "foo" to "foo/test" using in-order-to, and then only "foo/test" needs to depend on the test library -- "foo" itself doesn't need it.
HtH,
R
On 23 Dec 2019, at 19:08, Jay wrote:
Thanks.
I will follow up with Rob later.
Anyway, thanks for help in the past. I will liaise with Rob to figure out the best way forward.
Jay
Faré 6:52 PM (14 minutes ago) to me
I guess, I will have to step up at some point. I have a tool that's built
with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above.
Then, I will be able to tackle the graph thing, as you said it's no easy
fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with
easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at
#lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Mon, Dec 23, 2019 at 6:52 PM Faré <fahree@gmail.com> wrote:
I guess, I will have to step up at some point. I have a tool that's
built with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above.
Then, I will be able to tackle the graph thing, as you said it's no easy
fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with
easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at
#lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Fri, Nov 29, 2019 at 11:33 AM Faré <fahree@gmail.com> wrote:
I could not follow up in the past as I had to relocate between cities
(central to west coast) so that caused a lot of changes in my schedule. What is the state of the asdf contributors you mentioned last time -- the developers taking over asdf development albeit gradually.
No one has really stepped forward. It's still Robert Goldman maintaining things, while I have moved away from active maintenance and am only reviewing patches and giving advice, sometimes making small edits to documentation.
I just started working on my fairly large system again, and recently
had issues when I was trying to disentangle systems to avoid loading multiple test systems that clobber global variables. A secondary goal is to reduce unnecessary dependencies ( btw I use :class :package-inferred-system in my setup).
Is there some asdf option that I can use to get the graph for a given
loaded system or some score of complexity due to linkages. I have been able to improve this over the years and the system is a more stable but occasionally hangs when it has to load many packages (hundreds of files) after compiler update (using SBCL primarily)
Aha. Well, it used to be that POIU could extract a dependency graph from an ASDF system, and act on it to build; but this has bitrotten a bit with the latest changes in ASDF 3.3, and POIU is not currently very stable. If you could fix it, that would be great. But beware, it's no small undertaking: you have to really get into a lot of internals to do it right, and make non-trivial changes to UIOP, likely including subtle changes to ASDF itself.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics•
Appropriate fear keeps you alive. Excessive fear prevents the very thing it's supposed to protect: life. — Attila Lendvai
Thanks,
Jay
Updated: Fare, Sorry for late reply. Just got back from travels. Thanks for the update. I did try to look for the updated README but didn't find it. I will try again to find the file. As I said before, I am not an expert with ASDF, but I am trying to go over the coupling of my modules and review dependencies. Loading with SBCL exhibits weird behavior where it stops half (freezes) and start over again and it runs ok. Rob also provided some additional info regarding extracting dependency information and I'll explore these in addition. Thanks On Thu, Dec 26, 2019 at 5:20 PM Faré <fahree@gmail.com> wrote:
I updated the POIU README with remarks based on our discussion so far. After thinking a bit more, I suspect it is not *that* hard to completely fix POIU and extract (and display) a precise ASDF dependency graph from it. But it's still some amount of serious work and a walk through the internals or ASDF and POIU.
NB: I'm talking about POIU, not UIOP. Even I got confused in my previous messages at times. Oops.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The idea is not to confront bad ideas but to come up with good ideas. Otherwise, your enemies define the game and you are the loyal opposition. — Terence McKenna
On Tue, Dec 24, 2019 at 12:17 PM Robert Goldman <rpgoldman@sift.info> wrote:
What kind of dependency graph are you looking to make? Just the systems,
or also the individual system components?
And do you want a visible representation of this graph, or just some set
of data structures?
I assume the former, since you should be able to use the system
components themselves to get all the dependency links.
If it's the former, the easiest thing might be to use CL-DOT and write
the necessary methods to create a graph.
You might need to get ASDF to compute the plan for LOAD-OP as a way to
ensure that all the required system definitions are loaded.
With respect to test systems, I have usually found it best to have my
systems, say "foo", have "foo/test" related systems, delegate the test-op for "foo" to "foo/test" using in-order-to, and then only "foo/test" needs to depend on the test library -- "foo" itself doesn't need it.
HtH,
R
On 23 Dec 2019, at 19:08, Jay wrote:
Thanks.
I will follow up with Rob later.
Anyway, thanks for help in the past. I will liaise with Rob to figure out the best way forward.
Jay
Faré 6:52 PM (14 minutes ago) to me
I guess, I will have to step up at some point. I have a tool that's built
with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above.
Then, I will be able to tackle the graph thing, as you said it's no easy
fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with
easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at
#lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Mon, Dec 23, 2019 at 6:52 PM Faré <fahree@gmail.com> wrote:
I guess, I will have to step up at some point. I have a tool that's
built with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above.
Then, I will be able to tackle the graph thing, as you said it's no easy
fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with
easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at
#lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Fri, Nov 29, 2019 at 11:33 AM Faré <fahree@gmail.com> wrote:
I could not follow up in the past as I had to relocate between cities
(central to west coast) so that caused a lot of changes in my schedule. What is the state of the asdf contributors you mentioned last time -- the developers taking over asdf development albeit gradually.
No one has really stepped forward. It's still Robert Goldman maintaining things, while I have moved away from active maintenance and am only reviewing patches and giving advice, sometimes making small edits to documentation.
I just started working on my fairly large system again, and recently
had issues when I was trying to disentangle systems to avoid loading multiple test systems that clobber global variables. A secondary goal is
to
reduce unnecessary dependencies ( btw I use :class :package-inferred-system in my setup).
Is there some asdf option that I can use to get the graph for a given
loaded system or some score of complexity due to linkages. I have been able to improve this over the years and the system is a more stable but occasionally hangs when it has to load many packages (hundreds of files) after compiler update (using SBCL primarily)
Aha. Well, it used to be that POIU could extract a dependency graph from an ASDF system, and act on it to build; but this has bitrotten a bit with the latest changes in ASDF 3.3, and POIU is not currently very stable. If you could fix it, that would be great. But beware, it's no small undertaking: you have to really get into a lot of internals to do it right, and make non-trivial changes to UIOP, likely including subtle changes to ASDF itself.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics•
Appropriate fear keeps you alive. Excessive fear prevents the very thing it's supposed to protect: life. — Attila Lendvai
Thanks,
Jay
https://gitlab.common-lisp.net/qitab/poiu/blob/master/README.md —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org On Thu, Jan 9, 2020 at 8:29 PM Jay <jeronwunalu@gmail.com> wrote:
Updated: Fare,
Sorry for late reply. Just got back from travels.
Thanks for the update. I did try to look for the updated README but didn't find it. I will try again to find the file.
As I said before, I am not an expert with ASDF, but I am trying to go over the coupling of my modules and review dependencies. Loading with SBCL exhibits weird behavior where it stops half (freezes) and start over again and it runs ok.
Rob also provided some additional info regarding extracting dependency information and I'll explore these in addition.
Thanks
On Thu, Dec 26, 2019 at 5:20 PM Faré <fahree@gmail.com> wrote:
I updated the POIU README with remarks based on our discussion so far. After thinking a bit more, I suspect it is not *that* hard to completely fix POIU and extract (and display) a precise ASDF dependency graph from it. But it's still some amount of serious work and a walk through the internals or ASDF and POIU.
NB: I'm talking about POIU, not UIOP. Even I got confused in my previous messages at times. Oops.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The idea is not to confront bad ideas but to come up with good ideas. Otherwise, your enemies define the game and you are the loyal opposition. — Terence McKenna
On Tue, Dec 24, 2019 at 12:17 PM Robert Goldman <rpgoldman@sift.info> wrote:
What kind of dependency graph are you looking to make? Just the systems, or also the individual system components?
And do you want a visible representation of this graph, or just some set of data structures?
I assume the former, since you should be able to use the system components themselves to get all the dependency links.
If it's the former, the easiest thing might be to use CL-DOT and write the necessary methods to create a graph.
You might need to get ASDF to compute the plan for LOAD-OP as a way to ensure that all the required system definitions are loaded.
With respect to test systems, I have usually found it best to have my systems, say "foo", have "foo/test" related systems, delegate the test-op for "foo" to "foo/test" using in-order-to, and then only "foo/test" needs to depend on the test library -- "foo" itself doesn't need it.
HtH,
R
On 23 Dec 2019, at 19:08, Jay wrote:
Thanks.
I will follow up with Rob later.
Anyway, thanks for help in the past. I will liaise with Rob to figure out the best way forward.
Jay
Faré 6:52 PM (14 minutes ago) to me
I guess, I will have to step up at some point. I have a tool that's built
with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above.
Then, I will be able to tackle the graph thing, as you said it's no easy
fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with
easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at
#lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Mon, Dec 23, 2019 at 6:52 PM Faré <fahree@gmail.com> wrote:
I guess, I will have to step up at some point. I have a tool that's
built with CL. I'll start small from the easier tasks and work my way up the graph one you highlighted above.
Then, I will be able to tackle the graph thing, as you said it's no easy
fit.
Makes sense.
Are the TODO you mentioned ordered in some priority list or tagged with
easier/medium/hard.
Unhappily not. Give it a look and/or give a look at the issues on launchpad, and/or ask Robert, for guidance. Sorry I don't have time to sort out the issues right now.
And the repo is still the version on gitlab right? Does Rob hang over at
#lisp irc?
Yes the repo is on gitlab, but the issues are still mostly on launchpad.
Robert sometimes is on IRC as rpg (or is is rpgoldman? notthatrpg?).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "Floating point numbers are like sandpiles; every time you move one you lose a little sand and pick up a little dirt" — Vic Vissotsky
On Fri, Nov 29, 2019 at 11:33 AM Faré <fahree@gmail.com> wrote:
I could not follow up in the past as I had to relocate between cities
(central to west coast) so that caused a lot of changes in my schedule. What is the state of the asdf contributors you mentioned last time -- the developers taking over asdf development albeit gradually.
No one has really stepped forward. It's still Robert Goldman maintaining things, while I have moved away from active maintenance and am only reviewing patches and giving advice, sometimes making small edits to documentation.
I just started working on my fairly large system again, and recently
had issues when I was trying to disentangle systems to avoid loading multiple test systems that clobber global variables. A secondary goal is to reduce unnecessary dependencies ( btw I use :class :package-inferred-system in my setup).
Is there some asdf option that I can use to get the graph for a given
loaded system or some score of complexity due to linkages. I have been able to improve this over the years and the system is a more stable but occasionally hangs when it has to load many packages (hundreds of files) after compiler update (using SBCL primarily)
Aha. Well, it used to be that POIU could extract a dependency graph from an ASDF system, and act on it to build; but this has bitrotten a bit with the latest changes in ASDF 3.3, and POIU is not currently very stable. If you could fix it, that would be great. But beware, it's no small undertaking: you have to really get into a lot of internals to do it right, and make non-trivial changes to UIOP, likely including subtle changes to ASDF itself.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics•
Appropriate fear keeps you alive. Excessive fear prevents the very thing it's supposed to protect: life. — Attila Lendvai
Thanks,
Jay
participants (3)
-
Faré -
Jay -
Robert Goldman