"Robert Goldman" rpgoldman@sift.info writes:
This is true, but it causes just the kind of problems I have alluded to -- your ASDF configuration is now smeared all over your system, and debugging it becomes essentially impossible.
You're right that understanding what the source registry is doing is extremely difficult right now. (And I also find the twisty turny passages of the code difficult to follow). I started https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/156 to try and make it better.
The source registry computation works in two phases: the DSL is flattened into a list of directories to search, and then the directories are actually searched. There's a ton more info on the first commit to !156, but now provenance info is recorded in the flattening step. Recording it in the second step should also be trivial once we decide where to record it (*source-registry* or another variable).
Also not a fan of invisible config files. Yeah -- it's ok to have dot files in your home dir so that you don't get overwhelmed, but why is it a good idea to hide the file in your repo that configures the CI so that `ls` won't show it to you?
I personally lean toward hiding the files that are kind of background infrastructure and won't be edited all that frequently. But I also now not everyone does (obviously!). Perhaps one thing we can do is change ASDF to use cl-source-registry.cache if it exists, falling back to .cl-source-registry.cache if it doesn't? (Also mentioned this on !156)
Note also that if ASDF isn't configured in your lisp init file, but by a config file or environment variable, *it will be configured as it loads*. This means that all of your debugging tools are taken away from you. There will be no tracing, because by the time you can set up a trace, the damage will be done.
I don't think this is true. I thought ASDF configured the source registry on the first invocation of find-system? So you can set up traces before that happens. But that may not be useful since the first thing most people do when starting Lisp is load swank which ends up initializing ASDF (on SBCL at least) through a REQUIRE.
If you wanted to trace or otherwise debug a catastrophically broken source registry configuration using SLIME, I think adding a (asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration)) to your init file would let you get SLIME up and running. Then you can trace the config like normal by evaluating (asdf:initialize-source-registry nil)
And now for my turn on the soap box:
While configuring only through an init file may help localize the configuration from the perspective of a single developer, I think the system level fallbacks are useful to localize the configuration from the perspective of an integrator for a loosely coupled team. I manage multiple development Docker images for my group and have routinely used the system level source registry config to ensure that some of our development tools are available. This is nice because I don't have to badger N people to update their init files if I need to tweak something and they can mount their init files directly from their Docker host into multiple containers (with varying tools prebundled in each) and know that things will Just Work.
On the other end of the spectrum, I've found a use for CL_SOURCE_REGISTRY that I use to hyper-localize my configuration. Unless you jump through hoops, your Lisp init file is generally user wide config. I've been developing CLPM https://www.clpm.dev that configures ASDF on a per-project basis. One of the features it provides is a Ruby Bundler-like exec feature where you can run a command (say in CI or your terminal) and know that any instance of ASDF in that environment will see only the systems you've specified for that project[1]. CL_SOURCE_REGISTRY is the only way I can think of doing this that works seamlessly across all Lisps no matter how they're invoked.
-Eric
[1] plus the systems that are forced upon you by WRAPPING-SOURCE-REGISTRY.