(defun upgrade-asdf ()
    "Try to upgrade of ASDF. If a different version was used, return T.
   We need do that before we operate on anything that may possibly depend on ASDF."
    (let ((*load-print* nil)
          (*compile-print* nil))
      (handler-bind (((or style-warning) #'muffle-warning))
        (symbol-call :asdf :load-system :asdf :verbose nil))))

Can I upgrade ASDF by simply placing the built asdf.lisp file in ~/common-lisp/asdf/asdf.lisp and omitting everything else? Will ASDF find it?

On Tue, Jan 19, 2021 at 9:44 AM Robert Goldman <rpgoldman@sift.info> wrote:

Warning -- opinionated rant follows!

On 18 Jan 2021, at 20:50, Eric Timmons wrote:

That's not quite right. It could definitely be more friendly, but there
are a few ways to better control it.

To completely prevent ~/common-lisp/ from being traversed you could put
an :ignore-inherited-configuration directive somewhere in the
CL_SOURCE_REGISTRY envvar or
$XDG_CONFIG_DIRS/common-lisp/source-registry.conf. But that approach
also would prevent the files in
$XDG_CONFIG_DIRS/common-lisp/source-registry.conf.d from being parsed
(as well as any system level config). It might be nice to allow an
inheritance config directive to be specified in the configuration
directory parsing if it isn't already (there's an implicit
:inherit-configuration tacked on the end of the directory based config).

Another option is to drop a .cl-source-registry.cache file in
~/common-lisp/ or one of the sub directories.
https://common-lisp.net/project/asdf/asdf.html#Caching-Results. ASDF
will stop recursing if it finds that file and just use the info it
contains.

This is true, but it causes just the kind of problems I have alluded to -- your ASDF configuration is now smeared all over your system, and debugging it becomes essentially impossible.

I have repeatedly had to help people where I work who have put one of these magical -- and invisible -- files or configurations somewhere, forgotten it, and then don't understand why some aspect of their configuration is misbehaving. Now I tell people only to configure things in their lisp init files, not to use magical directories, and not to use environment variables. Then if something goes wrong, you know where to look for the culprit.

Note also that if ASDF isn't configured in your lisp init file, but by a config file or environment variable, it will be configured as it loads. This means that all of your debugging tools are taken away from you. There will be no tracing, because by the time you can set up a trace, the damage will be done.

asdf:*central-registry* is terribly inefficient, but it is simple and it can be inspected when things go wrong. None of these other schemes share that feature. I have tried to trace the control flow for interpreting the configuration DSL and it's a mass of twisty passages all exactly alike. Lots of key functionality is in variables, or in anonymous lambdas, making tracing effectively impossible. (Adding configuration logging would be a big help)

So -- if you are at Google or some other shop where you have a zillion systems coming together, yes, *central-registry* is too slow, and will kill you. For most of us, though, using one of the alternatives is premature optimization.

I am a strong believer that the only way to keep track of the burgeoning complexity of today's systems is to localize and simplify configuration, rather than disperse it. For some reason, the dispersion faction is winning the design game, though. I'm not sure why -- perhaps there's some notion of tidiness and elegance that encourages all of these configuration layering (look how elegant! You can override things or accept the default!) and dispersion (there's an individual config file for every purpose, instead of one monster -- and I admit my .emacs is a thing to strike fear in the heart).

Also not a fan of invisible config files. Yeah -- it's ok to have dot files in your home dir so that you don't get overwhelmed, but why is it a good idea to hide the file in your repo that configures the CI so that ls won't show it to you?

Finally, everything will break, so make sure you know how the poor user will debug it when it does. Make sure things are traceable in CL. Log things. Make your error messages understandable. Lots of ASDF doesn't follow this principle, and I would love to move towards making it easier to diagnose and debug.

--

Wilfredo Velázquez-Rodríguez