The problem with central-registry is a bootstrap problem. It requires everyone to be a system administrator and edit configuration files.
I don't follow this claim. All central registry requires is that you run some lisp code, generally in your lisp-init file.
Yes, and that's what I mean by "system administrator": you need to write code in configuration files. Something that end-users shouldn't have to do.
With ASDF 3.1's source-registry, end-users don't have to edit anything, and it will just work: just put your code under ~/common-lisp/ — or they can add one line worth of configuration per non-default source location they want to use.
I wanted to make ASDF suitable for writing code deployed to end-users. Now it is.
We have one piece of lisp code that does a simple depth-first search and jams things into ASDF:*CENTRAL-REGISTRY*. People stuff that into their lisp-init files and that's it.
Writing that code is nothing for an expert like you, but a huge hurdle for novices. Copy/pasting it without understanding it is also full of dangers.
You are right, of course, that we can do that with INITIALIZE-SOURCE-REGISTRY, too, but I don't understand your claims about needing root privilege or needing to understand the bootstrap environment any more than just understanding how to write a lisp-init file.
It's not about root privilege, it's about being a power user who has deep understanding of configuration and your Lisp application's bootstrap process.
You may indeed need to re-run (asdf:initialize-source-registry) after you install or remove .asd files. That is documented, and a small price to pay for scaling the registry much beyond what the *central-registry* will crumble with.
It's a small price to pay IF YOU NEED TO SCALE THE REGISTRY BEYOND WHAT *CENTRAL-REGISTRY* WILL CRUMBLE WITH. Otherwise, it's an annoyance created by an optimization you don't need.
The largest project I'm working on now has 212 entries in ASDF:*CENTRAL-REGISTRY*. Could be zippier, but that certainly doesn't cause the list representation to crumble.
I get it that the new structure scales better, but it is a tradeoff, and some users will find that it's not a tradeoff that works for them.
Having 212 entries is one to two order of magnitude more than most people need.
I consider my home installation advanced, and it has all of 6 lines of configuration, counting lines for each of the :source-registry tag and the :inherit-configuration directive.
For QRes, I had the makefile export CL_SOURCE_REGISTRY with a single entry, and that was it.
A novice does not need to understand it. If you follow the instructions, such as "install your software under ~/common-lisp/", then it Just Works™.
That doesn't work if you work on multiple different systems that have different source repositories. I typically work on at least three different CL projects at once, which means a single configuration is a non-starter for me.
The easiest way is to export CL_SOURCE_REGISTRY from the shell, and/or equivalently have each project's Lisp init script use asdf:initialize-source-registry with a parameter. Unlike loading asdf from .sbclrc, that works without polluting a shared file for other environments.
Experts can dump tables. I hesitated about replicating alexandria:hash-table-alist in uiop, to make it easy to dump some tables. Decided against it in the end, but if you believe novices should be able to dump those tables, that's something to consider.
Yes, I do believe that novices should be able to dump those tables, because I have seen novices make mistakes (e.g., having both lisp code configuration and a .conf file configuration -- the former being added after the latter was forgotten), and just be completely confused and lost.
I recently helped Raymond Toy debug an issue he had with ASDF, and for diagnosing his problem, I asked him to tell me what that form returned:
(list asdf:*central-registry* (uiop:getenv "CL_SOURCE_REGISTRY") asdf::*source-registry-parameter* (mapcar (lambda (x) (list x (uiop:read-file-string x))) (mapcan 'uiop:directory-files (list (uiop:subpathname (user-homedir-pathname) ".config/common-lisp/source-registry.conf") (uiop:merge-pathnames* ".config/common-lisp/source-registry.conf.d/" (user-homedir-pathname))))) (loop :for a :in (asdf/source-registry::flatten-source-registry) :when (and (getf (cdr a) :recurse) (uiop:probe-file* (car a))) :append (uiop:while-collecting (c) (uiop:collect-sub*directories (car a) (lambda (d) (uiop:if-let ((f (uiop:probe-file* (uiop:subpathname d ".cl-source-registry.cache")))) (c f)) t) (constantly t) (constantly t)))))
[It turns out his problem was that he was trying to manually load .asd files that were not in any of his registries, and expecting it to work even though they themselves called other unregistered .asd files via :defsystem-depends-on. Don't do that.]
But in addition to dumping the tables, everyone must be able to figure out where the table entries came from. That's why something like tracing is so important. Again, if anyone, including a novice, gets their configuration messed up -- and, indeed, novices are very likely to, because they often just cargo-cult configs from others -- they need to be able to see where that configuration comes from and how to fix it.
I agree that the source-registry is less easy to debug than could be. Yet, I have never been contacted for misuse of it, but I have been contacted a lot for misuse of *central-registry* (though it has gotten much better since the missing trailing / became detected by asdf itself).
The current "there's more than one way to do it" configuration process means that this is especially important, because we can no longer simply look in our lisp initialization to figure out why things have gone up the spout. Instead, we have to check hidden configuration directories, environment variables, lisp code, etc.
That is true. And it gets only more complicated when Quicklisp introduces its own extensions. Yet in the end, that was the true power of ASDF's extensible system search design (thanks, Dan Barlow!): let people define their own system search facilities, and see which grows and survives.
That code is not the most readable, but what it does it well documented, well debugged, and it's easy to look at the result, change something and look at the result again.
Only if you know where the configuration was that needs to be changed. I have definitely seen people who have configurations slopped all over the place, some forgotten, and had to try to help them debug.
We probably can provide better diagnostic tools. At the same time, if you grow complex configurations, you're probably doing it wrong.
Finally, if you configure with configuration files in your XDG config directories, there's not even a chance to trace, because by the time you could set up a trace, the configuration's been loaded.
It doesn't actually make tracing harder than usual. Tracing asdf configuration is a normally hard bootstrap issue.
Not if the configuration is in your lisp code it isn't. Only when we added the configuration files.
OK, so it makes things slightly harder for the expert who uses SLIME, but much easier for the novice who doesn't use it yet. I believe this is a vast net win.
On the other hand, the source-registry often "just works" out of the box, or requires a one-liner to configure. The *central-registry* is cognitively much heavier, especially so on novices.
My philosophy is to assume that code will always break and then consider how people can recover from that. So yes, I agree, ASDF now typically just works, but we haven't left users with much support when it doesn't.
I agree we can offer better debugging support.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Truth comes as conqueror only to those who have lost the art of receiving it as friend. — Tagore