Thanks all for the rest of your comments. Interesting notes about Clojure, Hans.

I agree that this problem is not unique to Lisp, but what is relatively unique is that it's a quiet problem. (Mismatching of transitive dependencies can go unnoticed until something eventually breaks.)

That's a problem with all languages that don't do static types. Node.js, Ruby, Python are all afflicted by this issue. Even with static types, that doesn't solve other types of problems, like a dependency that switched from O(n^2) to O(n*log(n)) implementation in a method, that exposed a race condition in the user code and caused the server to crash in production during the Black Friday peak.

The only question in response to many of your relatively-in-agreement-with-each-other comments is: do you think the relevant portions of Lisp, the relevant portions of the Lisp tool chain (ASDF, QL), and the way in which Lisp seems to be popularly used in open source are as a whole close to optimal in what we can do in 2016 to even detect, let alone address, these kinds of issues? Or is manual per-project vetting and curation of libraries the best possible?

There are ways, one of which would be a very strict build system like Bazel. That has several problems of its own, though. As somebody noticed in a blog post about the latest ELS, Bazel has a large overhead and is cumbersome for small projects.

On Thursday, May 19, 2016, Stelian Ionescu <sionescu@cddr.org> wrote:

I don't know much about Bazel, and I know a little about NixOS. Regardless, this seems to be moving the problem into how we globally synchronize our systems.

I am absolutely boggled by this attitude. This is an issue that one runs into when writing Common Lisp code, and not an issue that one runs into writing in another language and associated ecosystem. To me, that's a Lisp problem. We can do some creative academic definitions, I think, but it's a problem when choosing Lisp as a tool.

It's not a Lisp-only problem, it's a general boolean satisfiability problem that all languages have. The only way to deal with this is to ensure that your entire ecosystem works with the same set of dependencies, to check-in those dependencies in your repository and not rely on "semantic versioning" or other such illusions, run the tests(do actual QA) and be very conservative with upgrades.

Having a source control system that can keep everything in one repository, with partial checkouts, is very useful in achieving that. I've started to really appreciate Perforce in the last year or so.

At my previous company, we had so many problems with Node.js(good for prototyping) because of the fact that npm(the package manager) downloaded private copies of the libraries, that in the end I think they rewrote the server to something more sensible, Java.

--

Stelian Ionescu a.k.a. fe[nl]ix

Quidquid latine dictum sit, altum videtur.

Stelian Ionescu a.k.a. fe[nl]ix

Quidquid latine dictum sit, altum videtur.