puri was pulled into my system by quicklisp as s dependency of drakma, and asdf was installed by portage. I have never written a line of lisp that called puri directly. No one is ever persuaded by the "it works for me" argument. The issue is real. Are you using a clean install of asdf, uiop and puri? Or are you using your development branch of asdf? Developers can always get it to work on their workstations. Maybe it's the gentoo package for asdf or uiop. Maybe slime is providing some sort of wrapping environment that affects readtables. There are any number of things that could be at play that are beyond the user's control.

Let's say you are right in saying that puri is doing something naughty by polluting a readtable. I'll go along with that. I have no reason to doubt it. I'm on your side on this issue. Nevertheless we have a problem: this code has been in the wild since 2015 or earlier. A very ubiquitous library (drakma) depends on it. Cleaning up poor behavior is a bigger job than making a fundamental and core component like asdf no longer accept the behavior. The offending libraries have to be fixed in concert with this refactoring. Established libraries cannot simply have the rug pulled out from under them.

I'm with Stas. I cannot upgrade asdf if it makes it impossible for me to use libraries that I depend on. Whatever new things asdf does are less important to me than having a working drakma. I believe just about all users would agree with this sentiment.

I'm perfectly willing to contribute to the fixing of these libraries. But let's not do it as a mad scramble. Let's issue deprecation warnings, identify broken quicklisp packages, get the bugs in those packages fixed and then implement the new asdf behavior. I'm here to help.

Carlos

2017-10-11 20:31 GMT-06:00 Faré fahree@gmail.com:

I just tried compiling puri with asdf 3.3.0, using either quicklisp or the latest commit (b537e93 from August 29 2015) on sbcl 1.3.20, and had no issue whatsoever.

Odds are the problem is on your side, probably with your using with-standard-io-syntax or some such. Note that it is very rude to use set-dispatch-macro-character on a readtable you didn't specifically setup, especially since it might be a read-only readtable, or worse, a writable one used by people who assume it won't be modified.

I've had a branch of ASDF, called "syntax-control", supposed to bring *some* sanity in readtables used while building with ASDF. It has bitrotten in the last 4 years and is nowhere near being merged at this point.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• https://urldefense.proofpoint.com/v2/url?u=http-3A__fare. tunes.org&d=DwIFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LR xpb6__0PomBTQ&r=UXttlWX_R8MsWhBAnFwO2t5aPuH2x7HGk4gZpT TCo-UE-RFLppFABNHDw29AmFnQ&m=sNgypPrTU-ocowWcZQAmsxdniF2mxBwl_ 6BmfJxiyjY&s=8O9az24Gmg12t9C9jp5bOiJSXjwPqkIvmeFuYqIUCnw&e= Opportunity is missed by most people because it comes dressed in overalls and looks like work. — T. A. Edison

On Wed, Oct 11, 2017 at 8:41 PM, Konstanski, Carlos carlos.konstanski@verizonwireless.com wrote:

...

This is a bug report. I'm using puri as an example, but it's not a puri bug.

With the latest version of asdf (3.3.0) puri no longer compiles. The following compiler error is thrown:

SET-DISPATCH-MACRO-CHARACTER would modify the standard readtable. [Condition of type ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR]

In:

0: (SET-DISPATCH-MACRO-CHARACTER ## #\u #<FUNCTION PURI::SHARP-U> #<READTABLE {100002D6C3}>)

The code:

(defun sharp-u (stream chr arg) (declare (ignore chr arg)) (let ((arg (read stream nil nil t))) (if *read-suppress* nil (if* (stringp arg) then (parse-uri arg) else (internal-reader-error stream "#u takes a string or list argument: ~s" arg)))))

(set-dispatch-macro-character ## #\u #'puri::sharp-u)

What puri is doing re: SET-DISPATCH-MACRO-CHARACTER is totally by the book. Why is this suddenly an error? Is there a workaround?

Backing up to the next most recent release of asdf makes the problem go away:

dev-lisp/asdf-3.2.1-r1:0/3.2.1-r1 dev-lisp/uiop-3.2.1:0

I tried to find the asdf changelog to see if this is a documented change. But the link is broken. https://urldefense.proofpoint.com/v2/url?u=https-3A__common-

2Dlisp.net_project_asdf_Changelog&d=DwIFaQ&c= udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=UXttlWX_ R8MsWhBAnFwO2t5aPuH2x7HGk4gZpTTCo-UE-RFLppFABNHDw29AmFnQ&m=sNgypPrTU- ocowWcZQAmsxdniF2mxBwl_6BmfJxiyjY&s=-Ety3cawXfSBOqWK_ Ms5Hi2TwB62089buk66HVbLu-s&e=

...

uiop seems to be closely tied to asdf. Not sure which package is actually at fault. -- Carlos Konstanski

I'm with Faré on this one. I don't see evidence that this change is because ASDF is doing something bad. I believe it's consistent with the hypothesis that there was some imperfectly-controlled aspect of building that is done differently now (e.g., files loaded in a different order where both orders are compatible with the constraints in the system definitions).

This doesn't even look like an ASDF error to me -- it looks like an error that occurred while loading a system that ASDF has re-packaged.

So we might be able to help you debug this, but without more evidence, there's no reason to believe that ASDF has done anything to you: it looks like the change in ASDF just reveals a pre-existing bug.

On 12 Oct 2017, at 12:09, 73budden . wrote:

This tracing tool should help a lot.

I believe this tool should be supplied by asdf team. Even I begin to be more positive towards efforts of ASDF team to clean up all the mess that was in ASDF initially, but obviously society is not quite happy with breaking changes, so some small tool with a good manual would make life easier.

This will probably not happen for a while -- my work on other aspects, and just staying on top of bugs and testing takes most of the available time.

But I think I can provide pieces of a recipe for this kind of debugging and if someone out there could give feedback, I will see to it that the recipe gets into the manual.

I think if you want to see the plan that ASDF produces to perform a requested operation, you should use something like: ``` (setf *plan* (asdf/plan:make-plan nil (make-operation 'load-op) (find-system "sysname"))) ```

Then, to figure out what's happening, I would suggest ``` (trace asdf:operation-done-p) ``` ...to see if ASDF is wrong about whether or not it needs to do some operations.

Then I would try something like tracing `PERFORM`.

I'd have to think a little about what to do if `MAKE-PLAN` gives you a plan you don't expect.

cheers, r

Printing readtable before loading, I think, requires just a line or two. Dumping log of operations might be one (trace) call, so that's trivial for the person who knows how ASDF is organized. Writing a small two-paragraph addition to manual would relief a lot of pain and stress for all.

On Thu, Oct 12, 2017 at 5:09 PM, 73budden . budden73@gmail.com wrote:

Warning about modifying standard readtable is issued by SBCL (at least in my SBCL 1.3.18), please grep for the message in SBCL sources, and it seem to be introduced in 1.0.24. I took a look at puri's definition (maybe old, in my local copy of quicklisp) and it looks like it actually tries to modify standard readtable.

More precisely, puri modifies the _current_ *readtable*, which is very rude. If anyone maintains puri (fat luck), he should remove this bug and instead offer an interface using named-readtables.

The error happens when the current readtable happens to be the standard readtable, which will happen if someone uses with-standard-io-syntax.

Looking at ASDF sources, there is one place that uses it, and indeed, a bug was introduced during the refactoring of ASDF 3.3.0: whereas earlier variants of ASDF saved the *readtable* from outside the with-standard-io-syntax to restore it inside, the new ASDF fails to do it. Reduced test case: try to (asdf:load-system "ddop") with the following ddop.asd.

(defsystem "ddop" :defsystem-depends-on ("puri"))

I believe this justifies a 3.3.1 bugfix release indeed.

It also justifies spending more time on the syntax-control branch supposed to cleanup the readtable issues, and/or getting all software fixed so it never modifies the current readtable unless it explicitly makes and sets a new readtable.

It'd be very nice to have instructions on how to trace what happens while system is being built:that is, which files are compiled, which are loaded and what is a readtable in the beginning of each load operation.

You can pass a :verbose t flag to load-system and its friends.

As for tracing what readtable are used, there does not exist infrastructure for even identifying readtables. You're welcome to write one and seek assistance in writing a hook in ASDF (e.g. as a :before method on perform compile-op or load-op, which would not have helped you much in this case, or as a local modification to uiop:compile-file*, if you have a local ASDF source checkout).

Running builds in old and new asdf versions and comparing logs it would be relatively easy to figure out what is wrong.

Only if given enough information to reproduce, which the complainer failed to provide. Luckily, I investigated nevertheless.

For those in a hurry for a fix, here is the merge request: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The two most common things in the Universe are hydrogen and stupidity. — Harlan Ellison

Wow! Good news.

2017-10-12 21:08 GMT+03:00, Robert Goldman rpgoldman@sift.info:

On 12 Oct 2017, at 13:04, Faré wrote:

...

For those in a hurry for a fix, here is the merge request: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/85

I will merge this into master, but do not have time to make a new release, or even test much this week or next.

So this will simply remain in master for now.

On Thu, Oct 12, 2017 at 6:11 PM, Robert Goldman rpgoldman@sift.info wrote:

...

I will merge this into master, but do not have time to make a new release, or even test much this week or next.

So this will simply remain in master for now.

This patch is now available in ASDF 3.3.0.1

I also pushed a regression test directly to master.

I will also try to salvage the most important stuff to salvage from the syntax-control branch, namely some *shared-readtable* and *shared-print-pprint-dispatch* tables in asdf (or uiop?) to restore around the build (e.g. in asdf:operate) so it is well-defined what readtable is used when building with asdf (i.e. the readtable active when asdf was initially loaded, usually your Lisp's initial readtable).

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org [T]he “evolution stops at the neck” reflex. They're willing to accept that everything else about humans has evolved through evolution but the thing that is most important in explaining your personhood, namely your mind, somehow evolution doesn’t apply to it. — Gad Saad