[asdf-devel] bad interaction with new ASDF 2.015 and Quicklisp
Steps to reproduce: 1. Get ASDF 2.015 or .1 2. Install Quicklisp, (ql:quickload "flexi-streams") 3. Install Flexi-Streams outside quicklisp as well. (For the purposes of bug reproduction, putting a copy of flexi-streams.asd into eg ~/tmp is enough.) 4. Go to the directory with the non-Quicklisp copy of flexi-streams.asd. 5. In lisp: (push "./" asdf:*central-registry*) (asdf:load-system "flexi-streams") ...endless recursion that eventually blows either stack or heap. What is happening is approximately this: ASDF starts loading ~/tmp/flexi-streams.asd DEFSYSTEM FLEXI-STREAMS is ok. During processing of DEFYSTEM FLEXI-STREAMS-TEST, FIND-SYSTEM is called for FLEXI-STREAM-TEST -- ironically _after_ it has been registered and its pathname set. Source-registry doesn't find it because there is no file flexi-stream-test.asd, and neither does SYSTEM-FIND-ASDF, but Quicklisp which _does_ find it: ASDF starts loading quicklisp/.../flexi-streams.asd ...you can guess where this leads. But even without the endless recursion this find is bogus. This did not happen with prior versions of ASDF. This is really messing up things for me -- enough so that I'm contemplating reverting the SBCL provided ASDF to 2.014. Cheers, -- Nikodemus
On 10 May 2011 12:45, Nikodemus Siivola <nikodemus@random-state.net> wrote:
Steps to reproduce:
1. Get ASDF 2.015 or .1
2. Install Quicklisp, (ql:quickload "flexi-streams")
3. Install Flexi-Streams outside quicklisp as well. (For the purposes of bug reproduction, putting a copy of flexi-streams.asd into eg ~/tmp is enough.)
4. Go to the directory with the non-Quicklisp copy of flexi-streams.asd.
5. In lisp:
(push "./" asdf:*central-registry*)
(asdf:load-system "flexi-streams")
...endless recursion that eventually blows either stack or heap.
What is happening is approximately this:
ASDF starts loading ~/tmp/flexi-streams.asd
DEFSYSTEM FLEXI-STREAMS is ok.
During processing of DEFYSTEM FLEXI-STREAMS-TEST, FIND-SYSTEM is called for FLEXI-STREAM-TEST -- ironically _after_ it has been registered and its pathname set.
Source-registry doesn't find it because there is no file flexi-stream-test.asd, and neither does SYSTEM-FIND-ASDF, but Quicklisp which _does_ find it:
ASDF starts loading quicklisp/.../flexi-streams.asd
...you can guess where this leads. But even without the endless recursion this find is bogus.
This did not happen with prior versions of ASDF. This is really messing up things for me -- enough so that I'm contemplating reverting the SBCL provided ASDF to 2.014.
Ouch. Does 2.014 fix this? If it does, that makes sense indeed. I'll have to go back to the drawing board and maybe change the way find-system works, to make it more robust. [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] You can tell whether a man is clever by his answers. You can tell whether a man is wise by his questions. — Naguib Mahfouz
Nikodemus Siivola <nikodemus@random-state.net> writes:
What is happening is approximately this:
ASDF starts loading ~/tmp/flexi-streams.asd
DEFSYSTEM FLEXI-STREAMS is ok.
During processing of DEFYSTEM FLEXI-STREAMS-TEST, FIND-SYSTEM is called for FLEXI-STREAM-TEST -- ironically _after_ it has been registered and its pathname set.
Source-registry doesn't find it because there is no file flexi-stream-test.asd, and neither does SYSTEM-FIND-ASDF, but Quicklisp which _does_ find it:
ASDF starts loading quicklisp/.../flexi-streams.asd
...you can guess where this leads. But even without the endless recursion this find is bogus.
Thanks for the analysis. I'll update the Quicklisp system definition search function to only return a pathname if the pathname-name matches the name given to asdf:find-system. I hope to publish a client update sometime soon, maybe this week. Zach
Thanks for the analysis. I'll update the Quicklisp system definition search function to only return a pathname if the pathname-name matches the name given to asdf:find-system.
I hope to publish a client update sometime soon, maybe this week.
Zach
Maybe I could also have ASDF remember when it's inside load-sysdef and/or defsystem, and specially treat the find-system for a defsystem inside it? The second case promises to be tricky. I think the control flow from defsystem to find-system to load-sysdef is especially nasty, and I appreciate designs that would avoid (infinite) recursion. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The reason truth is stranger than fiction is that fiction has to make sense.
Those who run into this and need to get work done with devhead SBCL can in the meanwhile stick this in their .sbclrc after the #-quicklisp stanza. (The effect of the code is that REQUIRE and ASDF:LOAD-SYSTEM won't pull in quicklisp systems -- you need to use QL:QUICKLOAD to load things with dependencies satisfied by Quicklisp. It's ugly, but it does the job.) #+quicklisp (progn (setf asdf:*system-definition-search-functions* (remove 'ql-dist::find-asdf-system-file asdf:*system-definition-search-functions*)) (defvar *quickload* #'ql:quickload) (handler-bind ((style-warning #'muffle-warning)) (defun ql:quickload (&rest args) (let ((asdf:*system-definition-search-functions* (append asdf:*system-definition-search-functions* (list 'ql-dist::find-asdf-system-file)))) (apply *quickload* args))))) Cheers, -- Nikodemus
Nikodemus Siivola <nikodemus.siivola@gmail.com> writes:
Those who run into this and need to get work done with devhead SBCL can in the meanwhile stick this in their .sbclrc after the #-quicklisp stanza.
I think this should work as well - can you try it and let me know? (setf asdf:*system-definition-search-functions* (substitute (lambda (name) (let ((file (ql-dist:find-asdf-system-file name))) (when (and file (equal (pathname-name file) name)) file))) 'ql-dist:find-asdf-system-file asdf:*system-definition-search-functions*)) Thanks, Zach
OK, ASDF 2.015.2 should now be clever enough to avoid any such infinite loop. If you can reproduce the previously bad setup with the new ASDF and check that it's now working, that'd be great. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org He who refuses to do arithmetic is doomed to talk nonsense. — John McCarthy, in his webpage on Progress and Sustainability
On 11 May 2011 19:56, Faré <fahree@gmail.com> wrote:
OK, ASDF 2.015.2 should now be clever enough to avoid any such infinite loop. If you can reproduce the previously bad setup with the new ASDF and check that it's now working, that'd be great.
Works for my test-case. I've updated SBCL with 2.015.2. Cheers, -- nikodemus
On 11 May 2011 21:47, Nikodemus Siivola <nikodemus.siivola@gmail.com> wrote:
On 11 May 2011 19:56, Faré <fahree@gmail.com> wrote:
OK, ASDF 2.015.2 should now be clever enough to avoid any such infinite loop. If you can reproduce the previously bad setup with the new ASDF and check that it's now working, that'd be great.
Works for my test-case. I've updated SBCL with 2.015.2.
Ok, I don't know why I didn't notice this before... but it's not over yet. There is another -- very similar -- issue that causes trouble if there are EQL method specialized on the system in the .asd file for the system in question ... not exactly a rarity. Try with ITERATE. It recurses endlessly with 2.015.2 between local installation and Quicklisp one. The proximate casue is (FIND-SYSTEM :iterate): when the non-quicklisp .asd is processed, those methods find the Quicklisp definitions, and vice versa. Cheers, -- Nikodemus
Ok, I don't know why I didn't notice this before... but it's not over yet.
Ouch.
There is another -- very similar -- issue that causes trouble if there are EQL method specialized on the system in the .asd file for the system in question ... not exactly a rarity.
Try with ITERATE. It recurses endlessly with 2.015.2 between local installation and Quicklisp one.
The proximate casue is (FIND-SYSTEM :iterate): when the non-quicklisp .asd is processed, those methods find the Quicklisp definitions, and vice versa.
OK. In ASDF 2.015.3, I've built some more robust plumbing to prevent infinite recursion in finding systems. Guaranteed to not recurse infinitely while searching for systems if your systems are loaded through find-system (or its internal load-sysdef) or operate (and its wrappers oos, load-system, compile-system). Can you test again? Someone contributing a test case for ASDF would be nice. I'm too lazy/busy to do it right yet. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Quis custodiet ipsos custodes? (Who shall watch the watchmen themselves?) — Juvenal, Satires, VI, 347
On 14 May 2011 02:29, Faré <fahree@gmail.com> wrote:
Can you test again? Someone contributing a test case for ASDF would be nice.
Worksforme. Updated in SBCL. I'll see about a test-case next week. Cheers, -- Nikodemus
participants (4)
-
Faré -
Nikodemus Siivola -
Nikodemus Siivola -
Zach Beane