This appears to happen both on 1.369 and whatever is shipped with fresh SBCL.
Prerequisites: ensure a setup where both MCCLIM and CL-UTILITIES are loadable after ASDF is loaded.
root@auriga:/home/deepfire/source# sbcl --eval '(require :asdf)' --eval '(asdf:find-system :mcclim)' This is SBCL 1.0.33.2, an implementation of ANSI Common Lisp. More information about SBCL is available at http://www.sbcl.org/.
SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. * (defun asdf-hidden-system-names (name) "Find out names of ASDF systems hiding in system going by NAME. A hidden system is a system with a definition residing in a file named differently from that system's name. Warning: demo-only, trimmed-down version." (let ((asdf::*defined-systems* (make-hash-table :test 'equalp))) (handler-case (let ((*break-on-signals* nil)) (format t "about to find system ~A~%" name) (let ((system (asdf:find-system name))) (format t "finding system ~A: ~A, ~S~%" name system (asdf:system-definition-pathname system)))) (error (c) (format t "~@<; ~@;WARNING: error while querying ASDF about hidden names of system ~S: ~A~:@>~%" name c))) (hash-table-count asdf::*defined-systems*)))
ASDF-HIDDEN-SYSTEM-NAMES * (asdf-hidden-system-names :cl-utilities) about to find system CL-UTILITIES ; loading system definition from /root/.sbcl/systems/esa-mcclim.asd into ; #<PACKAGE "ASDF1"> ; registering #<SYSTEM :CLIM-LISP {1003773701}> as CLIM-LISP ; registering #<SYSTEM :CLIM-BASIC {10037D4331}> as CLIM-BASIC ; registering #<SYSTEM :GOATEE-CORE {10038425B1}> as GOATEE-CORE ; registering #<SYSTEM :CLIM-POSTSCRIPT {10038A3601}> as CLIM-POSTSCRIPT ; registering #<SYSTEM :CLIM-CORE {1003904611}> as CLIM-CORE ; registering #<SYSTEM :ESA-MCCLIM {100396A641}> as ESA-MCCLIM ; loading system definition from /root/.sbcl/systems/swank.asd into ; #<PACKAGE "ASDF2"> ; registering #<SYSTEM :SWANK {1003A6F6F1}> as SWANK ; registering #<SYSTEM :DREI-MCCLIM {1002F89911}> as DREI-MCCLIM ; registering #<SYSTEM :DREI-TESTS {1003004691}> as DREI-TESTS ; registering #<SYSTEM :CLIM {100306A641}> as CLIM ; registering #<SYSTEM :CLIM-CLX {10030C5591}> as CLIM-CLX ; registering #<SYSTEM :CLIM-BEAGLE {1003129621}> as CLIM-BEAGLE ; registering #<SYSTEM :CLIM-NULL {100319F5A1}> as CLIM-NULL ; registering #<SYSTEM :CLIM-GTKAIRO {10031FD5B1}> as CLIM-GTKAIRO ; registering #<SYSTEM :CLIM-GRAPHIC-FORMS {1003261AE1}> as CLIM-GRAPHIC-FORMS ; registering #<SYSTEM :CLIM-OPENGL {10032C6631}> as CLIM-OPENGL ; registering #<SYSTEM :CLIM-LOOKS {1003322B11}> as CLIM-LOOKS ; registering #<SYSTEM :MCCLIM {1003388291}> as MCCLIM finding system CL-UTILITIES: #<SYSTEM "cl-utilities" {1002C5F2F1}>, #P"/root/.sbcl/systems/cl-utilities.asd" 20 *
What happens here is that cl-utilities.asd imperatively requires sb-rotate-byte from within its supposedly declarative sysdef, loading of which, through some inexplicable twist of events, causes lookups of seemingly unrelated software.
A backtrace which reveals somewhat more of what's going on:
1: (FIND-SYSTEM "esa-mcclim" NIL) 2: ((SB-PCL::FAST-METHOD FIND-COMPONENT ((EQL NIL) T)) #<unused argument> #<unused argument> #<unused argument> "esa-mcclim" NIL) 3: (ASDF::PARSE-COMPONENT-FORM NIL (:MODULE "esa-mcclim" :PATHNAME #P"/little/home/empty/desr/git/mcclim/" :DEPENDS-ON (:CLIM-CORE) :COMPONENTS ((:MODULE "ESA" :COMPONENTS ((:FILE "packages") (:FILE "utils" :DEPENDS-ON #) (:FILE "colors" :DEPENDS-ON #) (:FILE "esa" :DEPENDS-ON #) (:FILE "esa-buffer" :DEPENDS-ON #) (:FILE "esa-io" :DEPENDS-ON #) (:FILE "esa-command-parser" :DEPENDS-ON #)))))) 4: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ASDF::PARSE-COMPONENT-FORM NIL (APPLY #'LIST :MODULE (ASDF::COERCE-NAME ':ESA-MCCLIM) :PATHNAME #P"/little/home/empty/desr/git/mcclim/" '(:DEPENDS-ON (:CLIM-CORE) :COMPONENTS ((:MODULE "ESA" :COMPONENTS #))))) #<NULL-LEXENV>) 5: (SB-INT:SIMPLE-EVAL-IN-LEXENV (DEFSYSTEM :ESA-MCCLIM :DEPENDS-ON (:CLIM-CORE) :COMPONENTS ((:MODULE "ESA" :COMPONENTS ((:FILE "packages") (:FILE "utils" :DEPENDS-ON #) (:FILE "colors" :DEPENDS-ON #) (:FILE "esa" :DEPENDS-ON #) (:FILE "esa-buffer" :DEPENDS-ON #) (:FILE "esa-io" :DEPENDS-ON #) (:FILE "esa-command-parser" :DEPENDS-ON #))))) #<NULL-LEXENV>) 6: (SB-FASL::LOAD-AS-SOURCE #<SB-SYS:FD-STREAM for "file /little/home/empty/desr/git/mcclim/mcclim.asd" {1002FABE21}> NIL NIL) 7: ((FLET SB-FASL::LOAD-STREAM) #<SB-SYS:FD-STREAM for "file /little/home/empty/desr/git/mcclim/mcclim.asd" {1002FABE21}> NIL) 8: (LOAD #P"/home/empty/desr/git/.asdf-registry/esa-mcclim.asd")[:EXTERNAL] 9: (FIND-SYSTEM :ESA-MCCLIM T) 10: ((SB-PCL::FAST-METHOD PERFORM :AROUND (T T)) #<unavailable argument> #S(SB-PCL::FAST-METHOD-CALL :FUNCTION #<CLOSURE # {1002F99259}> :PV NIL :NEXT-METHOD-CALL NIL :ARG-INFO (2)) #<LOAD-OP NIL {1002F6F791}> #<CL-SOURCE-FILE "package" {1002EC6341}>) 11: ((SB-PCL::FAST-METHOD PERFORM AROUND (LOAD-OP CL-SOURCE-FILE)) #<unavailable argument> #S(SB-PCL::FAST-METHOD-CALL :FUNCTION #<FUNCTION #> :PV NIL :NEXT-METHOD-CALL #S(SB-PCL::FAST-METHOD-CALL :FUNCTION # :PV NIL :NEXT-METHOD-CALL NIL :ARG-INFO (2)) :ARG-INFO (2)) #<LOAD-OP NIL {1002F6F791}> #<CL-SOURCE-FILE "package" {1002EC6341}>) 12: ((LAMBDA ())) 13: ((FLET SB-THREAD::WITH-RECURSIVE-LOCK-THUNK)) 14: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-RECURSIVE-LOCK]291)) 15: (SB-THREAD::CALL-WITH-RECURSIVE-LOCK #<CLOSURE (FLET SB-THREAD::WITH-RECURSIVE-LOCK-THUNK) {7FFFF519F9D9}> #S(SB-THREAD:MUTEX :NAME "World Lock" :%OWNER #<SB-THREAD:THREAD "initial thread" RUNNING {10029D7091}> :STATE 1)) 16: (SB-C::%WITH-COMPILATION-UNIT #<CLOSURE (LAMBDA #) {1002F97EB9}>)[:EXTERNAL] 17: (OPERATE LOAD-OP :SB-ROTATE-BYTE)[:EXTERNAL] 18: (ASDF::MODULE-PROVIDE-ASDF :SB-ROTATE-BYTE) 19: ((LAMBDA (#:G[REQUIRE]13)) ASDF::MODULE-PROVIDE-ASDF) 20: (SB-IMPL::%MAP-FOR-EFFECT-ARITY-1 #<CLOSURE (LAMBDA #) {10053D3949}> (ASDF::MODULE-PROVIDE-ASDF SB-IMPL::MODULE-PROVIDE-CONTRIB)) 21: (REQUIRE :SB-ROTATE-BYTE NIL) 22: ((LAMBDA ())) 23: (SB-INT:SIMPLE-EVAL-IN-LEXENV (HANDLER-CASE (PROGN (REQUIRE :SB-ROTATE-BYTE) (PUSHNEW :SBCL-USES-SB-ROTATE-BYTE *FEATURES*)) (ERROR NIL (DELETE :SBCL-USES-SB-ROTATE-BYTE *FEATURES*))) #<NULL-LEXENV>) 24: (SB-INT:SIMPLE-EVAL-IN-LEXENV (EVAL-WHEN (:COMPILE-TOPLEVEL :LOAD-TOPLEVEL :EXECUTE) (HANDLER-CASE (PROGN (REQUIRE :SB-ROTATE-BYTE) (PUSHNEW :SBCL-USES-SB-ROTATE-BYTE *FEATURES*)) (ERROR NIL (DELETE :SBCL-USES-SB-ROTATE-BYTE *FEATURES*)))) #<NULL-LEXENV>) 25: (SB-FASL::LOAD-AS-SOURCE #<SB-SYS:FD-STREAM for "file /little/home/empty/desr/git/cl-utilities/cl-utilities.asd" {10050BA141}> NIL NIL) 26: ((FLET SB-FASL::LOAD-STREAM) #<SB-SYS:FD-STREAM for "file /little/home/empty/desr/git/cl-utilities/cl-utilities.asd" {10050BA141}> NIL) 27: (LOAD #P"/home/empty/desr/git/.asdf-registry/cl-utilities.asd")[:EXTERNAL] 28: (FIND-SYSTEM "cl-utilities" T) 29: ((FLET #:FORM-FUN-[EXIT-TAG-[ASDF-HIDDEN-SYSTEM-NAMES]518]529)) 30: (DESIRE::ASDF-HIDDEN-SYSTEM-NAMES #S(CL-UTILITIES :MODULE DESIRE::CL-UTILITIES))
I find this effect too drastic for my liking. I hope I'm not alone on this :-)
regards, Samium Gromoff -- _deepfire-at-feelingofgreen.ru O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Can you explain how you got hold of McCLIM and CL-Utilities? I'd be interested in seeing if I can replicate this, but I haven't used McCLIM in years, and back in those dark days we used only CVS (and we walked to school barefoot, uphill /both ways/). Maybe provide git urls for both mcclim and cl-utilities?
I'm actually surprised to see that cl-utilities wants to load the swank asdf definition, too....
thanks, r
On Sun, 29 Nov 2009, Robert Goldman wrote:
Can you explain how you got hold of McCLIM and CL-Utilities? I'd be interested in seeing if I can replicate this, but I haven't used McCLIM in years, and back in those dark days we used only CVS (and we walked to school barefoot, uphill /both ways/). Maybe provide git urls for both mcclim and cl-utilities?
For McCLIM, I'd just use the last release or try asdf-install. http://common-lisp.net/project/mcclim/downloads/mcclim-0.9.6.tar.gz They also have a nightly CVS snapshot. http://common-lisp.net/cvs_tarballs/mcclim.tar.gz
Cl-utilities has had no commits after the latest tarball. http://www.common-lisp.net/project/cl-utilities/cl-utilities-latest.tar.gz
I'm actually surprised to see that cl-utilities wants to load the swank asdf definition, too....
It was McCLIM that wanted swank (IIRC for a simple IDE). Cl-utilities requires sb-rotate-byte.
- Daniel
Daniel Herring wrote:
On Sun, 29 Nov 2009, Robert Goldman wrote:
Can you explain how you got hold of McCLIM and CL-Utilities? I'd be interested in seeing if I can replicate this, but I haven't used McCLIM in years, and back in those dark days we used only CVS (and we walked to school barefoot, uphill /both ways/). Maybe provide git urls for both mcclim and cl-utilities?
For McCLIM, I'd just use the last release or try asdf-install. http://common-lisp.net/project/mcclim/downloads/mcclim-0.9.6.tar.gz They also have a nightly CVS snapshot. http://common-lisp.net/cvs_tarballs/mcclim.tar.gz
Cl-utilities has had no commits after the latest tarball. http://www.common-lisp.net/project/cl-utilities/cl-utilities-latest.tar.gz
I'm actually surprised to see that cl-utilities wants to load the swank asdf definition, too....
It was McCLIM that wanted swank (IIRC for a simple IDE). Cl-utilities requires sb-rotate-byte.
OK, I'm confused. Why is what we see here unusual? I'm looking at the transcript and what seems to happen is that SG initially causes the various and sundry McCLIM sysdefs to be loaded (they are all in mcclim.asd), and then asks for the cl-utilities asdf definition to be loaded.
From there I don't see anything particularly odd in the transcript. We
get CL-UTILITIES, a boatload of CLIM systems, and SWANK. SWANK is imperatively loaded by McCLIM. The CLIM systems are all defined in mcclim.asd, which has been loaded by an SBCL initarg.
So what's weird here? What's the seemingly-unrelated software that we are looking up? is it esa-mcclim? I don't know about this because esa-mcclim.asd is not in the mcclim release tarball.
If the objection is that the mcclim.asd file contains a boatload of other system definitions, I don't see what that has to do with ASDF proper. That seems like a mcclim issue.
Best, r
Robert Goldman wrote:
Daniel Herring wrote:
On Sun, 29 Nov 2009, Robert Goldman wrote:
I'm actually surprised to see that cl-utilities wants to load the swank asdf definition, too....
It was McCLIM that wanted swank (IIRC for a simple IDE). Cl-utilities requires sb-rotate-byte.
OK, I'm confused. Why is what we see here unusual? I'm looking at the transcript and what seems to happen is that SG initially causes the various and sundry McCLIM sysdefs to be loaded (they are all in mcclim.asd), and then asks for the cl-utilities asdf definition to be loaded.
From there I don't see anything particularly odd in the transcript. We get CL-UTILITIES, a boatload of CLIM systems, and SWANK. SWANK is imperatively loaded by McCLIM. The CLIM systems are all defined in mcclim.asd, which has been loaded by an SBCL initarg.
So what's weird here? What's the seemingly-unrelated software that we are looking up? is it esa-mcclim? I don't know about this because esa-mcclim.asd is not in the mcclim release tarball.
Ok. Cl-utilities only intends to load sb-rotate-byte. Unfortunately, FIND-SYSTEM actually has to load ASDF files to find the definition it needs; by default, it reads each .asd in *asdf-c-r* until it finds the one it needs. Many system definitions are declarative; loading them causes negligible side effects. However, some asd files are more involved, in some cases even dumping fasls.
I think Samium was objecting to the registration of unrelated systems as part of ASDF's FIND-SYSTEM traversal. To me, the other side effects in some asd files are a bigger issue.
There would be several benefits if FIND-SYSTEM were to simply PROBE-FILE for a system definition in each of the configured paths (e.g. path1/system.asd, path2/system.asd) until one is found. Better efficiency, no pollution from unrelated packages, etc.
- Daniel
There would be several benefits if FIND-SYSTEM were to simply PROBE-FILE for a system definition in each of the configured paths (e.g. path1/system.asd, path2/system.asd) until one is found. Better efficiency, no pollution from unrelated packages, etc.
I agree except that ASDF doesn't require that the definition of foo be found in foo.asd or that bar.asd contain system bar. If we enforced this then we could say that find-system is a wrapper for a clever probe-file. Otherwise, we need to load definitions to make sure that things are happy. n'est pas?
-- Gary Warren King, metabang.com Cell: (413) 559 8738 Fax: (206) 338-4052 gwkkwg on Skype * garethsan on AIM * gwking on twitter
On Tue, 1 Dec 2009, Gary King wrote:
There would be several benefits if FIND-SYSTEM were to simply PROBE-FILE for a system definition in each of the configured paths (e.g. path1/system.asd, path2/system.asd) until one is found. Better efficiency, no pollution from unrelated packages, etc.
I agree except that ASDF doesn't require that the definition of foo be found in foo.asd or that bar.asd contain system bar. If we enforced this then we could say that find-system is a wrapper for a clever probe-file. Otherwise, we need to load definitions to make sure that things are happy. n'est pas?
Consider my comment as a feature request. Drepper's dso howto explains issues that might negatively affect ASDF systems in the large.
- Daniel
Can you provide a more complete pointer to this post? I'm afraid I don't know what you refer to. Thanks, R
On Dec 1, 2009, at 21:38, Daniel Herring dherring@tentpost.com wrote:
On Tue, 1 Dec 2009, Gary King wrote:
There would be several benefits if FIND-SYSTEM were to simply PROBE-FILE for a system definition in each of the configured paths (e.g. path1/system.asd, path2/system.asd) until one is found. Better efficiency, no pollution from unrelated packages, etc.
I agree except that ASDF doesn't require that the definition of foo be found in foo.asd or that bar.asd contain system bar. If we enforced this then we could say that find-system is a wrapper for a clever probe-file. Otherwise, we need to load definitions to make sure that things are happy. n'est pas?
Consider my comment as a feature request. Drepper's dso howto explains issues that might negatively affect ASDF systems in the large.
- Daniel
asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
Robert P. Goldman wrote:
On Dec 1, 2009, at 21:38, Daniel Herring dherring@tentpost.com wrote:
Drepper's dso howto explains issues that might negatively affect ASDF systems in the large.
Can you provide a more complete pointer to this post? I'm afraid I don't know what you refer to.
Sorry; it was an obscure reference for a lisp mailing list. It discusses C/C++ linking issues, but many would still be relevant if OpenOffice were written in CL. http://people.redhat.com/drepper/dsohowto.pdf
Continuing OT, more recommended reading for people making build systems: http://www.gnu.org/prep/standards/html_node/Standard-Targets.html http://www.gnu.org/prep/standards/html_node/DESTDIR.html http://www.gnu.org/software/autoconf/manual/html_node/Introduction.html http://www.gnu.org/software/autoconf/manual/html_node/The-GNU-Build-System.h...
Again, there's not a 1-1 correspondence with CL, but the underlying usability issues are universal.
- Daniel