While working on the test suite of swap-bytes on SBCL(you'll need
=1.1.6 I think), I noticed two unexpected results in using :force :
1) (asdf:load-system :swap-bytes/test :force t) does not recurse into :defsystem-depends-on deps. Is this intentional ?
2) (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test)) needlessly compiles files from :swap-bytes on which it depends. I've attached a log that traces PERFORM and COMPUTE-ACTION-STAMP and shows that two files from :swap-bytes get recompiled even though they haven't changed.
I'm wondering if the problem lay with the ASDF extension in Madeira-port, but that's so simple I can't find any obvious fault. Ideas ?
If you want to test for yourselves clone https://github.com/sionescu/madeira-port.git and https://github.com/sionescu/swap-bytes.git
- (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test))
needlessly compiles files from :swap-bytes on which it depends. I've attached a log that traces PERFORM and COMPUTE-ACTION-STAMP and shows that two files from :swap-bytes get recompiled even though they haven't changed.
I'm wondering if the problem lay with the ASDF extension in Madeira-port, but that's so simple I can't find any obvious fault. Ideas ?
A) I see nothing wrong in madeira-port, but there is no reason for swap-bytes to use it, since i) madeira-port requires asdf 2.29 ii) asdf 2.27 and later already have :if-feature that provides the same basic service iii) you are not using any of the feature-eval extensions provided by madeira-port
B) your perform methods does a force of swap-bytes/test. Did you try to remove that, and see if the bug still happens? That might help narrow it down. I don't see why you need to force, anyway. (And yes, it looks like a bug.) Actually, it's quite possible that even without that force, this could possibly cause the second load-system to have a confused plan. Since you're already depending on asdf3, what about instead have in your defsystem swap-bytes something like the following statements (untested): :in-order-to ((test-op (load-op swap-bytes/test))) :perform (test-op (o c) (uiop:symbol-call :5am :run! :swap-bytes))
NB: ok, using uiop instead of asdf/package means you depend on 2.32 instead of 2.29.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Let's take compulsion out of compassion.
On Fri, 2013-07-26 at 14:47 -0400, Faré wrote:
- (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test))
needlessly compiles files from :swap-bytes on which it depends. I've attached a log that traces PERFORM and COMPUTE-ACTION-STAMP and shows that two files from :swap-bytes get recompiled even though they haven't changed.
I'm wondering if the problem lay with the ASDF extension in Madeira-port, but that's so simple I can't find any obvious fault. Ideas ?
A) I see nothing wrong in madeira-port, but there is no reason for swap-bytes to use it, since i) madeira-port requires asdf 2.29 ii) asdf 2.27 and later already have :if-feature that provides the same basic service
Oh, goodie.
iii) you are not using any of the feature-eval extensions provided by madeira-port
B) your perform methods does a force of swap-bytes/test. Did you try to remove that, and see if the bug still happens? That might help narrow it down. I don't see why you need to force, anyway. (And yes, it looks like a bug.)
When I test stuff I want the entire test suite to be recompiled on-the-fly so as to catch macro redefinitions, etc...
Actually, it's quite possible that even without that force, this could possibly cause the second load-system to have a confused plan.
This happens when I force-load the test suite from the repl too, not only from within the perform method.
Since you're already depending on asdf3, what about instead have in your defsystem swap-bytes something like the following statements (untested): :in-order-to ((test-op (load-op swap-bytes/test))) :perform (test-op (o c) (uiop:symbol-call :5am :run! :swap-bytes))
NB: ok, using uiop instead of asdf/package means you depend on 2.32 instead of 2.29.
Ok, will do
B) your perform methods does a force of swap-bytes/test. Did you try to remove that, and see if the bug still happens? That might help narrow it down. I don't see why you need to force, anyway. (And yes, it looks like a bug.)
When I test stuff I want the entire test suite to be recompiled on-the-fly so as to catch macro redefinitions, etc...
This is ASDF3. Dependencies are propagated correctly, unlike with ASDF 1&2. If you want to catch a macro redefinition, just define a dependency, as you should.
Actually, it's quite possible that even without that force, this could possibly cause the second load-system to have a confused plan.
This happens when I force-load the test suite from the repl too, not only from within the perform method.
Which files do or don't get compiled twice?
I can't use :in-order-to because it doesn't allow me to specify :force
You shouldn't need force. It's a bug if you do (either in your code or in ASDF).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Be wiser than other people if you can; but do not tell them so. — Chesterfield
On Fri, 2013-07-26 at 17:39 -0400, Faré wrote:
B) your perform methods does a force of swap-bytes/test. Did you try to remove that, and see if the bug still happens? That might help narrow it down. I don't see why you need to force, anyway. (And yes, it looks like a bug.)
When I test stuff I want the entire test suite to be recompiled on-the-fly so as to catch macro redefinitions, etc...
This is ASDF3. Dependencies are propagated correctly, unlike with ASDF 1&2. If you want to catch a macro redefinition, just define a dependency, as you should.
When I develop interactively and change stuff without saving files, perhaps because I'm working in a scratch buffer, I still want the suite to be recompiled.
Actually, it's quite possible that even without that force, this could possibly cause the second load-system to have a confused plan.
This happens when I force-load the test suite from the repl too, not only from within the perform method.
Which files do or don't get compiled twice?
:swap-bytes/test depends on :swap-bytes. When I (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test)), some files from :swap-bytes get recompiled too, namely the last two, network.lisp and endianness.lisp
On Fri, 2013-07-26 at 23:48 +0200, Stelian Ionescu wrote: [...]
Which files do or don't get compiled twice?
:swap-bytes/test depends on :swap-bytes. When I (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test)), some files from :swap-bytes get recompiled too, namely the last two, network.lisp and endianness.lisp
Now that I switched to using cl-source-file and :if-feature it no longer happens, so it must have been caused by madeira-port's extension. It's interesting that the two files that were getting spuriously recompiled were the two following the last :madeira-port file in the defsystem.
This is ASDF3. Dependencies are propagated correctly, unlike with ASDF 1&2. If you want to catch a macro redefinition, just define a dependency, as you should.
When I develop interactively and change stuff without saving files, perhaps because I'm working in a scratch buffer, I still want the suite to be recompiled.
If you recompiled a file but did not recompile some files that depend on it, ASDF3 will correctly recompile them when you (test-system :swap-bytes). No need to force anything. This is not ASDF 1&2.
Which files do or don't get compiled twice?
:swap-bytes/test depends on :swap-bytes. When I (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test)), some files from :swap-bytes get recompiled too, namely the last two, network.lisp and endianness.lisp
Indeed, this is a subtle bug in madeira-port: when you depend on a component that is excluded from the build by madeira-port, the component fails to declare an empty set of output-files, while its perform method fails to create the declared output-files, and therefore ASDF while planning decides that this build step is incomplete, as well as anything that depends on it directly or indirectly.
The solution is for madeira-port to also provide a :around method for output-files. That's subtle: I didn't think of it when reading the source code.
For once, this wasn't an ASDF bug.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If once you have paid him the Dane-geld / You never get rid of the Dane. — Rudyard Kipling
On Fri, 2013-07-26 at 18:05 -0400, Faré wrote:
This is ASDF3. Dependencies are propagated correctly, unlike with ASDF 1&2. If you want to catch a macro redefinition, just define a dependency, as you should.
When I develop interactively and change stuff without saving files, perhaps because I'm working in a scratch buffer, I still want the suite to be recompiled.
If you recompiled a file but did not recompile some files that depend on it, ASDF3 will correctly recompile them when you (test-system :swap-bytes). No need to force anything. This is not ASDF 1&2.
If I edit a macro in a scratch buffer ASDF cannot possibly detect it because no file on disk has changed. Maybe I'm being a little too cautious here but for the moment I'll keep forcing the reload of the test suite.
Which files do or don't get compiled twice?
:swap-bytes/test depends on :swap-bytes. When I (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test)), some files from :swap-bytes get recompiled too, namely the last two, network.lisp and endianness.lisp
Indeed, this is a subtle bug in madeira-port: when you depend on a component that is excluded from the build by madeira-port, the component fails to declare an empty set of output-files, while its perform method fails to create the declared output-files, and therefore ASDF while planning decides that this build step is incomplete, as well as anything that depends on it directly or indirectly.
The solution is for madeira-port to also provide a :around method for output-files. That's subtle: I didn't think of it when reading the source code.
Thanks, fixed.
On Fri, Jul 26, 2013 at 6:16 PM, Stelian Ionescu sionescu@cddr.org wrote:
On Fri, 2013-07-26 at 18:05 -0400, Faré wrote: If I edit a macro in a scratch buffer ASDF cannot possibly detect it because no file on disk has changed. Maybe I'm being a little too cautious here but for the moment I'll keep forcing the reload of the test suite.
If your system depends on a macro that is not included in the source code, that's a bug in your system, and failure is the correct behavior for ASDF. So don't.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org If the human mind were simple enough to understand, we'd be too simple to understand it. — Pat Bahn
On Fri, 2013-07-26 at 14:47 -0400, Faré wrote: [...]
Since you're already depending on asdf3, what about instead have in your defsystem swap-bytes something like the following statements (untested): :in-order-to ((test-op (load-op swap-bytes/test))) :perform (test-op (o c) (uiop:symbol-call :5am :run! :swap-bytes))
I can't use :in-order-to because it doesn't allow me to specify :force
Yes, this is probably a bug. In register-system-definition, we should get the force argument, and propagate to a variant of load-systems that itself should take keyword arguments, and is called by load-systems.
Regarding swap-bytes, there may (or may not) be useful stuff in sb-rotate-bytes and nibbles.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Director is a misnomer. You're a hoper. You put all these people together and you hope it all works out. — Frank Oz, director of "Dirty Rotten Scoundrels"
On Fri, Jul 26, 2013 at 1:27 PM, Stelian Ionescu sionescu@cddr.org wrote:
While working on the test suite of swap-bytes on SBCL(you'll need
=1.1.6 I think), I noticed two unexpected results in using :force :
- (asdf:load-system :swap-bytes/test :force t) does not recurse
into :defsystem-depends-on deps. Is this intentional ?
- (asdf:load-system :swap-bytes/test :force '(:swap-bytes/test))
needlessly compiles files from :swap-bytes on which it depends. I've attached a log that traces PERFORM and COMPUTE-ACTION-STAMP and shows that two files from :swap-bytes get recompiled even though they haven't changed.
I'm wondering if the problem lay with the ASDF extension in Madeira-port, but that's so simple I can't find any obvious fault. Ideas ?
If you want to test for yourselves clone https://github.com/sionescu/madeira-port.git and https://github.com/sionescu/swap-bytes.git
-- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib
On Fri, 2013-07-26 at 14:04 -0400, Faré wrote:
Yes, this is probably a bug. In register-system-definition, we should get the force argument, and propagate to a variant of load-systems that itself should take keyword arguments, and is called by load-systems.
I gather you're talking about the first issue. What about the second one ?
Regarding swap-bytes, there may (or may not) be useful stuff in sb-rotate-bytes and nibbles.
swap-bytes is a complement to sb-rotate-bytes and nibbles does something completely different.
On Fri, Jul 26, 2013 at 1:27 PM, Stelian Ionescu sionescu@cddr.org wrote:
While working on the test suite of swap-bytes on SBCL(you'll need
=1.1.6 I think), I noticed two unexpected results in using :force :
- (asdf:load-system :swap-bytes/test :force t) does not recurse
into :defsystem-depends-on deps. Is this intentional ?
On Fri, Jul 26, 2013 at 2:04 PM, Faré fahree@gmail.com wrote:
Yes, this is probably a bug. In register-system-definition, we should get the force argument, and propagate to a variant of load-systems that itself should take keyword arguments, and is called by load-systems.
So I started working on that by defining a function load-systems* that accepts a list of systems and keywords (unlike load-systems). [Patch attached]
Issue: even once such a function is defined, I realized that ASDF3 does NOT have a special variable *force* or *plan* or anything from which to extract which :force value to use. I'm not convinced that such a special variable would be a win. In any case, adding one is an architectural decision for the maintainer, not a superficial change that a non-maintainer like me can take. [And if I were still maintainer, I would have a strong prejudice against it, and wouldn't see this current feature request as a sufficient motivation to add such a variable.]
Therefore, this patch does the easy part of the job of defining a new function, but doesn't actually propagate require flags through defsystem-depends-on'dencies.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Young man, in mathematics you don't understand things, you just get used to them. — John von Neumann (1903-1957)
-
Faré -
Stelian Ionescu