set variables before executing the "test-op"
Hi all, I am using cl-project to generate project skeleton for the library including separate asd file for unit tests. The test framework I use is the default for cl-project: prove. Using this framework I would like to set some special variables to control test execution _before_ running the asdf:test-system i.e. (setf prove.color:*enable-colors* nil) (setf prove:*default-reporter* :tap) But the problem is when I set them directly in .asd file, the CL reader complains what the prove.color and prove packages do not exist (which is true, since the prove dependency is not yet loaded). I could set them in every test file, but I would like to set them globally. How could I set them after the dependency to unit test library (prove) is loaded and therefore package exists? This is what I have in .asd file for tests: :defsystem-depends-on (:prove-asdf) :perform (test-op :after (op c) (funcall (intern #.(string :run-test-system) :prove-asdf) c) (asdf:clear-system c))) Br /Alexey
It's probably wrong to set those settings from your .asd file, since they may be set or reset before your project runs, or in between two runs. If you actually care about those variables, define a function that sets them, and call it at the beginning of those files. If you have a lot of files, define a class for those files that does it in its perform method for basic-load-op. As for defining accessors before the packages are interned, to be executed by a function run *after* they are interned, you can use such idioms as: (setf (symbol-value (find-symbol* :*enable-colors* :prove.color) nil) Note that find-symbol* is defined by uiop, which is :use'd by :asdf-user. Alternatively, you could (load-system :prove) in your .asd file, but it's ugly. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org On Tue, Sep 6, 2016 at 6:58 AM, Alexey Veretennikov <txm.fourier@gmail.com> wrote:
Hi all,
I am using cl-project to generate project skeleton for the library including separate asd file for unit tests. The test framework I use is the default for cl-project: prove. Using this framework I would like to set some special variables to control test execution _before_ running the asdf:test-system i.e. (setf prove.color:*enable-colors* nil) (setf prove:*default-reporter* :tap)
But the problem is when I set them directly in .asd file, the CL reader complains what the prove.color and prove packages do not exist (which is true, since the prove dependency is not yet loaded). I could set them in every test file, but I would like to set them globally. How could I set them after the dependency to unit test library (prove) is loaded and therefore package exists?
This is what I have in .asd file for tests:
:defsystem-depends-on (:prove-asdf) :perform (test-op :after (op c) (funcall (intern #.(string :run-test-system) :prove-asdf) c) (asdf:clear-system c)))
Br /Alexey
It's probably wrong to set those settings from your .asd file, since they may be set or reset before your project runs, or in between two runs. If you actually care about those variables, define a function that sets them, and call it at the beginning of those files.
If you have a lot of files, define a class for those files that does it in its perform method for basic-load-op.
As for defining accessors before the packages are interned, to be executed by a function run *after* they are interned, you can use such idioms as: (setf (symbol-value (find-symbol* :*enable-colors* :prove.color) nil) Note that find-symbol* is defined by uiop, which is :use'd by :asdf-user.
Alternatively, you could (load-system :prove) in your .asd file, but it's ugly.
This sounds like a good moment to come up with an interface between test-system and the test suite runner, so that you can pass arguments to the test runner directly through asdf:test-system. Using dynamic variables for this use case is a bad idea. -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur.
On Tue, Sep 6, 2016 at 9:03 AM, Stelian Ionescu <sionescu@cddr.org> wrote:
It's probably wrong to set those settings from your .asd file, since they may be set or reset before your project runs, or in between two runs. If you actually care about those variables, define a function that sets them, and call it at the beginning of those files.
If you have a lot of files, define a class for those files that does it in its perform method for basic-load-op.
As for defining accessors before the packages are interned, to be executed by a function run *after* they are interned, you can use such idioms as: (setf (symbol-value (find-symbol* :*enable-colors* :prove.color) nil) Note that find-symbol* is defined by uiop, which is :use'd by :asdf-user.
Alternatively, you could (load-system :prove) in your .asd file, but it's ugly.
This sounds like a good moment to come up with an interface between test-system and the test suite runner, so that you can pass arguments to the test runner directly through asdf:test-system. Using dynamic variables for this use case is a bad idea.
Please propose an actual interface. I don't see how that can be done in the current architecture. Fixing the architecture and doing it right would be a good exercise for whoever volunteers to write ASDF 4. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Not only is there no contradiction between egoism and altruism, but no altruism is possible without egoism — for what betterment to wish to an other person devoid of selfish desire, to whom any change is indifferent?
On 9/6/16 Sep 6 -8:03 AM, Stelian Ionescu wrote:
It's probably wrong to set those settings from your .asd file, since they may be set or reset before your project runs, or in between two runs. If you actually care about those variables, define a function that sets them, and call it at the beginning of those files.
If you have a lot of files, define a class for those files that does it in its perform method for basic-load-op.
As for defining accessors before the packages are interned, to be executed by a function run *after* they are interned, you can use such idioms as: (setf (symbol-value (find-symbol* :*enable-colors* :prove.color) nil) Note that find-symbol* is defined by uiop, which is :use'd by :asdf-user.
Alternatively, you could (load-system :prove) in your .asd file, but it's ugly.
This sounds like a good moment to come up with an interface between test-system and the test suite runner, so that you can pass arguments to the test runner directly through asdf:test-system. Using dynamic variables for this use case is a bad idea.
Offhand, I don't see how this is possible. If a test library decides to use dynamic variables to control its function, how can ASDF fix that? Mostly I try to handle this by writing methods on PERFORM :around ((op test-op) (system (eql (find-system ...)))) Actually, Alexey's case looks more complex, since it involves some call to CLEAR-SYSTEM. This suggests that there's some setup and teardown that isn't being handled by the test library itself. What I have done in my work on testing with 5AM is to add special error classes for test-failures, an unexpected number of tests (we had problems once where due to a bug tests were not run, which didn't yield errors but.... wasn't working, either), and unexpected test passes. That addresses the problem of ASDF not returning values from its operations. Best, r
Hi, Thanks, I decided to follow your first advice with some difference: I've created a file base.lisp with the following contents: (in-package :cl-user) (defpackage test.base (:use :cl :prove)) (in-package :test.base) ;; turn off ansi colors in report output (setf prove.color:*enable-colors* nil) ;; change type of the reporter to Test Anything Protocol (setf prove:*default-reporter* :tap) and added this file to components section of the test asd file: :components ((:module "t" :components ((:file "base") (:test-file "pack-test") ... I've also added :prove to depends-on section of the test asd file. This works for me and I don't have to modify all test files. Is this approach ok or bad(dangerous, bad style etc.)? Faré <fahree@gmail.com> writes:
It's probably wrong to set those settings from your .asd file, since they may be set or reset before your project runs, or in between two runs.
If you actually care about those variables, define a function that sets them, and call it at the beginning of those files.
If you have a lot of files, define a class for those files that does it in its perform method for basic-load-op.
As for defining accessors before the packages are interned, to be executed by a function run *after* they are interned, you can use such idioms as: (setf (symbol-value (find-symbol* :*enable-colors* :prove.color) nil) Note that find-symbol* is defined by uiop, which is :use'd by :asdf-user.
Alternatively, you could (load-system :prove) in your .asd file, but it's ugly.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
On Tue, Sep 6, 2016 at 6:58 AM, Alexey Veretennikov <txm.fourier@gmail.com> wrote:
Hi all,
I am using cl-project to generate project skeleton for the library including separate asd file for unit tests. The test framework I use is the default for cl-project: prove. Using this framework I would like to set some special variables to control test execution _before_ running the asdf:test-system i.e. (setf prove.color:*enable-colors* nil) (setf prove:*default-reporter* :tap)
But the problem is when I set them directly in .asd file, the CL reader complains what the prove.color and prove packages do not exist (which is true, since the prove dependency is not yet loaded). I could set them in every test file, but I would like to set them globally. How could I set them after the dependency to unit test library (prove) is loaded and therefore package exists?
This is what I have in .asd file for tests:
:defsystem-depends-on (:prove-asdf)
:perform (test-op :after (op c)
(funcall (intern #.(string :run-test-system) :prove-asdf) c)
(asdf:clear-system c)))
Br /Alexey
-- Br, /Alexey
On Tue, Sep 6, 2016 at 3:49 PM, Alexey Veretennikov <txm.fourier@gmail.com> wrote:
Hi,
Thanks, I decided to follow your first advice with some difference: I've created a file base.lisp with the following contents:
(in-package :cl-user) (defpackage test.base (:use :cl :prove)) (in-package :test.base)
;; turn off ansi colors in report output (setf prove.color:*enable-colors* nil) ;; change type of the reporter to Test Anything Protocol (setf prove:*default-reporter* :tap)
and added this file to components section of the test asd file: :components ((:module "t" :components ((:file "base") (:test-file "pack-test") ...
I've also added :prove to depends-on section of the test asd file.
This works for me and I don't have to modify all test files. Is this approach ok or bad(dangerous, bad style etc.)?
This looks like it's good style to me. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Give up all hope for a better yesterday, even a better just now. Never give up hope for a better tomorrow. — Patri Friedman
participants (4)
-
Alexey Veretennikov -
Faré -
Robert Goldman -
Stelian Ionescu