On 29 Dec 2021, at 9:32, Wilfredo Velazquez wrote:

FYI mostly unrelated 2c - Opening issues for questions is not overkill IMO. In fact it serves as a great knowledge base with discussion that is easily searchable. Just close the issue when the question is resolved .

That would be fine. I will add "Question" as a ticket label. GitHub also has "discussions" separate from "issues," but GitLab does not have this, at least not yet.

On Tue, Dec 28, 2021 at 10:52 PM Albus Matt Piroglu <mattapiroglu@gmail.com> wrote:

...

Note I have opened an issue on gitlab, but I know it's an overkill to open issues for questions. I couldn't find an appropriate channel until I found this mailing list. I'll close the issue, and here is my question for your consideration:

Hi, I'm trying to test a functionality that depends on loading systems. Thusly I ended up writing some tests that need to load a "test-system" during the tests, and I clear the system afterwards.

A simplified code for the test in question is:

(test check-the-system-is-loaded-as-expected () (asdf:load-system "my-test-system-under-test-folder") (is --check that the system loaded as expected--) (asdf:clear-system "my-test-system-under-test-folder") (delete-package "PACKAGE-FROM-TEST-SYSTEM"))

and, to clarify, another test relies on the same system not being loaded, so I don't think a dependency would solve the issue, it is rather a dynamic decision to load or not.

Now, I know that this feature is deprecated and may not work. What I'm wondering if there's a safer way to achieve this. Wouldn't some tests for asdf itself need loading some test systems for example? (hmm it actually might be a good idea to go check them now). OK, for example there's a hello-world-example.asd in test, which looks similar to my my-test-system-under-test-folder.

So, instead of asdf:load-system in the above test code, should I be using a lower-level operation call?

Thanks!

Albus

-- Wilfredo Velázquez-Rodríguez

Thanks for the replies! A few clarifications and further questions: I'm not entirely sure what you mean by "test" in the above. If you look

at the tests in ASDF (in the test/ subdirectory), you will see that these are typically run from the shell, and so don't have any lasting effects. Lisp starts, the test is run, lisp exits. So typically we don't have to worry about clearing systems.

Yes, when I wrote 'test', I was pointing to the test subfolder in asdf. And as you described, the way those tests work is different than what I wanted to achieve. I am currently calling (asdf:test-system "lib-helper") to do my tests, and would like to continue using the same lisp image from that point on. For reference, I uploaded related changes (with the recursive load-op currently in the code) to my lib: https://github.com/albuspiroglu/cl-lib-helper The test of concern is: https://github.com/albuspiroglu/cl-lib-helper/blob/f98f39dd1460287ddcb77525b... And the system this test loads is under its sub-folder test-system/ . I just needed the simplest system definition with two symbols in a package. Then the lib-helper will create some target packages and reference those symbols in different ways, depending on whether the user has chosen to pre-load, and map to their chosen set of systems.

If you want to understand better how ASDF does its testing, a good resource is to read the file test/script-support.lisp which has the testing utilities. (If you would like to submit a merge request with additional comments and/or docstrings for this file, that would be welcome!)

When I read the code you mentioned, I'll make sure to do some improvements, or at least attempt!

You will see that this file contains a def-test-system macro (unfortunately, without a docstring) that can be used in tests to create a system without needing an asd file. See, for example, test/test-sysdef-asdf.script for uses of def-test-system

I was looking for something like this yesterday, before creating a dummy system under my test/ folder. Then I realised it was convenient to create the asd file and a lisp file along so that I can define a test package along with the system, because my cl-lib-helper checks for some symbols in a package which is expected to be asdf-loaded.

This file also has an example of the use of CLEAR-SYSTEM, which is exported by ASDF and is defined as follows:

...

"Clear the entry for a SYSTEM in the database of systems previously defined. However if the system was registered as PRELOADED (which it is if it is IMMUTABLE), then a new system with the same name will be defined and registered in its place from which build details will have been cleared. Note that this does NOT in any way cause any of the code of the system to be unloaded. Returns T if system was or is now undefined, NIL if a new preloaded system was redefined."

Note from the above that you cannot unload a system. Loading a system has arbitrary effects on the running lisp image, and those effects cannot, in general, be undone. This is part of the reason that each ASDF test is run in a separate lisp process.

I always thought it's a matter of what the library's code brings to the lisp image that creates this complication, thus I started putting some library cleanup functions to help with reloading my libs without restarting lisp. What I do is, generally, delete the packages of the library and then call asdf:clear-system. Maybe do any other generic method / class definition deletion as required, which I haven't needed yet. For example this function from the same library I mentioned does exactly that: https://github.com/albuspiroglu/cl-lib-helper/blob/f98f39dd1460287ddcb77525b... I wonder if there are other side-effects that asdf itself creates which would be more difficult to clean up?

As for testing whether or not a system has been loaded, typically what is done is to define a package and a variable in that package in the test system, and then check for that variable having the expected value in order to see if the system has been loaded.