fiveam-devel

fiveam-devel@common-lisp.net

10 discussions

PATCH: SIGNAL a condition on test failures, for use with ASDF:TEST-SYSTEM
by Vladimir Sedach 15 Sep '19

15 Sep '19

Hello! I have a proposed patch to FiveAM on GitHub: https://github.com/sionescu/fiveam/pull/58 I recently wrote a script to run FiveAM tests for one of my libraries on many different implementations on your own machine: https://gitlab.common-lisp.net/uri-template/uri-template2/blob/master/run-t… I do not want to copy-paste that script for all of my libraries. It would be nice to contribute a generalized version of the script to Roswell (on which the script is based), and just be able to say "ros test my-system in all installed implementations" for any system. The first step to doing that is to get ASDF:TEST-SYSTEM to report errors using a common interface. Because of the way ASDF is designed, ASDF:TEST-SYSTEM needs to use conditions to signal test failures. I like this idea. Since you do not have to handle conditions raised by SIGNAL, the functionality can be added without impacting existing use or interfaces of FiveAM (the use of WARNING instead of ERROR as the parent condition of TEST-SPEC-FAILURE also ensures this, as many people tend to abuse HANDLER-CASE to indiscriminately catch ERRORs). With this patch, all projects that use FiveAM and define ASDF test-op should be testable from a generic version of the Roswell script without any changes. The basic mechanism for test automation with FiveAM is then very simple: (handler-case (asdf:test-system "system") (fiveam:test-spec-failure (condition) (princ condition uiop:*stderr*) (uiop:quit 1))) Other test libraries can have supported added as well: (handler-case (asdf:test-system "system") ((or fiveam:test-spec-failure prove:test-failure) (condition) (princ condition uiop:*stderr*) (uiop:quit 1))) The goal is to introduce a parent condition in ASDF that FiveAM, etc., will then inherit from: (handler-case (asdf:test-system "system") (asdf:test-failure (condition) (princ condition uiop:*stderr*) (uiop:quit 1))) Ultimately, any system using any test library should be testable this way, ensuring that test automation becomes trivial to build. I am going to propose adding the parent condition to ASDF, but in the meantime (until the ASDF patch makes it to the repository, another release comes out, release becomes widely available…), I would like to get this code into FiveAM. Vladimir

2 2

Is this project still active?
by Jon Boone 01 Jul '15

01 Jul '15

Folks, Is this project still active? I'm interested in understanding how to use it - so that I can extend it to talk to growl. --jon

3 2

FiveAM docs
by Robert P. Goldman 21 Nov '13

21 Nov '13

I just grabbed myself the latest FiveAM from quicklisp. Seeing no docs, I tried to use make-qbook.lisp to build the html docs. This broke with the following error: No initial heading in #P"/Users/rpg/quicklisp/dists/quicklisp/software/fiveam-1.1/fiveam.asd". Is this expected? I noticed that when I tried to build the docs, a lot of arnesi libraries came in, although FiveAM proper has moved away from arnesi. So I *suspect* this is bit-rotted. But perhaps I've done something wrong, or make-qbook needs updating? thanks, r

1 0

[fiveam-devel] how do you use fiveam?
by Marco Baringer 11 Dec '12

11 Dec '12

hi, i've been thinking about the def-suite* vs in-suite* thing, and talking to some people who use fiveam, and i realized that even though i'd like to optimize fiveam's api for the "common case," the only common case i know of is me. and i'm not always the most common user. if you have 5 minutes (or 10), i'd really appreciate it if you could tell me: 1 - do you use test suites? 1a - if so, do you generally have one suite you're working on or do you switch from suite to suite regularly? 2 - do you run tests via asdf (or whatever) regularly? 3 - do you C-cC-c single tests in a file or do you just recompile entire files? 4 - do you use *debug-on-failure* ? 5 - do you test before the code runs or do you test to check that the code does actually run? 6 - do you cut 'n paste repl interactions to write your test from? 7 - have you ever used the function 5am:! ? thanks, -- -marco

3 2

[fiveam-devel] Github issues, IN-SUITE* semantics
by Olof-Joachim Frahm 08 Dec '12

08 Dec '12

Hi all, I've looked through the open issues on Github and would like to get some input on a possible change/fix for issue #1, IN-SUITE* semantics: A patch would be at [1], but I'm not sure this is the way to go. The change is to modify IN-SUITE*, so that it updates the existing suite if possible. That was not quite straight forward as I'd hoped, in particular, the use of *SUITE* as the default value for :IN is a bit dangerous in this situation, because without checking for that case, one can quite easily create a loop from the updated suite to its parent, that is: itself, if the form is evaluated multiple times. So now I'm checking for that, but really, isn't T as the default parent better? Updating would be really easy that way. Anyway, a testcase of the expected(?) behaviour is included. Of course deprecating IN-SUITE* (in favour of DEF-SUITE*?) would fix this, but keeping defined tests alive is a nice feature as well. (Also, it doesn't seem to be used all that often - on my system only elephant had some calls; admittedly grepping all quicklisp packages to get more information would be nice.) Other than that, issue #3, named lambdas is already fixed, right? As far as I can tell there's no way to force any more information into the backtrace and NAMED-LAMBDA is already used, so the backtrace actually contains enough hints about the involved test case. Cheers, Olof [1]: https://github.com/Ferada/fiveam/commit/67eea929cb38b01a84a5285baf35e297e21…

2 1

[fiveam-devel] (no subject)
by Eliam Pacheco 03 Dec '12

03 Dec '12

HI! I’m using FiveAM but I have a problem. I need to make a unit test for UCW actions but for obvious reasons I get: #<IT.BESE.ARNESI::CLOSURE/CC {100734E37B}> how I can get the output values???

1 0

[fiveam-devel] Unit Test Framework
by Tapiwa Gutu 30 Nov '12

30 Nov '12

Faithful hackers, I decided to take up the challenge laid down here http://fare.livejournal.com/169346.html and try to consolidate the Common Lisp unit testing frameworks. I have written a framework that aims to consolidate all the major features of all your frameworks mentioned in this blog http://aperiodic.net/phil/archives/Geekery/notes-on-lisp-testing-frameworks… . You can find it on Github https://github.com/tgutu/clunit. I also wrote a blog on the development of the framework and reasons for it here http://ml.sun.ac.za/2012/11/09/developing-a-unit-test-framework-part-1/ if you are interested. I would very much appreciate it, if you could join me in this effort and we all work together torwards making this project a success. Regards, Tapiwa

4 5

[fiveam-devel] patch to RESULTS-STATUS
by Robert Goldman 25 Nov '12

25 Nov '12

RESULTS-STATUS tells you if there are any non-"TEST-PASSED" results. But it doesn't provide any way for you to get your hands on these results. Originally, I thought that one could simply test for them, but the results categories aren't actually exported from FiveAM, so this is difficult. To make it easier to find "interesting" results, I modified RESULTS-STATUS to return a second value. The first value is a boolean, as before, telling you if the tests all passed. If they didn't, the second value is now a list of the non-passing results. I believe this to be a benign change from the standpoint of backward compatibility, since anyone just looking for a single return value will get the same boolean value as before. It's possible that the actual value --- not interpreted as a boolean --- will be different, since the new code uses REMOVE-IF instead of EVERY. But if a caller takes information out of the results list in a function whose documentation proclaims it to be boolean, I submit that the caller is doing so at his/her own risk! Proposed patch is attached. Best, Robert

2 3

[fiveam-devel] small default formatting output patch
by Matthew Curry 18 Nov '12

18 Nov '12

Hi, Attached is a diff of check.lisp. I got annoyed with the (otherwise really really awesome) default 'is' macro output, which when testing large-ish s-exps, it all gets crammed to the right margin. I've also attached a before and after text file so you can see what I mean. If you can incorporate my change (or if you know a better way to do this) that would be great! -Matt

2 1

[fiveam-devel] FiveAM release 1.0
by Stelian Ionescu 03 Jun '12

03 Jun '12

Yay, the first release is here :) You can download it from common-lisp.net or github, otherwise wait until the next Quicklisp udate -- Stelian Ionescu a.k.a. fe[nl]ix Quidquid latine dictum sit, altum videtur. http://common-lisp.net/project/iolib

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

fiveam-devel