29.12.2011, 15:08, "Luís Oliveira" luismbo@gmail.com:
On Thu, Dec 29, 2011 at 3:23 AM, Anton Vodonosov avodonosov@yandex.ru wrote:
Not yet, because I am in the beginning of the project and only starting to see the first results (it's not really a buildfarm, but a small system where anyone may run a simple command, like (load "agent.lisp"), and it runs the tests and submits the results to the central server; so that we will be able to accumulate the statictics; currently this command just runs tests on whatever is the current quicklisp distribution is installed on the system, and I am the only person who runs this command - I am testing how it works and polishing the corner cases).
OK, cool stuff. Have you considered using something like Jenkins?
Jenkins is an interesting software (thanks for the reference BTW), but it (and other continuous integration servers) has not so much overlap with what I do.
For Jenkins to run tests we need to provide a command "run tests". Out of box it will be unable to run CL tests. So it's the first thing I do - unify the interface of running test suites of different CL libraries.
Also it is important to represent results so that we can see side by side results of a particular library on different CL implementations; or results of all the libraries on different quicklisp distributions and so on. Continuous integration servers will not provide such a representation.
Another goal is that people can contribute test results for their platforms. This will save us from dependency and maintenance of some central build farm with all the possible combinations of OS and CL implementations.
So, while continue integration servers main function is to run some command - "run tests" - on some schedule or by evens like source control commits.
In my opinion scheduling is not the main task to solve. With cl-test-grid I go bottom-up by providing an easy way to run tests on any platform and share results.
If we have several contributors who agree to run a simple command manually say once a month, it should be enough (for the beginning at least) to monitor the CL landscape and detect regressions.
After this will work, probably someone will want to configure a cron job for this command and don't spend any attention on this at alll, but on some platforms there might be limitations which will require starting tests manually. Today it's a bit early to think about this.
Best regards, - Anton