On Mon, Feb 29, 2016 at 3:21 PM, Robert Goldman rpgoldman@sift.net wrote:
This isn't really a standard bootstrapping issue, and I think it's better to break the vicious cycle by simply booting the scripting engine out into a separate repository.
Indeed, one big issue is that the asdf under test shouldn't be used to compile asdf-tools. I believe what's currently provides a solution by building asdf-tools the first time with a hopefully stable asdf. Certainly, it all needs to be better documented / made more usable when upgrade is needed, etc.
Here's the (I believe simple) approach:
- Set up a repository that provides a *general* CL-scripting capability.
This is not going to happen for a while. Also, part of the appeal of CL scripting is the ability to use standard ASDF to load systems to extend the scripting infrastructure. If the scripting systems are preloaded and can't be upgraded, that's a problem. If they CAN be upgraded, using ASDF, we're back to the same problem. If they CANNOT be upgraded, that sucks. If they CAN be upgraded, but using something else than ASDF... then either that something else is better, and then we can drop ASDF but we have the same issue with that ASDF replacement; or it's not better and then it's yet another bad tool that makes CL scripting bad, too.
- Build that capability using only stable tools.
Nope. There is an essential bootstrapping issue, here. We can move it around, but it will remain.
- Provide in the ASDF repository code that simply uses the CL-scripting
capability. I.e., just provide CL scripts there, and not the scripting engine.
Well, first, a lot of the dependencies are used in the tests themselves. Also, a lot of the code is ASDF-specific, and yet is written in a modular way, which, in CL, means using ASDF. Even if somehow dependencies are magically included in the generic tool, we'll still want to use ASDF to load the asdf-specific code in asdf-tools.
At this point we would have broken the vicious cycle, and the CL scripting engine would play a role akin to perl, make, etc., in the ASDF development process.
No, we haven't broken the vicious circle. The only way to break the vicious circle for ASDF is if there's a non-ASDF tool being used to build the code, in which case THAT tool has the vicious circle. That said, if we adopt, e.g. Bazel as the standard way to build and test ASDF, then yes, we pushed back the bootstrapping issues onto Bazel.
Instead, right now we have things like ASDF requiring to test that it successfully be able to build a tool suite using a new ASDF capability.
Well, that's why I introduced an extra step, i.e. build_asdf_tools, which builds a ./build/asdf-tools binary, and that you should run with a stable asdf, so you can later use it to test an unstable asdf.
Note that this is what I'm suggesting *in the abstract*. Right now, the CL scripting framework is not, IMO, ready for prime time. I'm not willing to wrestle with undocumented, untested code as a precondition for simply fixing ASDF bugs.
I'm not saying that it's bad that CL scripting is at a primitive state now -- that's a state all software has to go through. All I'm saying is that CL scripting has a lot of promise, but it's not the kind of tested tool I require for ASDF development, and it's not going to get there (IMO) for some years yet.
asdf-tools depends on many systems, most of which are relatively old and stable and widely used in the community and/or well-tested, and not scripting-specific.
Three systems that I wrote may count as new, unstable, and scripting-related: inferior-shell, cl-launch, and cl-scripting.
inferior-shell, which provides a user-friendlier front-end to uiop:run-program, including string interpolation, rich fd redirections and piping, is moderately documented, and has a small test suite. I believe it has several users besides me, and the trivial parts used by asdf-tools are stable.
The cl-scripting system that handles don't-stop behavior akin to make -k on the other hand is relatively new and largely undocumented indeed. But it now has a test suite (that I added after you reported some bad bugs indeed), and the code is quite small and straightforward. ASDF is the only user so far (beside other personal scripts of mine) and I apologize for having things break on you, but I believe the bugs have now been fixed, and with tests added, and the parts used by ASDF should now be stable.
Finally, I believe cl-launch has several users besides me, and it has had an extensive test suite for many years. The test suite doesn't cover the relatively new cl-launch/dispatch code that asdf-tools relies upon, but that code is very small and simple, stable, and heavily used by myself (and by anyone else who might use cl-launch to produce multi-call binaries). There haven't been any reported problems with that code, so I presume it's stable for good.
So I am backing slowly away from it....
The cl-scripting functionality (now that it's debugged) is miles ahead of what make -k provided in terms of usability: it gives you a readable summary of all the failures that happened during a run of test-all-no-stop, with enough context to replay. Detailed instructions to replay the failures are also included in the log file — but the log file isn't vomited at the console as it is with the shell-based testing framework, so your testing status isn't lost in horrible testing output.
Yes, there have been issues in the past, and you've been bitten by a lot of them, especially as you've been testing on Windows and Mac which I don't have regular access to (but I could steal my wife's mac once in a while and install and configure a Windows VM at some point).
There are issues, notably bootstrapping issues, with using CL to test ASDF. I believe these issues are fixable, that most have been fixed, and that the result is better than depending on an unreadable hodge-podge of Make, shell, sed and perl, that also have to be installed and curated.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org The probability of the people in power being individuals who would dislike the possession and exercise of power is on a level with the probability that an extremely tender-hearted person would get the job of whipping-master on a slave plantation. — Frank H. Knight