- asdf-devel - mailman3.common-lisp.net

[asdf-devel] Problems with different versions of ASDF
by Robert Goldman 08 Apr '15

08 Apr '15

I have been running with ASDF from ACL and sometimes overloading it with ASDF from git. This gives me package errors like the following: ; Fast loading /Users/rpg/.cache/common-lisp/acl-8.2-macosx-x64/Users/rpg/lisp/asdf-context/set-context.fasl Error loading #P"/Users/rpg/clinit.cl": No package exists of name ASDF/LISP-ACTION. I suspect this happened because ASDF-context was compiled in an image with ASDF 3 loaded, and is being loaded in an image with ASDF 2.x (shipped by Franz with Allegro). Is there something I can do in systems that build on ASDF to avoid this from happening? Putting (:depends-on ASDF) in the relevant system (here my "asdf-context" system) does not solve the problem. It seems that we need a mechanism that says "if ASDF has changed its package structure, you must recompile." I understand the original motivation for subdividing the packages, but I think in future we need to be careful not to do this without an explicit plan to ensure it doesn't cause this kind of bug. It's an interesting unforeseen issue in the plan to be able to upgrade ASDF -- you need also to be able to downgrade! I.e., if I want to be able to test new ASDF, I may still need to work with colleagues who are using an older, implementation-provided version. For now, I can simply blow away my cache, and I expect that will solve the problem, but it's something to consider. thanks, r

2 1

[asdf-devel] Why Lisp is Now an Acceptable Scripting Language
by Faré 08 Apr '15

08 Apr '15

My cl-launch demo: cl -sp lisp-stripper -i '(print-loc-count "asdf.lisp")' Source code for lispwc: #!/usr/bin/cl -sp lisp-stripper -E main (defun main (argv) (if argv (map () 'print-loc-count argv) (print-loc-count *standard-input*))) And of course: for l in sbcl ccl clisp cmucl ecl abcl \ scl allegro lispworks gcl xcl ; do cl-launch -l $l -i \ '(format t "'$l': ~S~%" `#5(1 ,@`(2 3)))' \ 2>&1 | grep "^$l:" # LW, GCL are verbose done Backstory: After recent exposure to Python at work, I grew an intense dislike for that language, and realized that I could do everything in CL instead, and that the remaining stumbling blocks to using CL as a "scripting language" were quite small — just missing the ability to put #!/usr/bin/cl in a script, and there you go. And so, I added just this capability to cl-launch, improved it to have some of the features of buildapp, and made sure it works without a hitch on each of the 11 implementations with a command-line interface. This in turn requires updating ASDF and UIOP, and so here we are with ASDF 3.1.0.86. Then, I found that I had to take the word out, and started editing that way a paper that I'm preparing a paper for ELS 2014 (or else ILC 2014): ASDF3, or Why Lisp is Now an Acceptable Scripting Language It's incredible how having to explain the code forces you to make it actually simpler to use. Hence new option -sp in cl-launch, to load a system and change the current package, all in one go. Hence also having cl-launch build software with a sequence of --system --load --eval instead of a single setup file and toplevel system like before: people already understand a sequence of build commands. Also new in ASDF3: command line argument support was improved with a LispWorks workaround and a new argv0 function; with-input and with-output accept pathnames as designating a file to open; the bundle support was much improved and cleaned up; the upgrade mechanism was slightly improved to cope with the class refactoring; bugs were found and removed in pathname functions; a weird bug in SCL was worked around. Enjoy! TODO: I'm considering modifying the default behavior of cl-launch to launch a REPL and/or read commands from stdin, like a Unix shell does when given no script and no commands to run. I'm reserving that for another round of procrastination. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org To have a right to do a thing is not at all the same as to be right in doing it. — G.K. Chesterton

2 2

[asdf-devel] Merged Fare's RUN-PROGRAM topic branch
by Robert P. Goldman 08 Apr '15

08 Apr '15

I have merged Fare's RUN-PROGRAM topic branch and pushed it as 3.0.2.11. Passes all the tests for Fare and me on Linux and for me on Mac (modulo known bugs in bundle-op). My intention is to release this as 3.0.3 in the next few days, but I would like to make it available for people to play with first. I'm of course happy for you all to run the test suite on your own machines and implementations (contact me if you need help setting this up), but even more, I'd be obliged if you were to test this on your own systems (again, ask for advice if you need help setting up to do this). I'd particularly urge you to try this on systems where you have made your own extensions (new component or operation classes), or where you reach out into the environment to do something else (e.g., run "make" or the C compiler for FFI). Those would be the most likely to expose bugs in the new facilities. Best, r

2 1

[asdf-devel] Pushed 3.0.2.8
by Robert Goldman 08 Apr '15

08 Apr '15

Tweaked the bundle op so that writing DLLs on Mac generates files with "dylib" as extension instead of "so". To do so, I added os-macosx-p to UIOP. Cheers, r -- Robert P. Goldman Principal Scientist Smart Information Flow Technologies (d/b/a SIFT, LLC) 211 N. First St., Suite 300 Minneapolis, MN 55401 Voice: (612) 326-3934 Email: rpgoldman(a)SIFT.net

1 0

[asdf-devel] I think I have a case where ASDF catching warnings hurts...
by Robert P. Goldman 08 Apr '15

08 Apr '15

Here's a case we have: System uses bad old version of FiveAM. BOV5A signals ARNESI:DEFLOOKUP-REDEFINITION because of bug in EVAL-WHEN logic. We put our system's compilation inside Jenkins, failing on warnings. Jenkins fails. We see error. We wrap call to LOAD-SYSTEM in (handler-bind ((arnesi:deflookup-redefinition #'(lambda (e) (muffle-warning e))) (asdf:load-system "our-system")) This does not fix the problem. I believe that this is because ASDF is trapping *ALL* warnings, hanging onto them, and then resignaling them, wrapped in new ASDF condition objects. So our ARNESI:DEFLOOKUP-REDEFINITIONs are all gone, and can't be used for muffling. It seems like this is going to bork any system that wants to muffle specific warnings, or handle specific errors. Not so? Or have I misunderstood somehow? Thanks, r

2 1

[asdf-devel] Regression on Allegro Common Lisp
by Robert P. Goldman 08 Apr '15

08 Apr '15

I pulled an update this afternoon, and now on ACL 9.0 on Mac OS X I get a failure on asdf-pathname-test.script I get an error as follows: Above backtrace due to this condition: attempt to take the length of a non-sequence: :UNSPECIFIC I get an enormous backtrace, but this is I think the direct call that gets an error: (LET ((STREAM (OPEN . #16=((ASDF/COMPONENT:COMPONENT-PATHNAME ASDF-PATHNAME-TEST::FILE) :DIRECTION :OUTPUT :IF-EXISTS :RENAME-AND-DELETE :IF-DOES-NOT-EXIST :ERROR))) (#12=#:WITH-OPEN-FILE-ABORT-398 . #61=(T))) (UNWIND-PROTECT (MULTIPLE-VALUE-PROG1 (PROGN . #17=((PRINT ASDF-PATHNAME-TEST::START-TIME STREAM) T)) (SETQ #12# . #58=(NIL))) (WHEN (STREAMP STREAM) (CLOSE STREAM :ABORT #12#)))) More as I find out more. Best, r

3 4

Re: trying some ASuDF tests on Windows (was: Re: [asdf-devel] Volunteer for LispWorks testing?)
by Dave Cooper 08 Apr '15

08 Apr '15

Fare wrote: > > The test-bundle.script output suggests that, on Windows, there might > be a bug in ccl:fasl-concatenate. Please create a reduced test, and > open a bug against CCL if that is indeed the case, using > http://trac.clozure.com/ccl/newticket > Ok, I went ahead and opened a CCL bug, which was (almost immediately) reported as fixed in the subversion trunk and the ticket (#1109) was closed. Then I built a CCL from the trunk (which is 1.10-dev), and indeed this failure goes away when using 1.10-dev. See the results in the 2013-09-05 folder in the link below. > > The test-multiple.script failure reveals an error in run-program, that > looks like it does a (cons "cmd" argument) instead of a (list "cmd" > argument). Same failure in test-run-program.script. > I tried changing the code in uiop/run-program.lisp from: #+clozure (cons "cmd" (strcat "/c" command)) to #+clozure (list "cmd" (strcat "/c" command)) but still get some failures in test-run-program.script. Maybe they are different failures. See results below. https://www.dropbox.com/sh/jc2cqwpkp06dupm/1FZ_YlVvVW/cl/tests/asdf/output By the way, the run-program tests seem to use the command "ln" for testing -- this command isn't there by default on Windows, is it? The issue with ";" vs. ":" for path separator I'm not sure exactly where to fix, so I leave that alone for the time being (anyone else care to jump on it?) --- apparently that's not CCL-specific, but a general Windows asdf testing issue? On Wed, Sep 4, 2013 at 5:07 PM, Robert P. Goldman <rpgoldman(a)sift.info>wrote: > Dave Cooper wrote: > > > > Hi Robert, > > > > [also cc'ing this to asdf-devel in case anyone else has feedback] > > > > Ok, shifting gears a little bit here first... > > > > I'm trying a few test runs of the ASDF test suite on Windows. Platforms > > on Windows which I have available to test right now are: Allegro CL > > (several variations), and CCL. > > > > To prepare to run the tests on Windows, I installed cygwin and cygwin's > > make. Then to run the tests, I do (for lack of having this more > > automated so far): > > > > 1. set ASDF_TEST_LISPS appropriately. > > > > 2. Edit run-tests.sh to point to appropriate Lisp executables. > > You actually shouldn't need to do that -- there is one environment > variable for each of the executables, and you can set those, if the > default values don't work for you. > > > > > 3 Invoke "make test-all-no-upgrade" from the cygwin command shell. > > > > Let's try CCL first. Here are the outputs: > > > > > https://www.dropbox.com/sh/jc2cqwpkp06dupm/-vvd5gFwTk/cl/tests/asdf/output/… > > > > Summary: 42 out of 47 pass (5 failures). > > > > I can try to dive into these further, but I thought I would post the > > results here first in case anything is obvious to anyone. > > > > Incidentally, I have been running ASDF in production with Allegro CL for > > some time now (including asdf:monolithic-fasl-op) and it has been > > working well. There are some startup glitches for running ASDF tests on > > Windows/Allegro right now. After the CCL issue is resolved I will return > > to trying to get the tests to run for Allegro on Windows. > That would be great. > > By the way, the bundle tests aren't working for me on a couple of other > lisps. > > cheers, > r > > > -- My Best, Dave Cooper, Genworks Support david.cooper(a)genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699

4 17

[asdf-devel] In defense of ASDF & Semantic versioning
by Robert P. Goldman 08 Apr '15

08 Apr '15

Anton Vodonosov wrote: > The manual says: > "the :version, :author, description and other [defsystem] fields are not required but they provide documentation and information for people that want to use this system. > It also says in the section about asdf:operate: > "If a version argument is supplied, then operate also ensures that the system found satisfies it using the version-satisfies method." > > From some other places in the manual we can even guess how version-satisfies work. > > Still, I think it was too much to expect from closer-mop to encode > API compatibility information in the version identifier. > Note also, in 2009 ASDF didn't consider version "0.6" as satisfying > requirement for "0.55": Yes, that's because the version scheme in ASDF is not a sequence of period-separated integers, but a sequence of period-separated strings. That was in ASDF 1, and we didn't dare mess with it. Pascal was very kind about this oddity, and has tweaked his revision numbering to fit ASDF's constraints. > http://lists.common-lisp.net/pipermail/moptilities-devel/2009-December/date… When Faré wrote up the design rationale for ASDF, one of the principles was that ASDF should ask the right person for the right information. For example, the library should not dictate where files go -- that's the job of the library's installer (user). The library itself knows where its components are *relative to the installation* -- that's information the library implementor will have, which the installer will not have. Similarly, if I am the library supplier, I am the one who knows when I have changed the API incompatibly. That is why major component version changes were intended *not* to satisfy version requirements that have different major components. This is reasonable, because it provides a channel of information from the library supplier, that can be checked automatically, and if the downstream is still compatible, it should be a relatively easy fix (but is it? I don't know if there's a good way to say "I'll accept version 0.55+ or 1.0+"). The only other option would be to have the library client provide both upper and lower bounds on the version numbers that the client will accept. But, except for cases of *known* upgrade incompatibility, this is information that the library client simply cannot have. The rules here are simple and easy to understand: if you change the API in an incompatible way, bump the major version number. Yes, this rules out the false modestly of version numbers like 0.0.145, which looks like an alpha, but in fact turns out to be the fifth full release, but that's a sacrifice we can all live with! ;-) As my own devil's advocate, the counter-argument is that the state of the art in CL libraries is so poor that even getting a version number is unusual. But this seems to be a counsel of despair: it says that since we have a bad state of affairs now, we are doomed to live with it forever. I suppose I could take a little while and write up a candidate "versioning systems with ASDF" node for the ASDF manual, and push it for consideration by the community. > > Also, as we speak about versioning, I have been trying to use semantic > versioning as described at http://semver.org/ and I don't think it is > a silver bullet - it doesn't solve all problems. Isn't the subtitle of /The Mythical Man-Month/ "No Silver Bullet"? Yes, this doesn't solve every problem, but it might give you a useful warning if your upstream has changed under you.... Pascal's case is an odd one, because the bump to 1.0 is, in some sense, a false positive. OTOH, it's not really so bad to have to take a look at a library that has seen no commits in three years. Cheers, r

9 27

[asdf-devel] One failure on ASDF 3.1.0.70 on Allegro/Windows
by Dave Cooper 08 Apr '15

08 Apr '15

Sorry to be late to the party again with Windows testing... So far I'm seeing a consistent failure for every Allegro variant on Windows for test-sysdef-asdf. Here is the error: These two expressions fail comparison with equal: (asdf-test::system-lisp-files :uiop) evaluates to ("uiop\\package.lisp" "uiop\\common-lisp.lisp" "uiop\\utility.lisp" "uiop\\os.lisp" "uiop\\pathname.lisp" "uiop\\filesystem.lisp" "uiop\\stream.lisp" "uiop\\image.lisp" "uiop\\run-program.lisp" "uiop\\lisp-build.lisp" "uiop\\configuration.lisp" "uiop\\backward-driver.lisp" "uiop\\driver.lisp") (asdf-test::makefile-lisp-files "driver-files") evaluates to ("uiop/package.lisp" "uiop/common-lisp.lisp" "uiop/utility.lisp" "uiop/os.lisp" "uiop/pathname.lisp" "uiop/filesystem.lisp" "uiop/stream.lisp" "uiop/image.lisp" "uiop/run-program.lisp" "uiop/lisp-build.lisp" "uiop/configuration.lisp" "uiop/backward-driver.lisp" "uiop/driver.lisp") Script failed -- My Best, Dave Cooper, Genworks Support david.cooper(a)genworks.com, dave.genworks.com(skype) USA: 248-327-3253(o), 1-248-330-2979(mobile) UK: 0191 645 1699

7 27

[asdf-devel] Releasing asdf 3.1.1 ?
by Faré 08 Apr '15

08 Apr '15

Dear Robert, do you think we can release this year? All tests pass for me on Linux, and I believe all tests were passing for Dave Cooper on Windows. There has been very light development this past month, with no intent for further development ahead, but notable new features and bug fixes since the last release in October. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org A student who changes the course of history is probably taking an exam.

5 21