- asdf-devel - mailman3.common-lisp.net

Question about version in submitting patches
by Mark Evenson 27 Nov '20

27 Nov '20

I’m a little unsure of whether the “Committee for Ongoing and Perpetual ASDF maintenance” (hi Robert!) wishes us to include the results of “<file:bin/bump-version>” in submitted patches. I have a small ABCL-specific patch dealing with UIOP:PARSE-UNIX-NAMESTRING when loading system definitions from zip archives for which I have used bump-version to denote as version “3.3.4.0.1”. I’ve not quite finished my testing to ensure that previous versions of ABCL work well with it, but when I do, do you wish me to include the use of “bump-version” with the patch or is that something the Committee prefers to do on its own? yours, Mark -- "A screaming comes across the sky. It has happened before but there is nothing to compare to it now."

6 6

Unexpected error ..
by gwbennett 26 Nov '20

26 Nov '20

Good morning from Greg Bennett who is running sbcl 2.0.10 under linux mint 19.2 64-bit and using asdf 3.3.6 with mcclim from mcclim-20201016-git via quicklisp. I am working on a large project and wish to capture the results of a trial compile in a file so that I can check notes, warnings, errors, etc.. Such a compile essentially loops over a set of asdf load-systems. Not unnaturally for me, I make mistakes, and sometimes I get blocked by an unexpected error which I believe I have managed to isolate. I am quite unsure about the locus of the unexpected error .. asdf ? sbcl ? mcclim ? their interaction ? so I am starting with asdf-devel. I am uncertain how much to include by way of source files, so i have opted to summarise things here first. New sbcl 2.0.10 in emacs listener Load a set-up-file to ensure I am using asdf 3.3.6 [I gather this is significant for sbcl]. And some of my own logging tools. Two files attached. Load mcclim Running (in-package :cl-user) (a-l "mcclim" "doit" (doit) "Z" :base-name "/home/gwbennett/mcclim-code/") which has the effect of writing the results to a file, in this case /home/gwbennett/LOGS/MCCLIM/MCCLIM2020-lsbcl-to-Z-26-NOV-at-14:5:2.lsp" The process exits with: ; wrote /home/gwbennett/.cache/common-lisp/sbcl-2.0.10-linux-x64/home/gwbennett/mcclim-code/check-qmb/one/test-qmb-tmpJU0JWO19.fasl ; compilation finished in 0:00:00.051 ; No value Now suppose I do something which contains an error, such as CL-USER> (asdf-version) Since (asdf-version) is not in the cl-user package, this is an error. What I get back is a different error message: #<SB-SYS:FD-STREAM for "file /home/gwbennett/LOGS/MCCLIM/MCCLIM2020-lsbcl-to-Z-26-NOV-at-14:5:2.lsp" {1008BEC8A3}> is closed [Condition of type SB-INT:CLOSED-STREAM-ERROR] Which is my unexpected error. Sometimes after ABORT (twice), this error can block me from defining any new function or loading a new function .. all I receive is a repeat of this message. Thank you for your patience Cheers Greg Bennett

2 1

Looking for some ECL help...
by Robert Goldman 19 Nov '20

19 Nov '20

I have pulled in a patch independently derived by Luis Oliveira and Martin Simmons that fixes an issue with launch program. Unfortunately, the test I developed for it *reveals* an issue in ECL. Here's the report: https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/144 If anyone with ECL expertise has time to give a look into this, I would be grateful. Even if you just have time to tell me how to deal with ECL on this -- it's an issue of ECL not being able to read a pipe stream. Best, R

1 0

Jenkins expertise?
by Robert Goldman 16 Oct '20

16 Oct '20

Does anyone have expertise configuring Jenkins to run against a remote Gitlab install? If so, would you please reach out to me? I have *finally* restored some of my CI support for ASDF, but I don't know how to set up my Jenkins to run against merge requests on Gitlab (if that is even possible). I have a backlog of patches that I would like to get evaluated and merged, but could use some help with the automation. Thanks!

7 22

Don't follow symlinks in uiop:directory* on ECL
by Moritz Petersen 09 Oct '20

09 Oct '20

Hi, I recently noticed that directory* follows symlinks on ECL, whereas it would be desirable if it didn't. ECLs directory* function follows symlinks by default, but can be instructed not to. Please see below for the tweak and a suggested test case, which I based on test-symlink-loop.script. Thank you, Moritz Petersen

1 0

A CFFI -ASDF integration bug
by Ilya Perminov 28 Aug '20

28 Aug '20

Hi, I found a ASDF-related CFFI bug a couple of days ago. Can anyone think of a good way of fixing it? A method in grovel/asdf.lisp adds output files of process-op to output files of compile-op: ;;; Declare the .o and .so files as compilation outputs, ;;; so they get picked up by bundle operations. #.(when (version<= "3.1.6" (asdf-version)) '(defmethod output-files ((op compile-op) (c wrapper-file)) (destructuring-bind (generated-lisp lib-file c-file o-file) (output-files 'process-op c) (declare (ignore generated-lisp c-file)) (multiple-value-bind (files translatedp) (call-next-method) (values (append files (list lib-file o-file)) translatedp))))) As a result inputs and outputs of the ops look like this: process-op: input: wrapper-file output: bindings-file.lisp file.c FILE.O FILE.SO compile-op: input: bindings-file.lisp output: bindings-file.fasl FILE.O FILE.SO The problem is that process-op generates file.o and file.so before it generates bindings-file.lisp. Thus compile-op gets re-executed all the time, because its output files file.o and file.so are always older than its input file bindings-file.lisp. And an execution of the compile-op does not change anything, because it does not really generate .o and .so. One way to fix it would be to "touch" .o and .so to get the right order of file modification times, but there may be a better way. CFFI bug report: https://bugs.launchpad.net/cffi/+bug/1889491 Thanks, Ilya

4 11

Re: A CFFI -ASDF integration bug
by Robert Goldman 04 Aug '20

04 Aug '20

On 4 Aug 2020, at 17:00, Ilya Perminov wrote: > I think there is a problem with the :around method approach - it is > not safe for an ASDF extension to define methods on ASDF's > operations/components, because other ASDF extensions may want to do > the same thing. Imagine there is CFFI* and I want to use both CFFI and > CFFI*. That is true -- that's why I always try to have either my own operation class or my own component class, or both. So to do this, you would want to have your own subclass of `system` that you wish to have work this way (e.g., `cffi-extended-system`) -- you generally do not want to add methods to the standard classes in ASDF, both because of potential collisions, and because you don't know what else they might interfere with. > > On Tue, Aug 4, 2020 at 2:35 PM Robert Goldman <rpgoldman(a)gmail.com> > wrote: >> >> On 4 Aug 2020, at 16:18, Ilya Perminov wrote: >> >> The around method will look something like this >> >> (defmethod input-files :around ((o asdf/bundle::gather-operation) (c >> system)) >> (unless (eq (asdf/bundle::bundle-type o) :no-output-file) >> (append (call-next-method) >> (remove-if-not >> (asdf/bundle::pathname-type-equal-function >> (asdf/bundle::bundle-pathname-type >> (asdf/bundle::gather-type o))) >> (mappend (lambda (c) (output-files >> 'process-op c)) >> (asdf/component:sub-components c >> :type 'wrapper-file)))))) >> >> To my taste it knows too much about ASDF guts, but maybe it is not >> too bad. >> >> I'm looking at the code now, and >> >> bundle-type is exported from asdf/bundle (but not asdf) (it could use >> a docstring if you can propose one) >> >> So is bundle-pathname-type >> >> pathname-type-equal-function is not exported, but it's a trivial >> combination of pathname-type and equalp >> >> gather-type is the only "really internal" function. (A block comment >> would be nice here, too.) >> >> gather-operation is not exported, but I think that's probably wrong >> on our part: I don't believe there should be hidden operations. If I >> get around to it, I will probably export it from the asdf (really >> asdf/interface) package, unless someone has a good argument for it >> continuing to be internal. >> >> So I don't think that is a lot of ASDF internals, especially not for >> customizing the ASDF protocol.

1 0

Re: A CFFI -ASDF integration bug
by Ilya Perminov 04 Aug '20

04 Aug '20

The around methed will look something like this (defmethod input-files :around ((o asdf/bundle::gather-operation) (c system)) (unless (eq (asdf/bundle::bundle-type o) :no-output-file) (append (call-next-method) (remove-if-not (asdf/bundle::pathname-type-equal-function (asdf/bundle::bundle-pathname-type (asdf/bundle::gather-type o))) (mappend (lambda (c) (output-files 'process-op c)) (asdf/component:sub-components c :type 'wrapper-file)))))) To my taste it knows too much about ASDF guts, but maybe it is not too bad.

2 3

Re: A CFFI -ASDF integration bug
by Ilya Perminov 04 Aug '20

04 Aug '20

On Tue, Aug 4, 2020 at 12:51 PM Robert Goldman <rpgoldman(a)sift.info> wrote: > I don't claim to understand this process, but wouldn't it be possible for you to make your own input-files :around method for gather-operation that would collect the outputs from the process-op's and add them to what you want? I think it is a reasonable option, at least I like it more than what CFFI does now. input-files for gather-operation is defined on SYSTEM, so the around method will have to fetch all WRAPPER-FILEs from a system. gather-type and probably some other functions, that the around method will need, are not exported from ASDF though. > That said, I can think of a simpler, and easier method, and that would be to override the operation-done-p method for the compile-op so that it knows that the .o and .so files are generated by the compile-op, and only pays attention to the relationship between bindings-file.lisp and bindings-file.fasl. I do not think it will work, because up-to-date check happens outside of operation-done-p. compute-action-stamp: (and all-present ;; if all filesystem effects are up-to-date up-to-date-p (operation-done-p o c) ;; and there's no invalidating reason. > One thing you didn't say was what it means that the compile-op is done over and over -- is it only compiling the bindings-file, or is it doing something to the object files? I don't believe it should change the .o and .so files, because those are already compiled by process-op, right? Yes, only the lisp file gets recompiled. compile-op does not do anything to the .o and .so files, they are just its fake outputs.

1 0

Apologies for slow progress...
by Robert Goldman 02 Aug '20

02 Aug '20

I'm afraid I have been very busy at work, but also the linux box I have been using as my Jenkins platform had a disk failure, taking it out of action. No data was lost, but my employer has been supplying this, and I have to wait in the queue, after business requirements, for the IT folks to get me a new server configured for this purpose. This has unavoidably hindered my work on ASDF. I'm open to suggestions about alternative resources for running these tests, but there are special challenges because I need to be able to put commercial software (Allegro CL and LispWorks) on any test server, which I think rules out using Travis. If anyone has hosting suggestions, please let me know: otherwise I'm afraid we'll be limping along till September, which is unfortunate, since we have a bunch of fixes and bug reports, and I would very much like to be able to get these wrapped up into a bug fix release. I hope you all are staying safe and healthy!

6 6