September 2016 - asdf-devel - mailman3.common-lisp.net

Recent disappearance of asdf:bundle-system.
by Jean-Claude Beaudoin 18 Oct '16

18 Oct '16

Hello ASDF devs, I noticed recently that asdf/bundle:bundle-system has disappeared from ASDF. MKCL is/was a user of that function as a convenient entry point to the ASDF bundle facility. My impression was that, along the lines of asdf:load-system and asdf:compile-system, asdf:bundle-system was playing a (small but) useful role as a convenience function. I will most probably simply have to fold that function inside the build code of MKCL if it is to be removed from ASDF for good. Yet, to see a publicly exported function disappear between version 3.1.7.10 and 3.1.7.12 to most likely result in a 3.1.8 release gives pause. Are we really seeing here a public API code breaking modification being done between two patch level releases? BTW, there seems to have been a few other MKCL related breakage issues introduced by the recent wave of changes. I will have to examine each one in turn... Cheers, JCB

6 12

Window of inaction through Tuesday
by Robert Goldman 30 Sep '16

30 Sep '16

I have personal things (all good) that will keep me busy through the end of Monday. I will start to spin back up on ASDF activities Tuesday morning. Have a good weekend, everyone. r

1 0

Locations
by Greg Bennett 26 Sep '16

26 Sep '16

Good morning from Greg Bennett. I am trying to learn enough asdf to port some systems which now use mk:defsystem. Running under Linux Mint 17.2 and sbcl (or clozure) and using asdf3.1.7, My (current) difficulty is in understanding the specification of the locations of source and binary files. I *suspect* that I fall among those described in Section 9.2: "This incompatibility shouldn’t inconvenience many people. Indeed, few people use and customize these packages; these few people are experts who can trivially adapt to the new configuration. Most people are not experts, could not properly configure these features (except inasmuch as the default configuration of |common-lisp-controller| and/or |cl-launch| might have been doing the right thing for some users), and yet will experience software that “just works”, as configured by the system distributor, or by default" I'm not an expert, and I am very pleased in somethings "just work", but I am inconvenienced. I find code such as (defsystem zot :source-pathname (etc..) :binary-pathname (etcc..) :components (( ....) clear and helpful. I have tried to understand Sections 8 and 9 of the ASDF manual, indeed I thought I had Section 8 on source locations straight and looked ahead into Section 9, only find myself without any usable idea of how to proceed. I have scanned through assorted .asd files in the hope of seeing a working example, but none of them use asdf other than as a mechanism for specifying dependencies. I can define a system pqr with zot.asd, stored in the same directory as the source files. I can run (asdf:operate 'asdf:compile-op 'pqr) and have all the .fasl files squirreled away down in ~/.cache/common-lisp/lots-of-other-depths/. I should appreciate it very much if someone could point me to examples of working code for specifying source and binary locations. Thanks for any and all assistance. Cheers /Greg Bennett

3 2

Just pushed 3.1.7.26: RUN-PROGRAM extensions
by Robert Goldman 26 Sep '16

26 Sep '16

I have just pushed 3.1.7.26, which features merge of Elias Pipping's extensive re-engineering of UIOP:RUN-PROGRAM, and addition of UIOP:LAUNCH-PROGRAM. Thanks to Elias for an enormous amount of work. Thanks also to Faré for his extensive reviewing, to Daniel Kochmanski for his extensive work on UIOP (and apologies for my inability to figure out how to put an accent properly in his name!). RUN-PROGRAM has changed quite a bit, and whenever we move to a new release, I propose to bump to 3.2, to make it easier for people to detect these extensions. Best, Robert

2 2

Discussions about merge request 3
by Robert Goldman 24 Sep '16

24 Sep '16

Elias requests we merge this sooner, rather than later, as he has to go off-line for several days. I'll try to get this done tomorrow, but my review will probably be sub-ideal, since I really am not fully on top of this very complex chunk of code (aside from the tests, I've only really used it on SBCL and ACL). I'd particularly like feedback from Faré, obviously, and opinions about whether it's ready for a merge. Thanks, r

2 2

Enforcing assumptions
by Robert Goldman 24 Sep '16

24 Sep '16

I have been working on enforcing assumptions recently added to the ASDF docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD. I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE on OPERATIONs in the ASDF codebase itself. Cheers, r

3 9

Operations on package-inferred-system
by Eric Timmons 22 Sep '16

22 Sep '16

I *really* like using package-inferred-system for defining systems, but I often find it doesn't behave as I expect when performing operations on it. I'm curious if these behaviors are intended or if the implementation is just incomplete. I approach package-inferred-system as a way of automatically finding dependencies between files and on third-party systems for a given system I'm writing. Under the hood, I know it makes a new system for every source file, but I consider that just an implementation detail and that when I refer to the primary system, I really mean everything that logically belongs to that system and not just the source files declared in the system's :components option. (Perhaps this assumption just needs to be adjusted on my end). The difference between expectation and reality often manifests itself in two places for me: + When I'm trying to distribute my system with a non-monolithic operation. E.g. `(asdf:perform 'asdf:concatenate-source-op :some-package-inferred-system) results in only the files explicitly declared in the system's :components option being concatenated. (Similar things happen with compile-bundle-op). + `(asdf:load-system :some-package-inferred-system :force t)` results in only the files explicitly declared in the system's :components option being reloaded. I was planning to write my own system class to address these issues, but figured I should see if this is even intended behavior before doing that =). -Eric

2 3

Question about DEFSYSTEM-DEPENDS-ON
by Robert Goldman 22 Sep '16

22 Sep '16

Per my recent post to launchpad, I don't think dependency updating is the fatal flaw in D-D-O. D-D-O's bad interaction with the package system is worse. Specifically, D-D-O cannot be used to introduce ASDF extensions unless those ASDF extensions live in the ASDF package. Otherwise, by the time a DEFSYSTEM form is read, the enclosing package's imports are already established. This means that ASDF has no protection against namespace pollution, since extenders are practically forced to add symbols to ASDF. For example, we have a FIVEAM-ASDF extension that introduces a new defsystem class that has features for using 5AM as an implementation for the TEST-OP. But we cannot put a DEFSYSTEM-DEPENDS-ON for FIVEAM-ASDF in a system and reference FIVEAM-ASDF:FIVEAM-TESTER-SYSTEM as the system class, because the FIVEAM-ASDF package does not (in general) exist when the enclosing DEFSYSTEM is read. I can think of three ways of handling this: 1. Deprecate and remove D-D-O. This seems undesirable, since D-D-O is commonly used. 2. Replace the use of the lisp reader for processing DEFSYSTEM forms, and introduce some kind of parser. This seems like an enormous amount of work and difficult to get right. 3. In some way extend the keyword-processing features of DEFSYSTEM to allow it to reference symbols that are not only in the current package and ASDF, but also programmer-specified packages. Unfortunately, processing of keyword symbols in DEFSYSTEM is very case-specific -- we know particular locations where there use is permitted, and handle each one specifically. Without a cleaner way to handle this, we are likely to simply be backing ourselves into alternative #2. OTOH, if we *could* do this, it would be a great solution, because it would maintain the maximal amount of backwards-compatibility. This is the problem I would most like to see solved in ASDF.

3 5

Another follow-up from yesterday's doc reviews
by Robert Goldman 22 Sep '16

22 Sep '16

The comments in the file say that we *must* use load-asdf to load a .asd file, but not *WHY* we must do so. Why is this necessary? E.g., I have a makefile that lets me build a number of executables using my choice of lisp. Each time, to build the executable, it starts a lisp, loads the ASDF file (using --load or equivalent), and then uses asdf:make. I could use --eval (asdf:load-asdf "myfile.asd") but why is it mandated?

5 11

Reviewing bundle.lisp
by Robert Goldman 22 Sep '16

22 Sep '16

I have been making my first experiments with the bundle operations, to try to get a better understanding of this corner of ASDF. One thing I see in the new documentation is the following: (defclass bundle-op (basic-compile-op) + ;; NB: use of instance-allocated slots for operations is DEPRECATED + ;; and only supported in a temporary fashion for backward compatibility. + ;; Supported replacement: Define slots on program-system instead. ((build-args :initarg :args :initform nil :accessor extra-build-args) (name-suffix :initarg :name-suffix :initform nil) (bundle-type :initform :no-output-file :reader bundle-type) - #+(or clasp ecl) (lisp-files :initform nil :accessor extra-object-files))) + #+(or clasp ecl) (lisp-files :initform nil :accessor extra-object-files)) + (:documentation "base class for operations that bundle outputs from multiple components")) I think this is worth a discussion, before we get it set in stone. I can see two reasons why putting this in the system definition ("slots on program-system)" instead of in the operation might not be desirable: 1. It violates "the Faré principle" of the person who knows getting to specify. The programmer of the system doesn't know where the user wants the bundle delivered, but by putting this information in the system definition, it's the programmer, not the user, who gets to choose. Compare this with the way output translations work, where it's the user who chooses, not the developer. Another analogy: it's like "make install" without PREFIX. 2. This makes things cumbersome when you have a system that has the same codebase, but multiple different entry points. Consider a compression program that uses the same codebase, but wants to deliver 2 different executables, "squish" (compress) and "unsquish" (decompress). If we have the build destination hard-coded into the system definition, this is impossible. I was just experimenting with such a case myself, and ended up with a main system and 4 "slashy" systems, one for each entry point. Not a disaster, but not obviously a Good Thing, either. OTOH, putting slots on operations is a nuisance, because then we need to propagate those slot-values from operations to derived operations. I think it would be good to ponder this issue a bit. Cheers, r

2 1