- asdf-devel - mailman3.common-lisp.net

Discussions about merge request 3
by Robert Goldman 24 Sep '16

24 Sep '16

Elias requests we merge this sooner, rather than later, as he has to go off-line for several days. I'll try to get this done tomorrow, but my review will probably be sub-ideal, since I really am not fully on top of this very complex chunk of code (aside from the tests, I've only really used it on SBCL and ACL). I'd particularly like feedback from Faré, obviously, and opinions about whether it's ready for a merge. Thanks, r

2 2

Enforcing assumptions
by Robert Goldman 24 Sep '16

24 Sep '16

I have been working on enforcing assumptions recently added to the ASDF docs. Specifically, that OPERATION instances only be created by MAKE-OPERATION and SYSTEMs only be parsed inside LOAD-ASD. I should have a merge request up for review soon, but find that it's more tricky than I expected, because we don't even play by the rules ourselves! Specifically, there are calls to MAKE-INSTANCE on OPERATIONs in the ASDF codebase itself. Cheers, r

3 9

Operations on package-inferred-system
by Eric Timmons 22 Sep '16

22 Sep '16

I *really* like using package-inferred-system for defining systems, but I often find it doesn't behave as I expect when performing operations on it. I'm curious if these behaviors are intended or if the implementation is just incomplete. I approach package-inferred-system as a way of automatically finding dependencies between files and on third-party systems for a given system I'm writing. Under the hood, I know it makes a new system for every source file, but I consider that just an implementation detail and that when I refer to the primary system, I really mean everything that logically belongs to that system and not just the source files declared in the system's :components option. (Perhaps this assumption just needs to be adjusted on my end). The difference between expectation and reality often manifests itself in two places for me: + When I'm trying to distribute my system with a non-monolithic operation. E.g. `(asdf:perform 'asdf:concatenate-source-op :some-package-inferred-system) results in only the files explicitly declared in the system's :components option being concatenated. (Similar things happen with compile-bundle-op). + `(asdf:load-system :some-package-inferred-system :force t)` results in only the files explicitly declared in the system's :components option being reloaded. I was planning to write my own system class to address these issues, but figured I should see if this is even intended behavior before doing that =). -Eric

2 3

Question about DEFSYSTEM-DEPENDS-ON
by Robert Goldman 22 Sep '16

22 Sep '16

Per my recent post to launchpad, I don't think dependency updating is the fatal flaw in D-D-O. D-D-O's bad interaction with the package system is worse. Specifically, D-D-O cannot be used to introduce ASDF extensions unless those ASDF extensions live in the ASDF package. Otherwise, by the time a DEFSYSTEM form is read, the enclosing package's imports are already established. This means that ASDF has no protection against namespace pollution, since extenders are practically forced to add symbols to ASDF. For example, we have a FIVEAM-ASDF extension that introduces a new defsystem class that has features for using 5AM as an implementation for the TEST-OP. But we cannot put a DEFSYSTEM-DEPENDS-ON for FIVEAM-ASDF in a system and reference FIVEAM-ASDF:FIVEAM-TESTER-SYSTEM as the system class, because the FIVEAM-ASDF package does not (in general) exist when the enclosing DEFSYSTEM is read. I can think of three ways of handling this: 1. Deprecate and remove D-D-O. This seems undesirable, since D-D-O is commonly used. 2. Replace the use of the lisp reader for processing DEFSYSTEM forms, and introduce some kind of parser. This seems like an enormous amount of work and difficult to get right. 3. In some way extend the keyword-processing features of DEFSYSTEM to allow it to reference symbols that are not only in the current package and ASDF, but also programmer-specified packages. Unfortunately, processing of keyword symbols in DEFSYSTEM is very case-specific -- we know particular locations where there use is permitted, and handle each one specifically. Without a cleaner way to handle this, we are likely to simply be backing ourselves into alternative #2. OTOH, if we *could* do this, it would be a great solution, because it would maintain the maximal amount of backwards-compatibility. This is the problem I would most like to see solved in ASDF.

3 5

Another follow-up from yesterday's doc reviews
by Robert Goldman 22 Sep '16

22 Sep '16

The comments in the file say that we *must* use load-asdf to load a .asd file, but not *WHY* we must do so. Why is this necessary? E.g., I have a makefile that lets me build a number of executables using my choice of lisp. Each time, to build the executable, it starts a lisp, loads the ASDF file (using --load or equivalent), and then uses asdf:make. I could use --eval (asdf:load-asdf "myfile.asd") but why is it mandated?

5 11

Reviewing bundle.lisp
by Robert Goldman 22 Sep '16

22 Sep '16

I have been making my first experiments with the bundle operations, to try to get a better understanding of this corner of ASDF. One thing I see in the new documentation is the following: (defclass bundle-op (basic-compile-op) + ;; NB: use of instance-allocated slots for operations is DEPRECATED + ;; and only supported in a temporary fashion for backward compatibility. + ;; Supported replacement: Define slots on program-system instead. ((build-args :initarg :args :initform nil :accessor extra-build-args) (name-suffix :initarg :name-suffix :initform nil) (bundle-type :initform :no-output-file :reader bundle-type) - #+(or clasp ecl) (lisp-files :initform nil :accessor extra-object-files))) + #+(or clasp ecl) (lisp-files :initform nil :accessor extra-object-files)) + (:documentation "base class for operations that bundle outputs from multiple components")) I think this is worth a discussion, before we get it set in stone. I can see two reasons why putting this in the system definition ("slots on program-system)" instead of in the operation might not be desirable: 1. It violates "the Faré principle" of the person who knows getting to specify. The programmer of the system doesn't know where the user wants the bundle delivered, but by putting this information in the system definition, it's the programmer, not the user, who gets to choose. Compare this with the way output translations work, where it's the user who chooses, not the developer. Another analogy: it's like "make install" without PREFIX. 2. This makes things cumbersome when you have a system that has the same codebase, but multiple different entry points. Consider a compression program that uses the same codebase, but wants to deliver 2 different executables, "squish" (compress) and "unsquish" (decompress). If we have the build destination hard-coded into the system definition, this is impossible. I was just experimenting with such a case myself, and ended up with a main system and 4 "slashy" systems, one for each entry point. Not a disaster, but not obviously a Good Thing, either. OTOH, putting slots on operations is a nuisance, because then we need to propagate those slot-values from operations to derived operations. I think it would be good to ponder this issue a bit. Cheers, r

2 1

operation.lisp
by Robert Goldman 22 Sep '16

22 Sep '16

I was pretty surprised to read this in the latest draft of operation.lisp: ;; A memoizing way of creating instances of operation. ;; All operations MUST created through this function. (defun make-operation (operation-class &rest initargs) "This function creates and memoizes an instance of OPERATION-CLASS...." This was pretty surprising to me, and is a change to the contract we make with programmers. I have certainly written a lot of code that calls MAKE-INSTANCE on operation classes. Are you requiring this because you want to rely on EQ-testing? Or....? If we want to require the use of MAKE-OPERATION, and rely on the operations being interned, we should do something to make it more apparent, and not have things quietly do something wrong, if this assumption is violated. E.g., we could have MAKE-OPERATION do something detectable before invoking MAKE-INSTANCE on an operation type (e.g., it could pass :make-operation t to MAKE-INSTANCE with the other initargs). Then if MAKE-INSTANCE is called on an operation class *without* the magical marker from MAKE-OPERATION, we catch it (in a :BEFORE method?) and we throw an error, and indicate the need to call MAKE-OPERATION. Reasonable-p?

2 1

merge requests
by Faré 20 Sep '16

20 Sep '16

Dear Robert, in the aftermath of my interaction with Daniel Kochmanski, I decided that part of the problem with Daniel making wrong guesses about the purpose of some pieces of asdf functionality was that said functionality was insufficiently documented. So I did a pass of going through all of asdf/defsystem and improve the documentation, moving a few pieces of code around and renaming a few internals, leading to merge request 24: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/24 Along the way, I found many small bits of code that could be improved, many pieces of obsolete or incomplete code that should be removed, etc., which led me to write merge request 23: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/23 I promised not to push to master anymore, so these are two merge requests. Whichever you merge first, I can rebase the other one afterwards (there unhappily will be small conflicts to resolve). This gave me a few ideas on how to tackle some of the more important bugs — I believe the biggest architectural issue is lack of proper modeling for dependencies while loading .asd files. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Politicians are like diapers: they must be changed often. And for the same reasons. [Also, adults don't need either of them. — Faré]

1 0

Merge request 21 pushed as 3.1.7.24
by Robert Goldman 19 Sep '16

19 Sep '16

More when my blasted Linux box is healthy again.

1 0

Status of master
by Robert Goldman 18 Sep '16

18 Sep '16

I am recording these PRIOR TO merging request 21, which Elias reports fixes a number of errors, making all tests pass on Linux. I will retests with 21 next. On the Mac I have multiple new failures: clisp-test.text -- test-require.script cmucl-test.text -- test-require.script sbcl-test.text -- test-require.script ecl-test.text -- ecl-make-image.script test-require.script ecl_bytecodes.text -- test-require.script For ECL make image the error is as follows: Undefined symbols for architecture x86_64: "_hello_string", referenced from: __eclGFF58509LfRJL_K3WEOq21 in hello.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) TEST ABORTED: Error code 1 when executing (RUN-PROGRAM "/usr/bin/clang" ("-o" "/Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello-tmp7HF8FK1E.fas" "-L/opt/local/lib/" "/private/var/folders/xq/ll1d80qs7hnbw3253yc1_x6m0000gn/T/eclinitkpC0s0.o" "/Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello.o" "-bundle" "-L/opt/local/lib" "-Wl,-headerpad_max_install_names" "-arch" "x86_64" "-L/opt/local/lib" "-Wl,-headerpad_max_install_names" "-arch" "x86_64" "-lecl" "-lpthread" "-lm")) Thought this might have been an architecture issue, but it appears not: $ file /Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello.o /Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello.o: Mach-O 64-bit object x86_64 TBH, ecl-make-image.script is obscure to me, so I don't have a clue what's going wrong here. Unfortunately, my Linux box in the office seems to have crashed over the weekend, so I don't have Linux or Windows results. I will try request 21 and see if that fixes things on the Mac.

1 0