September 2016 - asdf-devel - mailman3.common-lisp.net

operation.lisp
by Robert Goldman 22 Sep '16

22 Sep '16

I was pretty surprised to read this in the latest draft of operation.lisp: ;; A memoizing way of creating instances of operation. ;; All operations MUST created through this function. (defun make-operation (operation-class &rest initargs) "This function creates and memoizes an instance of OPERATION-CLASS...." This was pretty surprising to me, and is a change to the contract we make with programmers. I have certainly written a lot of code that calls MAKE-INSTANCE on operation classes. Are you requiring this because you want to rely on EQ-testing? Or....? If we want to require the use of MAKE-OPERATION, and rely on the operations being interned, we should do something to make it more apparent, and not have things quietly do something wrong, if this assumption is violated. E.g., we could have MAKE-OPERATION do something detectable before invoking MAKE-INSTANCE on an operation type (e.g., it could pass :make-operation t to MAKE-INSTANCE with the other initargs). Then if MAKE-INSTANCE is called on an operation class *without* the magical marker from MAKE-OPERATION, we catch it (in a :BEFORE method?) and we throw an error, and indicate the need to call MAKE-OPERATION. Reasonable-p?

2 1

merge requests
by Faré 19 Sep '16

19 Sep '16

Dear Robert, in the aftermath of my interaction with Daniel Kochmanski, I decided that part of the problem with Daniel making wrong guesses about the purpose of some pieces of asdf functionality was that said functionality was insufficiently documented. So I did a pass of going through all of asdf/defsystem and improve the documentation, moving a few pieces of code around and renaming a few internals, leading to merge request 24: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/24 Along the way, I found many small bits of code that could be improved, many pieces of obsolete or incomplete code that should be removed, etc., which led me to write merge request 23: https://gitlab.common-lisp.net/asdf/asdf/merge_requests/23 I promised not to push to master anymore, so these are two merge requests. Whichever you merge first, I can rebase the other one afterwards (there unhappily will be small conflicts to resolve). This gave me a few ideas on how to tackle some of the more important bugs — I believe the biggest architectural issue is lack of proper modeling for dependencies while loading .asd files. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Politicians are like diapers: they must be changed often. And for the same reasons. [Also, adults don't need either of them. — Faré]

1 0

Merge request 21 pushed as 3.1.7.24
by Robert Goldman 18 Sep '16

18 Sep '16

More when my blasted Linux box is healthy again.

1 0

Status of master
by Robert Goldman 18 Sep '16

18 Sep '16

I am recording these PRIOR TO merging request 21, which Elias reports fixes a number of errors, making all tests pass on Linux. I will retests with 21 next. On the Mac I have multiple new failures: clisp-test.text -- test-require.script cmucl-test.text -- test-require.script sbcl-test.text -- test-require.script ecl-test.text -- ecl-make-image.script test-require.script ecl_bytecodes.text -- test-require.script For ECL make image the error is as follows: Undefined symbols for architecture x86_64: "_hello_string", referenced from: __eclGFF58509LfRJL_K3WEOq21 in hello.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) TEST ABORTED: Error code 1 when executing (RUN-PROGRAM "/usr/bin/clang" ("-o" "/Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello-tmp7HF8FK1E.fas" "-L/opt/local/lib/" "/private/var/folders/xq/ll1d80qs7hnbw3253yc1_x6m0000gn/T/eclinitkpC0s0.o" "/Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello.o" "-bundle" "-L/opt/local/lib" "-Wl,-headerpad_max_install_names" "-arch" "x86_64" "-L/opt/local/lib" "-Wl,-headerpad_max_install_names" "-arch" "x86_64" "-lecl" "-lpthread" "-lm")) Thought this might have been an architecture issue, but it appears not: $ file /Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello.o /Users/rpg/lisp/asdf/build/fasls/ecl-16.1.2-unknown-macosx-x64/asdf/test/ecl-make-image/hello.o: Mach-O 64-bit object x86_64 TBH, ecl-make-image.script is obscure to me, so I don't have a clue what's going wrong here. Unfortunately, my Linux box in the office seems to have crashed over the weekend, so I don't have Linux or Windows results. I will try request 21 and see if that fixes things on the Mac.

1 0

Needed change in governance
by Robert Goldman 18 Sep '16

18 Sep '16

I can no longer be the maintainer of ASDF unless I gain some control over the contents of the repository. Right now, I am just reacting to changes other people are making. This is ok if I am in the loop, but if other developers are simply wading into the repository and committing changes that I have no idea are coming, then I cannot do my job. The last straw for me was last night's 3.1.7.23 commits, which completely blind-sided me and also invalidated work-in-progress I had outstanding. So, we need to either 1. develop some informal means of clearing coming commits through me, perhaps using this mailing list; 2. lock out other developers and change the main repository only through merge requests (this is not my favored alternative, as it requires more busy work on my part); or 3. get a new maintainer. I decline to continue under the status quo.

3 4

RFC: Image dumping and program-op
by Elias Pipping 16 Sep '16

16 Sep '16

Dear list, from within a REPL, oftentimes, you can save the current status and create a file that’ll allow you to continue where you are now at a later point in time. This file could be a standalone executable or a core file that requires a lisp environment to run (e.g. via `lisp -core …`). These two cases can be subdivided into four overall use cases. (exe-repl) When the executable is run, you want it to print a banner and take you to a REPL. Maybe you’ve created this file after applying a patch to your lisp and you don’t want to do that over and over again. (exe-application) You’ve created an executable that should not print a banner and run an entirely new main() function. (core-repl) Analogous to the exe-repl case, but smaller at the cost of requiring a lisp to run (core-application) Analogous to the exe-application case (not sure if anyone does this). The function `uiop:dump-image` can be used to create images, where ‘image’ is an umbrella term for both standalone executables and cores. With :executable t, it creates the former; with :executable nil, the latter. The image is only told whether we want a REPL or application type image once it’s run and uiop:restore-image hands control over to what uiop:*image-entry-point*. That seems to work for some lisps, but for CMU CL it does not. CMU CL’s save-image provides options like :process-command-line t/nil :print-herald t/nil Whether the herald/banner is printed on startup is decided when save-image is called. It cannot be changed later. If you’re writing an application with a main method, and you don’t pass :print-herald nil, the banner will be printed before your application does anything else. If you want to read a list of options and arguments from the command line in your main method, this will not work if you passed :process-command-line nil to save-image, since then the command line arguments will be frozen (i.e., equal what they were when you called save-image). This suggests that for uiop:dump-image to cover relevant use cases on CMU CL, it will have to learn about the repl/application distinction, e.g. by growing another option. I’ve created a sample application that illustrates this and put it here(*): https://github.com/pipping/cl-echo echo.lisp does what its name suggests: It tells you what it is that you passed to it. If I want to create an exe-application type image from it, on sbcl e.g. that’ll work with uiop:dump-image and I can even use asdf’s program-op (please see build.lisp for how) For CMU CL, (uiop:dump-image :executable t) and asdf's program-op do something that I do not want: They leave the command line arguments frozen so that the resulting executable does not return what I want it to. I need something that’s different from what asdf’s program-op in build.lisp does (in particular, not pass :process-command-line nil), as is shown in build-cmucl.lisp. Were I to pass :print-herald nil with :executable t in uiop:dump-image on CMU CL, I’d make the exe-application case work as expected but not the exe-repl case. Hence my claim that we need a new option. If we add another option to dump-image, the question is how asdf's image-op and program-op should behave (here, the word image refers to core files only, as far as I can tell). Will we need new options here, too? If so, what should the defaults be? This write-up is the result of a conversation with Robert. Thanks again, Robert! I’d welcome feedback. Elias (*) The current version at the time of this writing is https://github.com/pipping/cl-echo/tree/faea91b6aa14ea81f2130edef3f6abccbd5… so if my notes don’t seem to make any sense, please give that revision a try.

2 2

Should marking a system preloaded drop its build information?
by Faré 15 Sep '16

15 Sep '16

In the patch he submitted, Daniel Kochmanski modified register-preloaded-system so that it eagerly replaces any previously loaded system with a placeholder object that has no build information. In the current ASDF, the preloaded system is only lazily registered if the system is requested but not previously loaded (or is cleared). In my current working branch, it is eagerly registered but only if no previous system was loaded or after clearing it. [Going from lazy to eager registration fixes a bug with already-loaded-systems and thus require-system.] The big design question is: should registering a system as preloaded drop its build information? Advantage of erasing the existing system information: the behavior of the image is uniform on all machines, and trying to resume development on a machine without the same source code at the same location won't cause a flurry of filesystem accesses at the wrong paths. [Note that immutable systems never try to access the filesystem, unlike regular preloaded systems, even with build information present.] Advantage of keeping the existing build information: you can resume the production image and resume incremental development on a machine that has the same source code at the same location. And if you to drop it, you can explicitly call clear-system after you call register-preloaded-system. I am strongly inclined to keeping the build information around: it's always easy to erase the information later, but hard to reconstitute dropped information. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org A Libertarian Constitutional Amendment: Congress shall make no law.

4 6

Windows status
by Robert Goldman 14 Sep '16

14 Sep '16

Merged Faré's patch and bumped to 3.1.17.20. Mostly working on Windows now, but still have one problem: run-program is failing on ecl_bytecodes. That's for tomorrow, though.

1 0

Immutable Systems
by Faré 14 Sep '16

14 Sep '16

Dear Dave and ASDF users, some of you like Dave may be using immutable-systems. Daniel Kochmanski noticed I made the mistake of advertising *immutable-systems*. I'm considering maybe deprecating this variable in a future release. If you're just accessing it to declare systems immutable, please instead use, e.g. (map () 'asdf:register-immutable-system (asdf:already-loaded-systems)) If you're accessing it to get a list, I'll export a new functional interface. —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Associate yourself with people of good quality, for it is better to be alone than in bad company. ― Booker T. Washington (1856–1915)

2 2

Windows problems: moratorium continues
by Robert Goldman 14 Sep '16

14 Sep '16

Unfortunately, the last time I was able to run the Windows tests before 2 days ago was in mid-August. Since then, there have been a flurry of commits, many of which could have impacted the state of test-force.script. Indeed, test-force.script *itself* has changed, so that, although we call it a "regression test," we have no reason to believe that it *ever* worked on Windows in its present form. Unfortunately, it may take me a while to get enough bisecting done to figure out when we jumped the rails. I'll keep everyone posted, but in the meantime, we should consider the repo locked against more changes. Windows is badly broken, and we shouldn't be trying to move forward again until we have restored function on this platform. thanks, r

2 1