Hi Faré,
On Tue, Apr 24, 2012 at 8:02 AM, Faré fahree@gmail.com wrote:
Dear Erik,
I thought I'd report back briefly on some results.
Thanks for trying. Can you share your code?
I'd be glad to -- perhaps after a bit more tinkering so that at least the changes are "switchable" via a global flag.
First, I spent some time attempting to implement "perform with session state". This is the idea that when working with ASDF, it would be useful to be able to invoke or create specific behavior for my use case and also to be able to capture work session state without relying on globals. This was partially successful, but I broke something and ran out of patience. ASDF is some wonderful code, but takes a bit to grok it all.
In the past, I've slightly modified ASDF to make it easier for POIU to do its job. If there's some interface I can expose so your hack can be an extension of ASDF rather than a wholesale replacement of its functionality, I'll be glad.
So I took the crude approach of just installing a global and using it to accumulate loaded files from within a perform. I created an .asd file which defined the dependencies I wanted the image to have. The loaded it with load-system. Then at the end of load-system I inserted code to save a structure containing a list of the *defined-systems* as well as a list of the loaded files (in order of loading).
NB: POIU also has a mode to record and replay a list of accumulated files.
Am I correct that these are the things you're trying to do? (1) load all the dependencies
yes
(2) record what are the things loaded so far
yes
(3) load your application (4) when you refresh your application, skip the recorded dependencies.
Basically, yes. I would state, though, that I want the ASDF to work like the primary contract of REQUIRE. That is, this clause from CLHS:
require tests for the presence of the module-name in the list held by *modules*. If it is present, require immediately returns.
I think that ASDF's default mode of grokking the filesystem for changes is what I want in a development environment.
However, it is not what I want for systems that I'm just depending on and (not necessarily) modifying. It should compile those if needed, and load those just one time.
And it it not what I want in a system that is deployed, because the source code for loaded modules will never change, and if I need to work some patches in or load some runtime functions, that will be under application control
I realize that I'm expecting ASDF to magically behave differently when developing code, to discover changed files and recompile and reload them. Thus the need for a global flag to tell ASDF its operating mode.
One culprit, it seems, is REQUIRE. Otherwise, ASDF wouldn't do anything in a normal system without ASDF specific calls. REQUIRE is not used all that much in Lisp code, not nearly as much as an import in other languages which depend on it. And I would partly blame the big confusion between PACKAGE, DEFSYSTEM, and REQUIRE -- it is kind of a mess. Still, REQUIRE does pop up in library code, and it is a very easy to use and easy to understand way of both expressing a dependency and possibly loading your toplevel system when developing, deploying, or running code ad-hoc. And it does sorta map from modules to systems, albeit with no facility for dependencies.
Another goal, though, is to reduce the noise in the generated image. If the image is created after ASDF has been doing its work, it is not necessarily suitable for deployment. Maybe the target application needs ASDF, maybe it doesn't. Plus I just get nervous with an image that has just been worked on (Lisp has enough moving parts as it is.) Having the load plan, or rather the plan implementation code, recorded, saved, and loaded into a fresh image would allow the omission of ASDF, its dependencies, or any other system-building code and data structures. The target image might not even have a compiler or code loading functionality. On the other hand, it might be desirable to include ASDF and initialize the ASDF system database with some or all of the loaded systems marked as not-reloadable.
If so, then I propose you use the functionality I added to 2.20.21 for the measly cost of 12 lines of code, registered-systems and :force-not:
Aww, you shouldn't have :)
(asdf:load-systems '(dependency-a dependency-b)) ;(1) (defparameter *recorded-dependencies* (asdf:registered-systems)) ;(2) (asdf:load-system :my-application) ;(3) (asdf:load-system :my-application :force-not *recorded-dependencies*) ;(4)
If you want more fine-grained control, you could provide an override defmethod operation-done-p :around
If you like my :force-not feature, I'd appreciate you write a test case in the asdf test suite.
It was about 2.5 seconds to traverse normally, and about 0.5 with this optimization for a require after the first one (of course for a deployed system not development.)
Can you confirm it works well with my proposed solution?
In any case load time is not my only interest. It is also a desire to be able to use ASDF more effectively for more use cases.
Same here.
I really appreciate the responses and the new code to play with. That really is exceptional. I'll send some feedback later, hopefully today, but I have a message queue processing system to build and a web site to get up :(
Erik.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org It is a miracle that curiosity survives formal education. — Albert Einstein