On Wed, Feb 20, 2013 at 12:46 AM, Dave Cooper david.cooper@genworks.com wrote:

Holy Cow I started looking through ADSF3 (asdf 2.29) and it is starting to do everything under the sun!

Well, not *everything*, just everything needed to portably bootstrap the building of Common Lisp code.

The new functionality like the ASDF/DRIVER compatibility layer, dumping images, and concatenated fasls look really useful and have potential to simplify things on our end a lot.

That's the whole point of it.

So ASDF/DRIVER has superseded asdf-utils, right?

Yes it has. There is a package nickname that says so, and asdf-utils is now an otherwise empty system that depends on asdf/driver.

The functions in "10.3 Miscellaneous Functions" of the documentation are the main functions now in ASDF/DRIVER? Or I suppose there are a lot more now -- and the main documentation for the moment are in the source code (understandable).

The manual has been minimally updated. Writing then debugging ASDF3 took so much energy that I didn't have much left for writing the manual, especially since things were moving fast while I made many passes to refactor the API until it was in a satisfying shape.

For dumping of images, I see for Allegro CL this maps to excl:dumplisp with only a couple of keyword arguments, which is fine for basic dumping of images when there's not a lot of need of fine control. As you may or may not know, Allegro CL has a couple other ways of making executable images: excl:generate-application and excl:build-lisp-image. For our distributions we sometimes have to call these functions instead of excl:dumplisp. When the ASDF3 dust has settled a bit and I understand the workings of ASDF3 a bit better, I hope I (or somebody -- maybe somebody from Franz?) will be able to extend the image-dumping support in ASDF to handle these Allegro CL functions as well. But, these functions take many (maybe dozens) of keyword arguments, so maybe it starts to go beyond what a reasonable portability layer like what ASDF is trying to achieve should be trying to provide... (actually excl:dumplisp takes many of the same keyword arguments as well... so I suppose the answer is that if one needs that find level of control, one had better go ahead and directly call the native version).

Yes, for the moment, you'll have to go native if you need the support. Nonetheless I still recommend you use ASDF3 and respect its hooks as you dump the image: (setf *image-dumped-p* (if executable :executable t)) (standard-eval-thunk *image-postlude*) (call-image-dump-hook)

Then use restore-image as your restart function.

If you contribute support for all these options and more to asdf/image, I would be grateful.

Anyway for the time being, if I am using ASDF3 with Allegro CL, and want to use excl:generate-application or excl:build-lisp-image, in order to clear the configuration, is the

(asdf:clear-configuration)

still going to work as in ASDF2?

Yes, it will; but still I recommend you use ASDF3 and its hooks instead (see above).

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org Throughout history, poverty is the normal condition of man. Advances which permit this norm to be exceeded — here and there, now and then — are the work of an extremely small minority, frequently despised, often condemned, and almost always opposed by all right-thinking people. Whenever this tiny minority is kept from creating, or (as sometimes happens) is driven out of a society, the people then slip back into abject poverty. This is known as "bad luck". — Robert Heinlein, "Time Enough For Love"

Hi Anton,

Thanks for asking about this -- I have not started experimenting with this yet in earnest.

My main concern is the internals of asdf and Quicklisp itself. Setting ql:*quicklisp-home* at new image startup time at least does appear to allow Quicklisp itself to function normally with the new local quicklisp/ directory. The main problem I was having trouble previously was re-initializing asdf output-translations. They were getting "stuck" to something hardcoded to an absolute pathname under "/home/dcooper8/.." (or Windows equivalent) which of course is a no-go for a target delivery machine.

I still have not managed to re-initialize asdf output-translations to the local home-directory cache of the target user, which is the desired outcome. The best I have managed so far is to disable output-translations entirely, so that the fasl's end up near the sources. This is not an ideal scenario because the target user loses for example the platform-separation benefits of output-translations.

I have not yet tried Fare's latest advice or looked at re-initializing output-translations recently. I have to do it very soon however so I will let you know where things end up.

For the particular case of webserver applications which rely on static files: I am not yet familiar with Hunchentoot, but Allegroserve has the concept of publishing static files with functions "publish-file" and "publish-directory."

At startup time, our standard restart-init-function (for internal development and for deployed applications) establishes the local installation directory, which also determines the location of ql:*quicklisp-home* -- and we simply re-publish (with Allegroserve's publish-file or publish-directory operators) the static files. We do not attempt to have Allegroserve magically reproduce the published locations of static files in a newly started image - we always explicitly call publish-file or publish-directory as part of the restart-init-function. I would think that any library which depends on static files in a similar way should provide a "publish-resources" function which can be called as part of a restart-init-function for any image.

Anyway I will keep you apprised of progress in the next days while I am trying to prepare images for distribution.

Regards

Dave

On Fri, Mar 1, 2013 at 1:32 PM, Anton Vodonosov avodonosov@yandex.ruwrote:

Dave, do you have success with moving saved lisp image to new machine/directory, when the image has asdf and quicklisp loaded?

I am trying this too, and (asdf:clear-configuration) is not enough for systems loaded from quicklisp.

For example many libraries keep static files, like .html, .css in their repositories. And at run-time access these resources with asdf:system-relative-pathname.

The simplest example is hunchentoot. If you do (ql:quickload :hunchentoot) (hunchentoot:start (make-instance 'hunchentoot:easy-acceptor :port 4242))

and open http://127.0.0.1:4242/ you see hunchentoot default page and documentation. These pages are served from quicklisp\dists\quicklisp\software\hunchentoot-1.2.7\www\

When I save a lisp image and copy the image together with quicklisp/ directory to another location, the libraries can not locate their static files. In my example, hunchentoot says "The requested URL / was not found on this server."

I have tried to (asdf:clear-configuration) before saving the image and after image is restored to (setf ql:*quicklisp-home* (truename (merge-pathnames "quicklisp/" *default-pathname-defaults*))) (ql:setup) but it didn't help.

So saving lisp image only work for me in tiny applications where I know there is no need for resources from file system.

For general use, where application depends on many libraries I am afraid there is no way to make relocatable lisp images. Even if I find a way to reinitialize asdf and quicklisp configurations, there are libraries which have their own custom variables holding file system paths. For such applications I only see one reliable way - prebuild all the .fals files with disabled asdf-output-translations so that .fals files are placed near the sources. Then copy full application and library sources to new location and reload them by (load "quicklisp/setup.lisp") (ql:quickload :my-application).

Best regards,

• Anton