On Thu, Apr 4, 2013 at 10:50 AM, Mark Evenson <evenson@panix.com> wrote:
On 4/4/13 12:44 AM, Anton Vodonosov wrote:
[…]
> General note about deployment CL systems.
>
> Often the deployment package should contain not only compiled code,
> but also resources: web libraries and frameworks often contain .css,
> .html, .javascript; bordeaux-threads uses version.lisp-sexp files,
> and so on.
>
> In addition to the #. reader macro as used in bordeaux-threads,
> applications/libraries access their resources during run-time using
> asdf:system-relative-pathname or similar functions. In other words,
> such libraries assume presence during run-time of their full source
> control checkout .

Well, systems should declare the resources they want packaged into their applications in their system files. That's a requirement both ECL and ABCL now put forward, but I think other implementations which are able to deliver more or less stand alone programs would want the same.
 
>
> A general purpose deployment solutions should account this aspect.
>
> One way to deploy applications with full content of the dependencies
> code and resources:
>  1. install quicklisp in a custom directory,
>  2.  (ql:quickload :my-application)
>  3. copy the directory of my-application, the custom quicklisp
>     directory and probably prebuild .fals files to server.
>
> A little bit more docs and example of this approach is here:
> https://github.com/avodonosov/heroku-buildpack-cl2

 [ ... ]

But because "universal distribution" (i.e. for the majority of users of
a given system) has more-or-less subsumed by Quicklisp, there is no real
constraint on the developers of system definitions to ensure that all
components are so enumerated:  loading the system from Quicklisp works
so nothing is broken, right?

Surely we can add tests for that in cl-test-grid as well?
 
After Xach, Anton probably has the most practical experience with
packaging ASDF definitions for deployment due to his pioneering work on
CL-TEST-GRID.  I find Anton comments to be a succinct summary of the
issues involved in packaging a given ASDF system in Quicklisp.  But when
facing deployment for an ASDF system that is not in Quicklisp, but has
dependencies on Quicklisp systems, Anton's approach won't work so well.
 One would have to find the .asd definition on the filesystem, then
transverse its sub-directories to find the source files.

The above is no longer true: with MONOLITHIC-FASL-OP you can build a fasl which holds a system and all its depedencies, including QL provided dependencies. This means you only need a working quicklisp setup on the development machine.
 
 This has the
problem that there is no guarantee that the components of an asd
definition exist in subdirectories, as theoretically the PATHNAME could
point to an ancestor.

True, but with the above, they don't need to anymore: they'll be included in the monolithic fasl. Even better: the monolithic fasl really doesn't require ASDF at all: you can simply load the FASL with a normal LOAD call and it will load all the embedded FASLs in the right (ASDF derived) order.
 
 And even if one discarded this possiblity, often
directory hierarchies share multiple ASDF definitions, so one would
package "source" components that aren't really source components.

I'm sorry, I don't understand this bit. Could you try to explain your concern with different words? 

So, assuming that "deployment outside of Quicklisp from an ASDF
definition" would be a greater good for the community, we seem to be
stuck at a bootstrap problem.  I can't think of a way to force ASDF
authors to enumerate their static components, but until such enumeration
is widespread, I can't demonstrate its utility by developing an
ASDF/BUNDLE operation.

Well, there is no need to develop the ASDF/BUNDLE operation: Fare did. Buth other than that, the fact that I can now succesfully package bordeaux-threads which we couldn't with ASDF-JAR, I think this is really a step ahead? 

Any ideas?

[I actually started writing this mail bottom to top, so what's below may be a reiteration...]

Well, the great thing about the solution that Fare has now rolled out is that I was able to package and load Bordeax-threads even though it is problematic because it uses the #. reader macro and does not include the version.lisp-sexp in the ASDF definition...

The infrastructure I've added to ABCL has been hooked up into ASDF with his new release. Much more doesn't seem to be required to get it all correctly packaged. Package providers can simply use #. reader macros to inject the list of :static-files they need to be distributed with their code and asdf will pick it up.  We'll need to work and test ASDF + ABCL to see if everything works as intened though.


Bye,


Erik.