> Well, systems should declare the resources they want packaged into their
> applications in their system files. That's a requirement both ECL and
> ABCL now put forward, but I think other implementations which are able
> to deliver more or less stand alone programs would want the same.

Updating the ASDF documentation on how to use the new ops as well as
recommendations for ASDF definitions would be good steps in this
direction. When time and resources are available, of course…

Completely agreed. I'm going to write a blog item about my own use case, but true, there's a lot more to it than just that.

> But because "universal distribution" (i.e. for the majority of users of
> a given system) has more-or-less subsumed by Quicklisp, there is no real
> constraint on the developers of system definitions to ensure that all
> components are so enumerated: loading the system from Quicklisp works
> so nothing is broken, right?
>
> Surely we can add tests for that in cl-test-grid as well?

Maybe we could implement an ASDF/LINT that checks the "well-formedness"
of a given .asd file, reporting potential problems? Conceptually such
checking needs to be maintained "closer" to ASDF than CL-TEST-GRID.
Implementation is the sincerest form of flattery, of course

That's not really what I meant, if I understand you correctly. What I meant is: if we can use cl-test-grid to test if libraries load, then we surely can use cl-test-grid to test if a "compiled-and-loaded-from-monolithic-fals" version works. If it doesn't, I'd take that as an indication the ASDF system is broken (that is, if the loader would work for the non-monolithic case).

> And even if one discarded this possiblity, often
> directory hierarchies share multiple ASDF definitions, so one would
> package "source" components that aren't really source components.
>
>
> I'm sorry, I don't understand this bit. Could you try to explain your
> concern with different words?

Think of the ABCL-TEST-LISP definition in abcl.asd: it only references
files in the test/lisp/abcl sub-directory.

That's fine. In this case, the current ASDF system builder includes only the lisp files it knows about.

So, if one one were to treat
the pathname of the .asd file it is defined in, one would include the
abcl Java source, possibly the build artifacts, etc. which is probably
not what one wants.

Right, so I'm not arguing we should include "everything below the system file". I'm saying that system definitions can use #. macros to include files dynamically into the system definition and once that definition is compiled and packed, there's no need to evaluate the #. macro anymore so, loading off a monolithic fasl is still a viable option.

Or consider CFFI with the cffi-ffi.asd definition,
which needs one specific subdirectory.

True. Isn't it the case that those subdirectories are currently included using reader conditionals? I'm assuming we're not cross-compiling systems for ABCL on SBCL here. I have a feeling I'm not understanding you on this point.

> So, assuming that "deployment outside of Quicklisp from an ASDF
> definition" would be a greater good for the community, we seem to be
> stuck at a bootstrap problem. I can't think of a way to force ASDF
> authors to enumerate their static components, but until such enumeration
> is widespread, I can't demonstrate its utility by developing an
> ASDF/BUNDLE operation.
>
> Well, there is no need to develop the ASDF/BUNDLE operation: Fare did.
> Buth other than that, the fact that I can now succesfully package
> bordeaux-threads which we couldn't with ASDF-JAR, I think this is really
> a step ahead?

Conceptually, it sounds like a step ahead but how do I use it. Could
you write up a quick document on how to use it? I'm still not sure how
things like ASDF:SYSTEM-RELATIVE-PATHNAME would work in the monolithic
fasl scenario, so I would like to get my hands dirty a bit here.

I'm not certain with respect to this point. Lets ask Fare about that :-)

> [I actually started writing this mail bottom to top, so what's below may
> be a reiteration...]
>
> Well, the great thing about the solution that Fare has now rolled out is
> that I was able to package and load Bordeax-threads even though it is
> problematic because it uses the #. reader macro and does not include the
> version.lisp-sexp in the ASDF definition...
>
> The infrastructure I've added to ABCL has been hooked up into ASDF with
> his new release. Much more doesn't seem to be required to get it all
> correctly packaged. Package providers can simply use #. reader macros to
> inject the list of :static-files they need to be distributed with their
> code and asdf will pick it up. We'll need to work and test ASDF + ABCL
> to see if everything works as intened though.

And then deprecate ASDF-JAR?

Only if that's the right thing to do in the sense that ASDF-JAR's functionality has been subsumed into ASDF3's build facility. However, I don't think we need to: it's a contrib in its own right until it's broken and nobody can be motivated to unbreak it.

In some ways, I thought that being able to
present ABCL components as jar files (abcl.jar, abcl-contrib.jar,
hunchentoot-all-1.2.14.jar) would be a more natural fit to typical Java
server deployment scenarios.

Well, the only difference betweer a FASL and a JAR is the MANIFEST file... I'm sure we can come up with a way to make the .abcls look like jars or even turn them into real ones. However, if there's a file called "my-application--systems.jar", why

Other than this more natural fit to JVM
deployments (whereby the pointy-headed boss would be none-the-wiser as
"everything just looks like jar dependencies") the only advantage of
using a jar format that still might exist would be to leverage the ["One
Jar" approach][one-jar] to actually use a custom classloader to make JVM
artifacts (libraries, resources, etc.) packaged in the deployment as
well. With this sort of approach we could include the jna.jar needed
for CFFI in the deployment artifact, as opposed to needing to override
going for the resolving the Maven dependency at deployment runtime.

[one-jar]: http://one-jar.sourceforge.net/

Bye,

Erik.