Well, systems should declare the resources they want packaged into their
applications in their system files. That's a requirement both ECL and ABCL now put forward, but I think other implementations which are able to deliver more or less stand alone programs would want the same.
Updating the ASDF documentation on how to use the new ops as well as recommendations for ASDF definitions would be good steps in this direction. When time and resources are available, of course…
Completely agreed. I'm going to write a blog item about my own use case, but true, there's a lot more to it than just that.
But because "universal distribution" (i.e. for the majority of usersof
a given system) has more-or-less subsumed by Quicklisp, there is noreal
constraint on the developers of system definitions to ensure that all components are so enumerated: loading the system from Quicklispworks
so nothing is broken, right?Surely we can add tests for that in cl-test-grid as well?
Maybe we could implement an ASDF/LINT that checks the "well-formedness" of a given .asd file, reporting potential problems? Conceptually such checking needs to be maintained "closer" to ASDF than CL-TEST-GRID. Implementation is the sincerest form of flattery, of course
That's not really what I meant, if I understand you correctly. What I meant is: if we can use cl-test-grid to test if libraries load, then we surely can use cl-test-grid to test if a "compiled-and-loaded-from-monolithic-fals" version works. If it doesn't, I'd take that as an indication the ASDF system is broken (that is, if the loader would work for the non-monolithic case).
And even if one discarded this possiblity, often directory hierarchies share multiple ASDF definitions, so one would package "source" components that aren't really source components.I'm sorry, I don't understand this bit. Could you try to explain your concern with different words?
Think of the ABCL-TEST-LISP definition in abcl.asd: it only references files in the test/lisp/abcl sub-directory.
That's fine. In this case, the current ASDF system builder includes only the lisp files it knows about.
So, if one one were to treat the pathname of the .asd file it is defined in, one would include the abcl Java source, possibly the build artifacts, etc. which is probably not what one wants.
Right, so I'm not arguing we should include "everything below the system file". I'm saying that system definitions can use #. macros to include files dynamically into the system definition and once that definition is compiled and packed, there's no need to evaluate the #. macro anymore so, loading off a monolithic fasl is still a viable option.
Or consider CFFI with the cffi-ffi.asd definition, which needs one specific subdirectory.
True. Isn't it the case that those subdirectories are currently included using reader conditionals? I'm assuming we're not cross-compiling systems for ABCL on SBCL here. I have a feeling I'm not understanding you on this point.
So, assuming that "deployment outside of Quicklisp from an ASDFdefinition" would be a greater good for the community, we seem to be stuck at a bootstrap problem. I can't think of a way to force ASDF authors to enumerate their static components, but until suchenumeration
is widespread, I can't demonstrate its utility by developing an ASDF/BUNDLE operation.Well, there is no need to develop the ASDF/BUNDLE operation: Fare did. Buth other than that, the fact that I can now succesfully package bordeaux-threads which we couldn't with ASDF-JAR, I think this is really a step ahead?
Conceptually, it sounds like a step ahead but how do I use it. Could you write up a quick document on how to use it? I'm still not sure how things like ASDF:SYSTEM-RELATIVE-PATHNAME would work in the monolithic fasl scenario, so I would like to get my hands dirty a bit here.
I'm not certain with respect to this point. Lets ask Fare about that :-)
[I actually started writing this mail bottom to top, so what's below may be a reiteration...]
Well, the great thing about the solution that Fare has now rolled out is that I was able to package and load Bordeax-threads even though it is problematic because it uses the #. reader macro and does not include the version.lisp-sexp in the ASDF definition...
The infrastructure I've added to ABCL has been hooked up into ASDF with his new release. Much more doesn't seem to be required to get it all correctly packaged. Package providers can simply use #. reader macros to inject the list of :static-files they need to be distributed with their code and asdf will pick it up. We'll need to work and test ASDF + ABCL to see if everything works as intened though.
And then deprecate ASDF-JAR?
Only if that's the right thing to do in the sense that ASDF-JAR's functionality has been subsumed into ASDF3's build facility. However, I don't think we need to: it's a contrib in its own right until it's broken and nobody can be motivated to unbreak it.
In some ways, I thought that being able to present ABCL components as jar files (abcl.jar, abcl-contrib.jar, hunchentoot-all-1.2.14.jar) would be a more natural fit to typical Java server deployment scenarios.
Well, the only difference betweer a FASL and a JAR is the MANIFEST file... I'm sure we can come up with a way to make the .abcls look like jars or even turn them into real ones. However, if there's a file called "my-application--systems.jar", why
Other than this more natural fit to JVM deployments (whereby the pointy-headed boss would be none-the-wiser as "everything just looks like jar dependencies") the only advantage of using a jar format that still might exist would be to leverage the ["One Jar" approach][one-jar] to actually use a custom classloader to make JVM artifacts (libraries, resources, etc.) packaged in the deployment as well. With this sort of approach we could include the jna.jar needed for CFFI in the deployment artifact, as opposed to needing to override going for the resolving the Maven dependency at deployment runtime.
Bye,
Erik.