Should have responded to this sooner.
The proposed change is to have a single form in the system definition that deals with all maven dependencies - that's the :mvn-module. It has the most important expressivity available in a POM, namely to list all dependencies, exclusions, and overrides (called managed depencies) which let you specify a specific version for a transitively gathered dependency.
Having these easily accessible makes the code that asks Maven to resolve the dependencies all at once easy to invoke.
That part is implemented.
For the case of managing multiple asdf systems that have maven dependency one way to go forward would be to have resolves dependencies in a system added (and overriding) managed dependencies in subsequent :mvn module resolution. This has the benefit that you can coordinate version choices across multiple systems by first loading a system definition that sets managed dependencies that stick across all subsequent mvn resolutions.
There's no guarantee that doing so will be adequate - if the collected set of maven dependencies is fundamentally conflicted you are still screwed. But we can't do anything to prevent that. however in my case that wasn't so and so this strategy solved my issue.
I think other parts of ABCL have maven dependencies that overlap the ones in LSW. If these load before user code and the versions chosen aren't optimal for downstream use dependencies we might need a mechanism for overriding them (e.g. The resolution during abcl chooses v2.4 but would work with v2.7, but user code might not work with 2.4. To reduce the incidence of such cases we should avoid using version specifications that are restrictive - say only to major version to next major version - and, when possible, defer resolution until needed at runtime. I do that in a couple of cases already. I think in OSGI, and in some code that uses a Java parser library. In those cases the dependencies are not recorded in the system definition. Instead resolve is called only when code that needs is executed.
Alan
On Tue, Apr 18, 2017 at 1:57 AM Mark Evenson evenson@panix.com wrote:
On 4/18/17 04:14, Alan Ruttenberg wrote: […]
There is OSGI which is able to do this sort of thing, and there is
already
prototype code in JSS to use it. However OSGI, or multiple classloaders, needs a fair amount of thought to use correctly, because you can easily arrive at situations where compatible classes can't be used because they were loaded into different classloaders.
The first step is to enable managing dependencies when it *is* possible
to
run without conflicts, as with the :mvn-module I showed above. In this
case
it turns out that the dependencies only *seemed* to conflict. With the appropriate use of Maven there is no conflict in this case.
The thing that is different in our environment as compared to the
standard
maven situation is that we don't have a central pom for an application
that
gathers all dependencies and resolves them at once. Instead we have separate ASDF systems which are loaded independently and yet may have intertwined dependencies.
I agree that incrementally fixing the current situation along the lines you have prototyped should occur before any dramatic overhaul with segregated classloaders.
Currently, I am working towards a [abcl-1.5.0][] next week with the following priorities:
- stabilizing support for Java6 (dropping support for Java5, which in
practice given the absence of an open Java6 VM doesn't exist)
- Resurrect the ABCL-BUILD system to build ABCL from Lisp by
downloading and installing Ant and Maven as needed.
- Provide a restart for failure to find Maven, that will download and
install it in the XDG filesystem.
Maybe we can try to squeeze your :MVN-MODULE proposal in as well. I've seen [your code that demonstrates the problem][1], but as I understand it there is no exact code as you need more understanding of the underlying ASDF mechanisms, right? If you could take the time in the next day or so to write up the minimal proposal for your current needs, I will see what I can do to take a stab at its implementation.
[1]:
https://gitlab.common-lisp.net/abcl/abcl/blob/master/t/resolve-multiple-mave...
-- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."