Hello folks,
I'm in the process of integrating Dynaspring [1] in a yet-to-be-released open source utilities project developed together with some (ex-)colleagues. The utilities project is built with Maven, so in order to use Dynaspring (and transitively ABCL) I had to write a simple pom.xml (Maven project descriptor) for ABCL to make it known to Maven (i.e. usable as a dependency in Maven projects). For the other developers' convenience, it would be nice if the poms were available from a publicly accessible Maven repository (Sonatype provides a free one, [2]). I'm going to create one there for Dynaspring, but if you're ok with it, I can create a separate one for ABCL, and from now on, publish ABCL releases there as well. What do you think?
Bye, Alessio
[1] http://code.google.com/p/dynaspring/ [2] https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+U...
Hi Alessio,
I'm in the process of integrating Dynaspring [1] in a yet-to-be-released open source utilities project developed together with some (ex-)colleagues. The utilities project is built with Maven, so in order to use Dynaspring (and transitively ABCL) I had to write a simple pom.xml (Maven project descriptor) for ABCL to make it known to Maven (i.e. usable as a dependency in Maven projects). For the other developers' convenience, it would be nice if the poms were available from a publicly accessible Maven repository (Sonatype provides a free one, [2]). I'm going to create one there for Dynaspring, but if you're ok with it, I can create a separate one for ABCL, and from now on, publish ABCL releases there as well. What do you think?
I'm not very familiar wit Maven and its repositories, so I have some additional questions:
* Is a maven-user restricted to a single maven repository per project? In other words, do all dependencies have to be in a single repository, or can dependencies be downloaded from different repositories? * You choose for this specific Maven repository. Is there a specific reason? Are they the best (and by which definition of best)? Are they the most well known? * Can the steps for publishing to the repository be built into the release process? Will it make our release process much heavier, or is uploading a purely technical step a computer can do with a human just supervising correct completion of the action? (In the latter case, we could just script the uploads to our website and the maven repository and wait a little more than before...)
In general, I think it's a good idea to hook into distribution mechanisms common to Java developers. Thanks for taking this initiative!
Regards,
Erik.
On Fri, Feb 4, 2011 at 12:03 PM, Erik Huelsmann ehuels@gmail.com wrote:
Hi Alessio,
I'm in the process of integrating Dynaspring [1] in a yet-to-be-released open source utilities project developed together with some (ex-)colleagues. The utilities project is built with Maven, so in order to use Dynaspring (and transitively ABCL) I had to write a simple pom.xml (Maven project descriptor) for ABCL to make it known to Maven (i.e. usable as a dependency in Maven projects). For the other developers' convenience, it would be nice if the poms were available from a publicly accessible Maven repository (Sonatype provides a free one, [2]). I'm going to create one there for Dynaspring, but if you're ok with it, I can create a separate one for ABCL, and from now on, publish ABCL releases there as well. What do you think?
I'm not very familiar wit Maven and its repositories, so I have some additional questions:
I'm not very familiar either, but I'll do my best ;)
* Is a maven-user restricted to a single maven repository per project? In other words, do all dependencies have to be in a single repository, or can dependencies be downloaded from different repositories?
They can be in different repositories; the repositories to use can be written in the project POM - but that is generally discouraged [1] - or in a local settings file.
* You choose for this specific Maven repository. Is there a specific reason? Are they the best (and by which definition of best)? Are they the most well known?
The "best" as in most widely used is Maven Central [2], the default one; however, getting something published there is not trivial. Sonatype is a company that's deeply connected with maven (they develop the m2eclipse plugin and the Nexus repository/proxy server), and the only one I found offering a free, Nexus-based repository for open source projects. I believe that in the end a Maven repo can just be a HTTP server publishing files following a certain directory structure. However, by using Nexus you get the benefit of an already made setup, a maven-specific web UI, and many other configuration options that are useless for ABCL and that I don't know much about (like configuring the nexus as a proxy of some third-party repo, useful in an enterprise setting).
* Can the steps for publishing to the repository be built into the release process? Will it make our release process much heavier, or is uploading a purely technical step a computer can do with a human just supervising correct completion of the action? (In the latter case, we could just script the uploads to our website and the maven repository and wait a little more than before...)
It can be automated. Releasing *should* amount simply to changing the ABCL version in the pom.xml and uploading the jar + pom to the repository (for doing that Maven itself can be used).
In general, I think it's a good idea to hook into distribution mechanisms common to Java developers. Thanks for taking this initiative!
Well, I took it out of necessity - I'm not too fond of Maven, but surely if it's not a great effort it's nice to have one more option for distribution. Anyway, first I'll experiment with Dynaspring, then when I understand better how the whole release process works, I'll report here with more information.
Bye, Alessio
[1] http://www.sonatype.com/people/2009/02/why-putting-repositories-in-your-poms... [2] http://repo1.maven.org/maven2/
On Fri, Feb 4, 2011 at 12:48 PM, Alessio Stalla alessiostalla@gmail.com wrote:
when I understand better how the whole [Maven] release process works, I'll report here with more information.
Ok, so I experimented a bit with Maven and the free Sonatype repositories. To do a release, you need to produce the jar, sources-jar, and javadoc-jar of your project, and sign them with a gpg key. You then have to publish them to the repository; doing it (i.e. signing + publishing) with Maven is probably the easiest thing, although there's a way to do it with Ant too (but you have to have additional Maven Ant tasks installed on your system). As a bonus, you can ask your repository to be automatically synced to Maven Central, making it super easy for Maven users to use your project. The release must be manually confirmed to be migrated from the "staging" repository to the "public releases" repository; this is the only non-automatic step. The full process is detailed here: https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide
As far as I can tell, no special format is *required* for the sources and javadoc jars, although the ones generated by Maven of course have a precise format: basically the packed src directory for sources, and the packed doc root directory for javadocs (you can see an example here: https://oss.sonatype.org/content/repositories/releases/com/googlecode/ehcache-spring-annotations/ehcache-spring-annotations/1.1.2/. ABCL doesn't have many javadocs, and anyway those are certainly not the main part of the documentation; but the javadoc jar is required and, if it follows the convention, it can be automatically picked up by Eclipse when you add ABCL as a Maven dependency.
Morale: this is not terribly complicated, and it would make ABCL available through another distribution channel. OTOH, I can't hide the fact that - besides my own personal convenience for my shared project - probably no-one is using ABCL in Maven projects right now, and having ABCL released through Maven would complicate its release process, not much for technical reasons but for organizational ones (e.g. who keeps ABCL's GPG key? How is it managed?). So, if you accept Maven release to be an official step in ABCL's release process, that would be great for me and I could of course help in setting it up, solving issues etc. Else, if you're ok with it, I can publish ABCL on Sonatype's repo on my own, without making it part of our official release policy; that means less hassle for ABCL, but more risk of stale Maven releases and other problems caused by having the release depend on a single person (of course I'll take the release seriously, but still, depending on a single individual is always problematic). The third option is NOT to release ABCL with Maven at all: if you think that it would hurt ABCL, either by complicating the release process too much, or by having too high a risk of out-of-date releases out in the wild, I will accept that.
Let me know of your opinions.
Bye, Alessio
On 2/9/11 9:36 PM, Alessio Stalla wrote:
On Fri, Feb 4, 2011 at 12:48 PM, Alessio Stallaalessiostalla@gmail.com wrote:
when I understand better how the whole [Maven] release process works, I'll report here with more information.
Ok, so I experimented a bit with Maven and the free Sonatype repositories. To do a release, you need to produce the jar, sources-jar, and javadoc-jar of your project, and sign them with a gpg key. You then have to publish them to the repository; doing it (i.e. signing + publishing) with Maven is probably the easiest thing, although there's a way to do it with Ant too (but you have to have additional Maven Ant tasks installed on your system).
[…]
I know that Maven can use Ant tasks, so the easiest way forward might be to have Ant continue to build while Maven packages it. Maven has the reputation of strong—i.e. inflexible–ideas about directory layouts that may prove to make this task difficult.
If we use Ant to package, I would like to continue to have the main build.xml file not have external Ant dependencies to enable the ABCL source distribution to be easily built without any additional downloads (other than Ant). A possible way to implement this would be to have an additional file, say 'maven-build.xml', that includes the main 'build.xml' while requiring the Maven Ant tasks.
As far as I can tell, no special format is *required* for the sources and javadoc jars, although the ones generated by Maven of course have a precise format: basically the packed src directory for sources, and the packed doc root directory for javadocs (you can see an example here:https://oss.sonatype.org/content/repositories/releases/com/googlecode/ehcache-spring-annotations/ehcache-spring-annotations/1.1.2/. ABCL doesn't have many javadocs, and anyway those are certainly not the main part of the documentation; but the javadoc jar is required and, if it follows the convention, it can be automatically picked up by Eclipse when you add ABCL as a Maven dependency.
At least this would put some pressure on us to add some more javadoc strings. A reasonable summary of the org.armedbear.lisp package pointing to the Wiki and mailing list might be a decent start.
Morale: this is not terribly complicated, and it would make ABCL available through another distribution channel. OTOH, I can't hide the fact that - besides my own personal convenience for my shared project
- probably no-one is using ABCL in Maven projects right now, and
having ABCL released through Maven would complicate its release process, not much for technical reasons but for organizational ones (e.g. who keeps ABCL's GPG key? How is it managed?).
Currently Erik maintains the GPG key we currently use for signing. Does anyone have a model by which we could somehow share it? It might be nice to start "rotating" the responsibility for the releases among the four core committers to help take some work off of Erik's plate.
So, if you accept Maven release to be an official step in ABCL's release process, that would be great for me and I could of course help in setting it up, solving issues etc. Else, if you're ok with it, I can publish ABCL on Sonatype's repo on my own, without making it part of our official release policy; that means less hassle for ABCL, but more risk of stale Maven releases and other problems caused by having the release depend on a single person (of course I'll take the release seriously, but still, depending on a single individual is always problematic). The third option is NOT to release ABCL with Maven at all: if you think that it would hurt ABCL, either by complicating the release process too much, or by having too high a risk of out-of-date releases out in the wild, I will accept that.
As much as I loathe Maven, it is certainly a contemporary commonly-used feature of the Java ecosystem, so I in favor of eventually make it part of our official release, but in measured increments. As steps forward, I would propose that we work on an acceptable mechanism to Mavenize ABCL releases, put it in trunk, and have Alessio make a release of the current ABCL via Sonatype. Then, we can work on making that a part of the official release.
As for supporting even more of the Java ecosystem, I would parenthetically add there are two more potential ways to broaden our reach that I can think of that are lying dormant around my various filesystems. One would be including Eclipse project information in the trunk, sharing most of the build instructions with Ant. The second would be including OSGi packaging information in our jar which I worked with Alan as part of the URI PATHNAME work to implement. My time is very constrained to move these along, especially with testing as Eclipse/Equinox are not in my daily toolbox, but if someone has an ability to test either, I can package up my current diffs for distribution via email.
On 10 February 2011 08:56, Mark Evenson evenson@panix.com wrote:
As for supporting even more of the Java ecosystem, I would parenthetically add there are two more potential ways to broaden our reach that I can think of that are lying dormant around my various filesystems. One would be including Eclipse project information in the trunk, sharing most of the build instructions with Ant. The second would be including OSGi packaging information in our jar which I worked with Alan as part of the URI PATHNAME work to implement. My time is very constrained to move these along, especially with testing as Eclipse/Equinox are not in my daily toolbox, but if someone has an ability to test either, I can package up my current diffs for distribution via email.
Eclipse seemed to import our ant build pretty effortlessly. I haven't needed any ready-made eclipse project file to run/debug abcl with eclipse.
On Thu, Feb 10, 2011 at 7:56 AM, Mark Evenson evenson@panix.com wrote:
On 2/9/11 9:36 PM, Alessio Stalla wrote:
On Fri, Feb 4, 2011 at 12:48 PM, Alessio Stallaalessiostalla@gmail.com wrote:
when I understand better how the whole [Maven] release process works, I'll report here with more information.
Ok, so I experimented a bit with Maven and the free Sonatype repositories. To do a release, you need to produce the jar, sources-jar, and javadoc-jar of your project, and sign them with a gpg key. You then have to publish them to the repository; doing it (i.e. signing + publishing) with Maven is probably the easiest thing, although there's a way to do it with Ant too (but you have to have additional Maven Ant tasks installed on your system).
[…]
I know that Maven can use Ant tasks, so the easiest way forward might be to have Ant continue to build while Maven packages it. Maven has the reputation of strong—i.e. inflexible–ideas about directory layouts that may prove to make this task difficult.
Wait - Maven (by default) requires a certain directory structure if you use it to build your project; but I'm not proposing that, it would be a daunting task with no foreseeable benefits. I'm just proposing to use Maven to "deploy" (i.e. copy to the repository) our already-made releases. You can do it with Ant + Maven tasks too, but imho it's simpler with Maven alone (which amounts to something like mvn gpg:sign-and-deploy-file /path/to/abcl.jar /path/to/pom.xml repo-url, same for sources and javadoc jars).
If we use Ant to package, I would like to continue to have the main build.xml file not have external Ant dependencies to enable the ABCL source distribution to be easily built without any additional downloads (other than Ant). A possible way to implement this would be to have an additional file, say 'maven-build.xml', that includes the main 'build.xml' while requiring the Maven Ant tasks.
That's certainly a possibility.
As far as I can tell, no special format is *required* for the sources and javadoc jars, although the ones generated by Maven of course have a precise format: basically the packed src directory for sources, and the packed doc root directory for javadocs (you can see an example here:https://oss.sonatype.org/content/repositories/releases/com/googlecode/ehcache-spring-annotations/ehcache-spring-annotations/1.1.2/. ABCL doesn't have many javadocs, and anyway those are certainly not the main part of the documentation; but the javadoc jar is required and, if it follows the convention, it can be automatically picked up by Eclipse when you add ABCL as a Maven dependency.
At least this would put some pressure on us to add some more javadoc strings. A reasonable summary of the org.armedbear.lisp package pointing to the Wiki and mailing list might be a decent start.
Agreed.
Morale: this is not terribly complicated, and it would make ABCL available through another distribution channel. OTOH, I can't hide the fact that - besides my own personal convenience for my shared project
- probably no-one is using ABCL in Maven projects right now, and
having ABCL released through Maven would complicate its release process, not much for technical reasons but for organizational ones (e.g. who keeps ABCL's GPG key? How is it managed?).
Currently Erik maintains the GPG key we currently use for signing. Does anyone have a model by which we could somehow share it? It might be nice to start "rotating" the responsibility for the releases among the four core committers to help take some work off of Erik's plate.
Well, if that's already managed somehow, it's ok. I didn't want to complicate the process with additional requirements.
So, if you accept Maven release to be an official step in ABCL's release process, that would be great for me and I could of course help in setting it up, solving issues etc. Else, if you're ok with it, I can publish ABCL on Sonatype's repo on my own, without making it part of our official release policy; that means less hassle for ABCL, but more risk of stale Maven releases and other problems caused by having the release depend on a single person (of course I'll take the release seriously, but still, depending on a single individual is always problematic). The third option is NOT to release ABCL with Maven at all: if you think that it would hurt ABCL, either by complicating the release process too much, or by having too high a risk of out-of-date releases out in the wild, I will accept that.
As much as I loathe Maven, it is certainly a contemporary commonly-used feature of the Java ecosystem, so I in favor of eventually make it part of our official release, but in measured increments. As steps forward, I would propose that we work on an acceptable mechanism to Mavenize ABCL releases, put it in trunk, and have Alessio make a release of the current ABCL via Sonatype. Then, we can work on making that a part of the official release.
Technically to "mavenize" ABCL releases, we need to generate the javadoc and sources jars. All the rest (signing and deploying to the repository) is done by Maven or Ant + Maven tasks; you can't generate the signed files, commit them, and upload them separately, as far as I know.
As for supporting even more of the Java ecosystem, I would parenthetically add there are two more potential ways to broaden our reach that I can think of that are lying dormant around my various filesystems. One would be including Eclipse project information in the trunk, sharing most of the build instructions with Ant. The second would be including OSGi packaging information in our jar which I worked with Alan as part of the URI PATHNAME work to implement. My time is very constrained to move these along, especially with testing as Eclipse/Equinox are not in my daily toolbox, but if someone has an ability to test either, I can package up my current diffs for distribution via email.
I use Eclipse, but I don't know anything about OSGi. If you have in mind specific tests I could do, please tell :)
Bye, Alessio
On Feb 10, 2011, at 10:17 , Alessio Stalla wrote:
[…]
Technically to "mavenize" ABCL releases, we need to generate the javadoc and sources jars. All the rest (signing and deploying to the repository) is done by Maven or Ant + Maven tasks; you can't generate the signed files, commit them, and upload them separately, as far as I know.
So, the first step is to add targets to 'build.xml' to package our source and javadocs?
"That's easy!" methinks…
-- "A screaming comes across the sky. It has happened before, but there is nothing to compare to it now."
On 2/10/11 10:41 AM, Mark Evenson wrote: […]
So, the first step is to add targets to 'build.xml' to package our source and javadocs?
"That's easy!" methinks…
With [r13211][1], the Ant target 'abcl.source.jar' produces a jar source archive that should be suitable for distribution via Maven.
I couldn't find an example of the naming/layout conventions for Maven jars, so I need to hunt around a little more for that.
[1]: http://trac.common-lisp.net/armedbear/changeset/13211
On Fri, Feb 11, 2011 at 7:34 AM, Mark Evenson evenson@panix.com wrote:
On 2/10/11 10:41 AM, Mark Evenson wrote: […]
So, the first step is to add targets to 'build.xml' to package our source and javadocs?
"That's easy!" methinks…
With [r13211][1], the Ant target 'abcl.source.jar' produces a jar source archive that should be suitable for distribution via Maven.
Thanks!
I couldn't find an example of the naming/layout conventions for Maven jars, so I need to hunt around a little more for that.
Naming: the name is given by Maven automatically when you deploy the jar (based on the information in the POM). Layout conventions: I couldn't find a written document explaining conventions. By inspecting existing Maven-generated artifacts, it appears that source jars are laid out exactly like binary jars [1] while Javadoc jars are simply the root docs directory jarred [2].
[1] that is: abcl-sources.jar org armedbear ... META-INF
[2] that is: abcl-javadoc.jar index.html *.html org armedbear ...
On 2/11/11 10:19 , Alessio Stalla wrote: […]
Naming: the name is given by Maven automatically when you deploy the jar (based on the information in the POM). Layout conventions: I couldn't find a written document explaining conventions. By inspecting existing Maven-generated artifacts, it appears that source jars are laid out exactly like binary jars [1] while Javadoc jars are simply the root docs directory jarred [2].
abcl-sources.jar
[…]
abcl-javadoc.jar
[…]
My patches named them as abcl-${abcl.version}-sources.jar and abcl-${abcl-version}-javadoc.jar. Feel free to rename them in build.xml if you need to remove them. The internal layout should match what you expect.
On Fri, Feb 11, 2011 at 1:01 PM, Mark Evenson evenson@panix.com wrote:
On 2/11/11 10:19 , Alessio Stalla wrote: […]
Naming: the name is given by Maven automatically when you deploy the jar (based on the information in the POM). Layout conventions: I couldn't find a written document explaining conventions. By inspecting existing Maven-generated artifacts, it appears that source jars are laid out exactly like binary jars [1] while Javadoc jars are simply the root docs directory jarred [2].
abcl-sources.jar
[…]
abcl-javadoc.jar
[…]
My patches named them as abcl-${abcl.version}-sources.jar and abcl-${abcl-version}-javadoc.jar. Feel free to rename them in build.xml if you need to remove them. The internal layout should match what you expect.
Actually the names you gave them are precisely those that Maven generates.
BTW, I have asked the Sonatype guys to create the project for ABCL; they should do it in a couple of days. Based on what we said on the other thread started by Erik, I could manage the whole deployment process, including signing the jars with my own gpg key (which I already used for Dynaspring). I will publish the already released version 0.24, but not the older ones (I don't think it would be useful). If there are no objections, I will ask them to activate sync with maven central for ABCL, so it will hopefully be available soon to everyone using Maven with no extra configuration required.
Bye, Alessio
On 2/10/11 10:41 , Mark Evenson wrote:
On Feb 10, 2011, at 10:17 , Alessio Stalla wrote:
[…]
Technically to "mavenize" ABCL releases, we need to generate the javadoc and sources jars. All the rest (signing and deploying to the repository) is done by Maven or Ant + Maven tasks; you can't generate the signed files, commit them, and upload them separately, as far as I know.
So, the first step is to add targets to 'build.xml' to package our source and javadocs?
"That's easy!" methinks…
And [13212][1] adds targets for javadoc, so as I understand things, we should have now be able to create all the necessary artifacts for Maven from out Ant build system.
[1]: http://trac.common-lisp.net/armedbear/changeset/13212
I know that Maven can use Ant tasks, so the easiest way forward might be to have Ant continue to build while Maven packages it. Maven has the reputation of strong—i.e. inflexible–ideas about directory layouts that may prove to make this task difficult.
Wait - Maven (by default) requires a certain directory structure if you use it to build your project; but I'm not proposing that, it would be a daunting task with no foreseeable benefits. I'm just proposing to use Maven to "deploy" (i.e. copy to the repository) our already-made releases. You can do it with Ant + Maven tasks too, but imho it's simpler with Maven alone (which amounts to something like mvn gpg:sign-and-deploy-file /path/to/abcl.jar /path/to/pom.xml repo-url, same for sources and javadoc jars).
From the start I was assuming that using Maven was indeed to deploy
ABCL and using Maven as a distribution mechanism. However, I'm now wondering if deployment-only is too restrictive. What happens when a deployed ABCL is downloaded from the Maven repository? The user gets access to it, but may want the option to rebuild the ABCL jar for whatever reason. With the deployment-only option, the user won't be able to do it from his integrated Maven-management environment, correct? If that's true, will 'deployment-only' be a limiting factor for deployment of ABCL through Maven, or is it a step forward and should we just see what comes next?
Bye,
Erik.
On Thu, Feb 10, 2011 at 9:57 PM, Erik Huelsmann ehuels@gmail.com wrote:
I know that Maven can use Ant tasks, so the easiest way forward might be to have Ant continue to build while Maven packages it. Maven has the reputation of strong—i.e. inflexible–ideas about directory layouts that may prove to make this task difficult.
Wait - Maven (by default) requires a certain directory structure if you use it to build your project; but I'm not proposing that, it would be a daunting task with no foreseeable benefits. I'm just proposing to use Maven to "deploy" (i.e. copy to the repository) our already-made releases. You can do it with Ant + Maven tasks too, but imho it's simpler with Maven alone (which amounts to something like mvn gpg:sign-and-deploy-file /path/to/abcl.jar /path/to/pom.xml repo-url, same for sources and javadoc jars).
From the start I was assuming that using Maven was indeed to deploy ABCL and using Maven as a distribution mechanism. However, I'm now wondering if deployment-only is too restrictive. What happens when a deployed ABCL is downloaded from the Maven repository? The user gets access to it, but may want the option to rebuild the ABCL jar for whatever reason. With the deployment-only option, the user won't be able to do it from his integrated Maven-management environment, correct? If that's true, will 'deployment-only' be a limiting factor for deployment of ABCL through Maven, or is it a step forward and should we just see what comes next?
Maven-deployed sources are not buildable in general, they're just meant as an aid to IDEs (to show the source of the Maven-imported libraries). So, a user wanting to build his/her own ABCL and integrate into his/her Maven-based project won't be able to do it from the abcl-sources.jar, and that's not specific to ABCL. Such a user will have to build ABCL and install it into his/her local Maven repository. If ABCL were buildable by Maven, that would amount to a single command (mvn install from ABCL's project directory). It will, instead, require two commands: ant abcl.jar and mvn install:install-file -Dfile=dist/abcl.jar -DpomFile=pom.xml. Or a single one: ant -f maven.build.xml abcl.maven.install provided a suitable Maven-enabled maven.build.xml exists. I still need to understand clearly what installing Maven support for Ant amounts to.
I hope this mail sheds a little more light on these dark matters :D
Cheers Alessio
Bye,
Erik.
armedbear-devel@common-lisp.net
-
Alessio Stalla -
Erik Huelsmann -
Mark Evenson -
Ville Voutilainen