Blogs

SpringSource Blog

Spring project infrastructure updates

Chris Beams

Introduction

Over the last year a number of significant changes have been made to the infrastructure and processes we use to keep the Spring family of projects running smoothly. You may have seen individual announcements about some of these as they happened, while others may have slipped under your radar. I'll recap these changes below. When put together they portray a bigger picture.


GitHub project hosting

Individual Spring projects have been migrating to Git and GitHub for quite a while. You may recall our announcement last Christmas that the Spring Framework itself had made the move. With the recent migration of Spring Web Flow, we're happy to announce that all major Spring projects are now hosted under the SpringSource organization at GitHub.

There are benefits for project committers and Spring users alike following the move to Git and GitHub. GitHub has an excellent UI for code browsing, history of changes, and commit comments. And with the amazing number of open source projects already hosted at GitHub, this means that you're using one well-understood UI and that you already know how to browse source control, examine recent changes and so on. But GitHub's real power is in the way it encourages and supports community contribution. This point is discussed further in the "contribution process" section below.

For now, check out the SpringSource organization at GitHub if you haven't already; you'll see we've been busy as usual.


Artifact management

The SpringSource repository at http://repo.springsource.org is an Artifactory instance dedicated to hosting snapshot, milestone, RC and GA versions of Spring project artifacts, as well as their transitive dependencies.

GA versions of Spring artifacts will continue to be published to Maven Central as always, but repo.springsource.org replaces our previous infrastructure at maven.springframework.org with a much more powerful solution — proper UI and search functionality, deep integration with our CI server at build.springsource.org, and many other features that help project teams streamline their release processes.

The upshot for Spring users is that the SpringSource repository is one-stop shopping for all your Spring dependency needs.

If you have maven.springframework.org URLs in your build scripts, they will continue working indefinitely* but these URLs should be considered "deprecated" in favor of repo.springsource.org going forward.

Further explanation and instructions for use can be found at the downloading Spring artifacts page in the spring-framework GitHub wiki. The SpringSource repository FAQ should be able to answer any questions that come up. If not, let us know!

A big thanks to the team at JFrog — the makers of Artifactory — for working with us to make this happen. Switching repository infrastructure across so many busy projects begs for disaster, but with their help it's been smooth. And thanks also to the team at Sonatype for their consistently prompt and helpful responses to our needs around Maven Central.


Gradle-based builds

Moving our builds from Ant, Ivy and Maven to Gradle is another migration process that has been happening on a project-by-project basis for quite a while. And with Gradle's 1.0 GA announcement still fresh in the news, it's a nice time to mention that the majority of Spring projects are now built with Gradle**.

Gradle has many innovative features that make it a pleasure to use, like an incremental build system that significantly cuts down on build times, minimalistic command-line output that shows you only what you need to know, and a concise syntax that makes it possible to build a large codebase like the Spring Framework with a remarkably small and readable build script.

For Spring users, the system used to build Spring projects may not seem to matter much. Indeed, just like most Linux users don't compile their own kernels, most Spring users never build Spring artifacts from source. But like any good open source project, some do. Whether simply to understand Spring better or to make changes and contribute back to the project, the number of people that build Spring from source is growing, thanks in part to the way that Gradle makes doing so quite trivial. One of Gradle's great features is the so-called Gradle Wrapper which is essentially a platform-native shell script that downloads and installs the exact Gradle version used by a project. This means that building a Gradle-based Spring project from source is literally as simple as typing:

$ git clone https://github.com/SpringSource/spring-framework.git
$ cd spring-framework
$ ./gradlew build

As you can see, the 'gradlew' script there is checked into project source control. So as the version of Gradle used to build the project changes, the wrapper stays up to date and new versions of Gradle are downloaded automatically when necessary. This means never hearing "the build is broken" only to find out that two different versions of the build tool were being used. And it means that Spring projects can immediately keep up to date with the latest changes to Gradle without risking breaking other committers' and contributors' builds.

Check out the complete (but still very simple) instructions in the building from source section of the Spring Framework readme. Compare the 100 words you see there to the 1,500-word blog post on Building Spring 3 that I wrote a few years ago. I hope you'll find it's quite an improvement. Give it a try — you might even like it:

The Gradle team have been a tremendous help in this process — Spring projects have regularly pressed Gradle to the limits and challenged the team with requirements over the last couple of years, and the Gradle team have been nothing but responsive and helpful. Many of the Spring projects are certainly better off today for Gradle, and we hope that Gradle itself is at least a little better for the challenges we threw at it. Congratulations to the team on their 1.0!


Contribution process

As I mentioned above, one of the best things about GitHub is the notion of pull requests. Read up if you're not already familiar, but suffice it to say that pull requests are like patches and code reviews all rolled into one tight process and simple UI. Take a look through the pull request histories for Spring Integration, Spring Framework, and many of the other Spring projects and you'll see plenty of interesting and useful examples. This process is already much better than attaching patch files to JIRA; when you combine it with the power of Git and the ease of building with Gradle, it means it has never been easier to contribute to Spring projects.

Since the Spring Framework moved to GitHub at the end of 2011 and to the time of this writing in June 2012, the project has received just over 100 pull requests. Not every pull request gets in. The outcome depends on the review process, but nevertheless many of these contributions have made their way in, sometimes after significant discussion and refinement. We've put together a contributor guidelines document based on what we've learned over this period; give it a read if you have something you'd like to give back to the framework. This is a great way of getting your issue to the front of the line. We naturally give a bit of priority to users who take the time to put together a high-quality contribution. Thanks to all those who have done so already, and thanks in advance to future contributors!


Mailing list for contributors

Since we began processing pull requests, we've noticed great discussions taking place. In a few cases they contained questions that might have benefited from a discussion prior to submitting the pull request. Furthermore, these discussions remain somewhat under the radar and visible only to those following the individual pull request. Undoubtedly many of you would like to be able to follow such discussions and participate if relevant. The benefits of having a channel for framework contributors are many.

We're glad to announce with this blog post the new spring-framework-contrib mailing list hosted at Google Groups. This list is specifically dedicated to developers who want to contribute code to the Spring Framework project. It's a place where you can talk directly with the committers and other contributors about your ideas — before or while writing code. We encourage anyone thinking about putting together a pull request to join the group and discuss your ideas with us there.


Issue lifecycle

Thanks to its many users, the Spring Framework project has a particularly high volume of issues; it's predictable we'll see the 10,000th JIRA issue sometime later this year. With that many issues coming in, and over the diverse featureset of the Spring Framework, you can imagine how difficult it might be to stay on top of all incoming issues. Over the last year, we've crafted a process of triage and backlogs that help everyone involved understand exactly what's going on with each issue. This has been very helpful to us and we've described it in the lifecycle of an issue so that everyone can understand this process.

You'll also notice a number of smaller changes around the Spring Framework JIRA project: A revised set of components, simplified issue creation screen and a number of other tweaks designed to make issue reporting as useful and painless as possible.


Community forums

For years, the Spring Forums have been and continue to be a great place for Spring-related Q&A. At the time of this writing there are over 100,000 threads (nearly 400,000 posts total!) by over 100,000 members. And that's saying nothing of the guests who visit the site by the thousands every day without ever logging in. No doubt this will continue to be a valuable resource for a long time to come.

There's another Q&A resource we'd like to make sure everyone in the Spring community is aware of. As you probably know, over the last few years Stack Overflow has become an extremely popular resource for programming Q&A. And Spring-related Q&A has been no exception to the phenomenon. If you visit http://stackoverflow.com/tags and type in "spring", you'll see there are many Spring-related tags already there, with spring and spring-mvc being the most popular. I and other members of the Spring team have often been impressed by the quality of conversation at Stack Overflow, and we're very happy to see and thank everyone providing great answers.

A tag-based system like the one at Stack Overflow is by nature a little harder to define than the strict project-oriented categories offered by the Spring Forums, but we'd like to suggest that if you have Spring Framework-specific questions, please use the 'spring' or 'spring-mvc' tags as appropriate. If you have Spring Integration-related questions, use the spring-integration tag and so on. It's pretty self-explanatory, really.


Summary

Hopefully you'll agree that the changes above make it easier than ever for all of us to work together and that's a key factor to the success of any open-source project. We encourage you to take time to review the resources mentioned in this post and to experiment and take advantage of what is available to you. Do keep in mind that contributions take on many forms — whether creating a well-defined issue in JIRA, participating in discussions, or submitting a pull request, you're helping to advance the conversation and the community as a whole!


Footnotes

* The older maven.springframework.org URLs continue to work because the DNS entry for maven.springframework.org now points to repo.springsource.org, and the /snapshot, /milestone and /release paths now map to 'virtual repositories' with the same names and contents at repo.springsource.org.

** Individual Spring project teams are free to choose the build tool that suits them best. This means that while many projects have moved to Gradle, it's predictable that other projects will stick with Maven.



Similar Posts

Share this Post
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • LinkedIn
  • Slashdot
  • Technorati
  • Twitter
 

9 responses


  1. One of the benefits of OpenSource is *transparency* – you always know what's (and why) is/was/will be happening. Seeing all Spring projects (although there's no Spring-WS yet :( ) in one place is a great news for Java Enterprise Community!


  2. Another praise.

    As I've once written, that Spring-Integration should be the "reference implementation" of "Enterprise Integration Patterns" book, now I can say that Spring-Framework (et al.) shout be the "reference implementation" of "Beautiful Code" book :) .


  3. Let us know if you are interested in having your Jenkins CI server hosted at CloudBees, we are already hosting plenty of FOSS projects! (http://www.cloudbees.com/foss/index.cb)

    Cheers,

    Sacha Labourey


  4. Chris, can we have a single repository group with combines snapshot, milestone and release all together? There is /repo but it combines proxied items too.


  5. @Michael-O

    We'll consider that; you're the first to ask for it so far. Did you notice the SpringSource repository FAQ [1] ? You'll see there that we don't recommend anyone use /repo (it's not even mentioned actually), but we do provide /libs-release, /libs-milestone, and /libs-snapshot, which successively aggregate each other and all necessary transitive dependencies. This means that when resolving against /libs-snapshot one has access to all snapshot, milestone, RC and GA Spring artifacts, as well as their dependencies. It sounds like you want combined access to Spring artifacts, but in a way that keeps transitive dependencies segregated. This sounds reasonable, though it would be helpful to hear more about your use case.

    [1]: https://github.com/SpringSource/spring-framework/wiki/SpringSource-repository-FAQ


  6. @Chris

    Yes, I read the entire FAQ that's why I made this request. I found /repo while browsing your Artifactory instance.
    My usecase is: I want to avoid to maintain serveral proxied repos in our Nexus instance but only one single remote repo which contains all Spring artifacts for production and development. The virtual repo must be completely selfcontained besides those artifacts which are in Maven Central.

    Most of the time we use release or RC/milestone components and are happy with them. I would be happy without snapshots too. The former would suffice.


  7. Hi,

    I've got a couple of questions on your build processes. I work for a software company and we have started to use Gradle for our build with an Ivy repository for our dependency management so
    I'm curious on the setup in your development team.

    How do your development teams manage all the various dependencies in their IDE? For example I see you have CI jobs to upload daily snapshots of various modules. Do your developers get notified to
    retrieve a new version of the snapshot or do you have some IDE plugin to retrieve the new versions from Artifactory (or do your developers simply have all projects open and synched from your source control with project dependencies in your IDE)?

    Also, how have you setup the dependencies in your CI server? For example if you have module X which depends on module Y (which is in your Artifactory repository), how is your CI server setup to
    rebuild module X when Y changes – have you had to manually setup project dependencies in your CI server so X gets rebuilt when Y changes? A lot of the dependency information is in our Gradle scripts and it seems like we are duplicating this with job dependencies in Jenkins.

    Sorry to bombard you with questions but we are struggling with some of this so wondered how things worked internally in your development teams.

    Thanks.
    Richard.


  8. @Richard,

    > How do your development teams manage all the various dependencies in their IDE?

    We use `gradle eclipse` to generate .classpath and .project files, and we do not check them into source control. The paths generated there are specific to the user's $GRADLE_HOME directory. Some teams also use the STS Gradle tooling that is currently under development. It can create classpath containers, etc, much like the Eclipse Maven tooling does. It's already quite capable, but can't yet cope perfectly with some of the more complex bits of the spring-framework build, so we just keep it simple there.

    > For example I see you have CI jobs to upload daily snapshots of various modules. Do your developers get notified to retrieve a new version of the snapshot or do you have some IDE plugin to retrieve the new versions from Artifactory

    New snapshots are automatically resolved and downloaded by Gradle. This is a hands-free process for developers. In this respect, our Gradle builds work very similar to the way Maven does. A build declares it depends on org.springframework:spring-foo:3.2.0.BUILD-SNAPSHOT, and Gradle simply makes sure that the latest snapshot is retrieved and used properly.

    > Also, how have you setup the dependencies in your CI server?

    We haven't. Our projects tend to be build frequently enough that integration problems are caught quickly anyway. One certainly could set up dependencies via the CI server, but it hasn't been worth the overhead for us in most cases thus far. If CI dependencies were automatically inferred from published Maven metadata or Gradle build scripts, that would be nice, but as far as I know it requires a duplication of effort (at least in Bamboo), with minimal return on the maintenance investment.


  9. Hi,
    We currently use eclipse, gradle and ivy, but we've been having a few problems with the ivy eclipse plugins and are currently investigating maven.

    What were the reasons behind your migration away from maven?

    Thanks,
    Steve

One trackback

Leave a Reply