22 June 2015
Disclaimer: I have 1 year experience with Jenkins and 0 experience with Bamboo, so I might be wrong on certain points Bamboo. Open for feedbacks in the comments.
We have just adopted Stash in our infrastructure, that means another Atlassian product adoption. There are motivations to sync up all of our infrastructure to use Atlassian products because they are known for their seamless integration. This research is done to explore the possibility of replacing Jenkins with Bamboo.
Winner: Bamboo
As an Atlassian product, Bamboo wins the fight hands down. It integrates with JIRA, Stash and FishEye seamlessly. All of the connections are maintained consistently in application links like any other Atlassian products. You can find the massive list of integration features here.
Jenkins has rough edges here. Jenkins can integrates with JIRA, Stash and FishEye as well but they are all maintained separately as a plugin. I would say most of the important features are available, but they are not implemented prettily.
The below are the integration features that we think are important in Bamboo and see how it’s being supported in Jenkins.
| Product | Integration feature | Jenkins support |
|---|---|---|
| Stash | Stash tells Bamboo when to build | Yes. Can be done by poll out of the box. Push trigger requires plugin config: webhook & stashbot |
| Stash | Stash tells Bamboo when to update plan branches to match changes in repository branches | Yes. Use job-dsl-plugin to hit Stash REST API. workflow-plugin claims to support this as well. |
| Stash | Stash commits are displayed in the relevant Bamboo builds | Yes. Supported out of the box by Git Plugin. |
| Stash | Bamboo notifies Stash automatically about build results | Yes. Supported via stashNotifier. It’s hitting Stash API. |
| JIRA | Deployment Panel | No. We can use JIRA REST API to add a comment at the very least. |
| JIRA | Viewing build results in JIRA | Yes. Supported by a Stash plugin or a Jenkins plugin. |
Winner: Tie
The user experience quality of Bamboo is very good. First of all it’s consistent with the rest of the Atlassian products. For a first time user of Bamboo, I manage to understand most of the features and it’s guiding you through all of the process of creating a new plan nicely. Atlassian documentations are very good for beginners as usual. The feature that I’m most impressed about is how it treats deployment as first class. The concept of Stage that replaces Jenkins downstreams are also very nice. Bamboo also has multiple branch build support out of the box. All of these requires manual configuration in Jenkins.
Bamboo fall short on some common tasks e.g. it can’t display Cobertura report. The parameterized build can’t be forced and can only be run via Run customized.
Jenkins UX on the other hand has a steeper learning curve and setup. Though once you are used to it, you will gain more features from it’s vast amount of plugins. Everybody knows that Jenkins UI is ugly though it’s been improving over time with the recent versions. The CSS is also configurable with a theme though that will not fix the entire UX.
Jenkins can only be complete with its plugins. Basic features are readily available out of the box and overlooked by Bamboo. For example customizing maven settings.xml via UI configuration. Or clearing down workspace which is very handy when they are dirty. Also for the downstreams, Jenkins has a Build Pipeline plugin that provides a full blown view on how the jobs are connected, this is all missing in Bamboo.
However due to it’s vast number of plugins, it can be a daunting task for a beginner to figure out which plugin actually works. A wrong combination of plugins might break your build as well. It will take time for the users to be familiar with the interface. Because the plugins are developed freely, some of them are having inconsistent UI.
Winner: Jenkins
Jenkins has job-dsl-plugin and workflow-plugin. These two plugins enables build engineers to orchestrate Jenkins with Groovy DSLs (and templates). All of the DSLs can be stored in VCS and changes can be properly audited and reverted if needed. job-dsl-plugin also supports job templates which reduce a lot of manual configurations. The other big win is that we can attract simple change to be done by developers because it’s all Java/Groovy code. Besides the DSL, Jenkins has a script console where the build engineers can execute any arbitrary Groovy scripts to configure the jobs. You can even execute these scripts remotely from your favourite IDE.
Not supporting template is one of the reason users move away from Bamboo. Bamboo however, has Bamboo CLI to overcome it’s bulk change limitation. Bamboo CLI interacts with bamboo remotely with a series of command lines execution. Even though you can add an arbitrary task with this tool, it’s very hard to determine how to add a new task with it hence lacks of flexibility.
Jenkins stores it’s configuration and build result as XML, Bamboo is storing them in RDBMS. The XML approach makes Jenkins very flexible and portable. You can easily migrate Jenkins jobs across multiple instances just by copying the configuration. You can’t do this in Bamboo. Storing the configuration in XML has enabled Jenkins to be configured if they are not available in the UI such as reseting build number that Bamboo can’t do. Also some of the system administrators who are more confident with their sys admin skills can configure the jobs by changing the XMLs directly.
Winner: Jenkins
To scale out CI servers, more remote slaves are needed. The more agents or slaves you have, the more horizontally scaled you are. Bamboo license are priced per remote agents. You can have unlimitted amount of slaves in Jenkins OSS. Jenkins enterprise are priced per slaves as well.
The steps to configure the agents in Bamboo are quite hefty. There’s no automatic install of the tools hence you will need to invest in other configuration management tools e.g. ansible, chef or puppet.
Jenkins wins here with the ease of spinning up a new remote agent/slave. You can literally use any empty machine as a slave without any setup. The slave are configured via Jenkins UI and all of the tools can be installed automatically. This is an out of the box feature from Jenkins. It also go to the extend of utilizing unused computing power with Swarm Plugin. This plugin enables automatic slaves discovery nearby Jenkins master.
Shall you be moving to SaaS model for the entire company, a well separation of concerns in between jobs are very important. Bamboo does a very good job here by by having the concept of projects and plans. One project has multiple plans, one plan has multiple jobs (and stages). With this concept, you will be able to support multiple projects well in Bamboo. Sleek!
Jenkins does the separation of jobs with folders or views, which is not as integrated as Bamboo’s concept of project. A common alternative would be for the individual teams to have their own Jenkins. We are definitely moving away from SaaS model if we do this, that’s why CloudBees has Jenkins Operations Center to manage multiple master nodes.
It is important to have your CI server with High Availability as we can’t afford down time if our CI server is being used by multiple global teams. Bamboo doesn’t support high availability/fail over out of the box, but you can achieve it traditionally. This is a first class support by Cloudbees in Jenkins enterprise.
When a software grows, a software will need to be tested in a matrix of platform e.g. multiple databases, webservers, jdk, etc. Jenkins supports Matrix project configuration. Configure one job, and the job will be executed repeatedly in all of the platform matrices configured. This is not supported in Bamboo.
Winner: Jenkins
Bamboo has a clear upgrade documentation though too hefty as compared to Jenkins. You need to do a backup for all of the home directories, babysit the update process and monitor the log files. Bamboo may also require a reindex after the upgrade.
What’s not easier than dropping a war to tomcat? Jenkins is clearly a winner. Jenkins doesn’t change your XMLs during the upgrade as the communities are crazy about backward compatibility. Not happy with the later version? Drop in the previous war. When you are happy with the new version, there is a click of a button away from updating all of your configuration to the latest model.
Winner: Jenkins
Winner: Jenkins
Bamboo is a commercial software. Jenkins comes with two flavors, free and enterprise. The commercial support of Jenkins Enterprise comes from Cloudbees. The price can’t be compared here as Cloudbees is not displaying the price in their site.
Both Jenkins and Bamboo have numerous bugs and you will definitely run into problems. Because Jenkins is open sourced, you are much more likely to have a workaround from the community until it’s fixed. You are stuck in Bamboo with Atlassian support. One of the biggest example is the build template ticket in Bamboo that raised 2 years ago with 99 votes now. In Jenkins, the plugins are supported by the community.
Besides support, Bamboo product naming makes it less Google-able.
Bamboo is easy to use, good looking and has good Atlassian integration. Jenkins is rough around the edges, but far more flexible than Bamboo. When Jenkins is running standalone, it’s obvious that it’s far more superior than Bamboo. Bamboo shines when it’s placed with other Atlassian applications with its good UX.
The decision of the adoption should fall down on the stakeholders priorities. Is integration more important? Is maintainability and scalability more important?
