Building a DevOps Toolchain
Building a DevOps Toolchain is challenging. Firstly its difficult to define what you need and secondly its difficult to select from the myriad of choices out there. In our own business we have selected a set of tools that form the basis for our DevOps solutions. But this came about through a combination of accident and design.
Our interest in the DevOps movement, and before that ALM, was grounded in the practical day-to-day running of our IT projects on behalf of clients. Daysha has always been an IT Project Services company and has been vendor agnostic. Our main business was providing experienced project managers and technical experts / teams to deliver client projects.
Daysha and DevOps
But in 2013 we noticed a pattern. Clients were asking us to recommend tools (not the human variety although we’ve come across plenty) to help them deliver their projects.
We embarked on a mission to pull together a set of project lifecycle tools, and this coincided with the emergence of DevOps and an upsurge in demand for Agile practitioners. And this in turn steered us the direction of the toolset we are now recommending.
Our tool set is not complete – a glaring hole is in the area of test automation. And the tool set is probably not suitable for certain niches, but to date we’ve been very happy with it and believe it to be the best in class, and best value right now.
When we started to identify tools the Atlassian tool suite became an obvious choice. We had years of experience with it through client projects and Atlassian were a dream to deal with, their partner program and support is well thought out and delivered, and there was a ready-made customer base in Ireland.
The Atlassian tool suite covers a broad spectrum of the requirements on a project. But there are obvious gaps in the areas of QA, Infrastructure Management and Operations. Whilst the Atlassian Marketplace provides many extensions, they don’t really go far enough in many cases, and Atlassian not trying to be all things to all customers.
The next product to catch our eye was AppDynamics. If you haven’t heard about it then stay tuned as we go into a deep-dive demo and blog post later in the series. In the meantime you can find out more here.
The infrastructure management space seemed to be where the most ‘DevOps’ related activity was happening. The emergence of Infrastructure as Code as a tenet of DevOps automation, and the proliferation of Container technology, made for a very confusing landscape. But after studying some of the competing products, after taking the pulse of the market, we decided that Chef was the most suitable product for us to take on. We found it relatively easy to understand (our background is mainly software engineering with a smattering of ops), and we found Chef as a company to be very approachable and supportive.
We had also seen that one area of the market that was ripe for disruption was the Test Case Management space. We had seen cheap cobbled-together solutions involving JIRA and Excel spreadsheets, and expensive solutions like HP Quality Center. But we had not come across many users that were satisfied with any solution. Everything seemed unnecessarily clunky and not well integrated.
In the Agile world, Dev and QA are supposed to work closely together – collaborating -it made sense to us that we would look at tools that closely integrated with Atlassian JIRA and JIRA Agile, since this is what we were recommending to our clients. Zephyr came to our attention. We had already sold their JIRA add-on to one of our customers, and shortly afterwards they released their enterprise test case management platform. This seemed like the perfect fit.
The Complete Toolset (Almost!)
So at this stage we felt like we had a reasonably complete and well integrated tool set that could be used to implement many of the recommendations and practices of the DevOps community
- We had a strong collaboration platform centered on a single common, open platform from Atlassian.
We had strong automation capabilities across all products
- We could implement workflow and common development practices across both Dev and Ops teams with a single common platform. The guys developing the application code could use the same set of tools and reporting as the guys developing the infrastructure code.
- The end to end coverage and visibility promotes ownership of the entire solution.
- The ability to get accurate diagnosis of issues in development and production, means that issues are, not seen as the big deal they used to be – yes they arise but they can be dealt with swiftly in a (hopefully) blameless culture.
- The toolset provides all the building blocks for continuous delivery pipeline.
4 Pillars of DevOps Tooling
In the Define & Plan stage we use the 3 Atlassian Products – Confluence, JIRA, and JIRA Portfolio. These 3 tools can work together very well to help those responsible for planning, designing, funding, and delivering a project/product, to first collaborate and then to visualize the work involved.
In the Develop and Test stage we turn to JIRA Agile and Stash as our go to solutions for Dev teams. Some dev teams still use Subversion, and this is covered with Fisheye (and Crucible for code reviews). Some dev teams opt to measure code coverage of their testing – enter Clover. And others go all out for SaaS based solutions and so prefer to have Atlassian host JIRA/JIRA Agile and use Bitbucket for source code control.
As we move into Build and Deploy, the DevOps rubber really hits the road. (Strictly speaking Chef should appear in the Develop and Test phase but we are taking some artistic license to balance our diagram here, and it probably sits better with those just transitioning to DevOps practices). We recommend using Bamboo to setup a full live-like as possible Continuous Integration environment. We further use Bamboo with Chef to automate provisioning of disposable infrastructure for test environments, and promote builds through a release management workflow.
And finally once our system is live, we need to close the loop. We need to constantly monitor performance and operations of our applications. We recommend AppDynamics for this. And if issues are found through automated mechanisms, or by customers, we use JIRA Service Desk to manage and triage these problems, passing them back into the development cycle as needs.
In the following blog series we are going to take you through the 4 stages and explain how the different tools are used. At different points we may take a few diversions to discuss topics like Docker, Log management, data analytics, and test automation.
In the next blog we will introduce how we think Confluence and JIRA can be use together to manage requirements.