Ryanair’s DevOps Journey
This blog on Ryanair’s DevOps Journey is based on a presentation made by Alan Coates (Head of Planning and Delivery Ryanair Labs) at the Atlassian European Conference in Barcelona in May 2017.
Also included are links to other Daysha offers and content which are relevant to how Daysha assists Ryanair Labs in their ongoing journey.
Ryanair will tell you their DevOps journey is a work in progress and this presentation shared what went wrong as well as what went right and what still puzzles them. This is a valuable resource for anyone starting or in the middle of a DevOps journey. Many thanks to Alan for doing this presentation at our request to the benefit of Atlassian customers.
One the most striking things about this clients story is the existence of one strong product owner, their CEO. This would seem to be a common aspect of DevOps success as Capital One CEO Scott Fairbanks has had a similar effect on their digital transformation.
The second noticeable contributory factor in empowering the team at Ryanair Labs to get to Continuous Deployment quickly was the business driver. The new ‘always getting better’ image Ryanair wished to project had to be backed up with a digital customer engagement that could nicely be described as ‘poor’ back in 2014.
The third and perhaps the most important factor was the absence of any pre-existing IT culture. There were a small number of strong developers and a top operations team but no silos and no waterfall behaviors. When the newly arrived Head of Ryanair Labs took the helm he commented ‘’having no culture was a huge advantage – we knew what good looked like so we could hire into that mindset” John Hurley CTO Ryanair Labs.
In his presentation Alan makes reference to the fact that Ryanair has the lowest average fare in Europe, and behind that is an all-embracing lean business culture. This helped create Lab’s own culture, taking the Ryanair Lean mindset into how software would be built as Ryanair played digital catch up with some of the other low cost carriers.
Forming a picture of what good looked like was an area where the Daysha team helped. Daysha attended various DevOps events in the UK and US, and relayed case studies into John’s team. Many of these case studies now form part of Daysha’s blog. This flow of information also worked the other way around and many of the links on our DevOps learning’s pages part of our site came from the team at Ryanair.
‘I retained all the Daysha emails with relevant case study information and revisited them periodically as we encountered new challenges’ Alan Coates, Head of Delivery Ryanair Labs.
Alan contends that DevOps culture still challenges them. So, for example, empowering teams to own code into production sounds straightforward, but recently they realized that their own DevOps team had become a silo and a bottleneck. Now, scrum teams own their code and release scripts as an atomic unit and each dev stream has its own assigned release engineer who is also a dev and who helps the other team members get the code into production as and when required.
More than once during their journey Ryanair Labs likened what happened to Cargo Cult.
Without truly understanding what good looked like, people behaved as if they did until it was clear they hadn’t internalized the impact CD would have on their day to day work. This wasn’t necessarily all bad as it illustrated that people were willing to change but needed help on what exactly it meant.
The training Daysha provided both soft and hard was of real value. Soft skills was about getting the culture right. Hard skills related to tooling especially Chef and Atlassian.
We really wanted to get dev and ops teams on the same page. We had them on the same floor and as far as we knew they engaged. We ran a private in house DevOps foundation course which Daysha had just started to deliver back in 2015 and then we started to use their public courses. Having Dev and Ops employees attend the courses together helped break down barriers and build a mutual understanding of where we needed to get to. Alan Coates, Head of Delivery Ryanair Labs.
Starting 2014, and moving fast, Ryanair Labs hired on 200 people spread across Dublin and Poland (and now opening in Madrid). They had a new mobile app, website, and personalization (and the rest) to build… all aimed at making Ryanair the ‘Amazon of Travel.
Agile was always going to work for the customer facing applications as the cut and thrust of airline competition means that to be successful you must be able to build/release and roll back if required within a matter of days….if not hours.
Features are built based on anticipated value to the business but if they don’t deliver – its fail fast, learn the lessons and move forward. Equally if the feature works well then Labs roll out by culture (Ryanair groups by language not country). Of course not all cultures behave the same way; for example Italians tend not to buy car insurance at the same rate as English people.’
In keeping with well-equipped labs Ryanair ensures teams are empowered and provided with the best tools. They also have an open culture and share build and operational information through screens and on white boards.
The current status is as follows:
- Speed of delivery to production has evolved from a single delivery/week to multiples deployments in a day.
- Information radiators, build screens & dashboards now in prominent areas of the development floor, moved from infrastructure to web monitoring.
- Traditional Infrastructure engineers are being reborn as SRE’s– higher up in the Systems stack.
- They are now comfortable with trending towards a cloud-first strategy for any new app deployments
More importantly the customers are happier!
Ryanair Labs has always embraced new technology and the future is no different as they look at wearables, Alexa, paperless cockpits and a raft of ideas all designed to improve the home to hotel bedroom experience for their Customers.
Alans parting advice boiled down to:
- Individuals and interactions over tools and processes and tools
- Small changes frequently
- Don’t be afraid to break things