In my last post, How to DevOps? Part 1: Why DevOps?, we discussed the why of a DevOps transformation. In this post and the rest of the 7-posts series, we will look into the how and using what tools and techniques.

The basics: the mindset

To take on any important transformation journey, the mindset of all the people involved should be aligned with a continuous improvement mindset. Errors will be made, wrong decisions will be taken, and that’s OK… as long as the teams learn from this and move on and get better. This requires an environment where team members are trusted, where failure is accepted, when failing fast is in fact the best possible way to fail (certainly better than failing too late!). Such an organization has the pre-requisite to be a learning organization (which is essentially the 3rd way of lean: see this post on this topic). And like individuals, a learning organization is an organization that progresses, that can distance its competition. With fear of trying, fear of failing, innovation is dead before the race even starts. Multiple tools can be used to foster such an environment:

Agile

Agile and Scrum provide a framework to organize work in a way that is transparent, that help in trusting the teams to deliver what is needed, by letting team members think about the how. Scrum is iterative by nature, permitting fast feedback loop with final users of the software being developed. The Agile origins are in the Agile Manifesto, where Agile is considered a mindset, described by 4 values, defined by 12 principles and manifested by multiples practices.

The 4 Agile values

Agile practitioners value:

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

It is important to notice that the manifesto explains that while there is value in the items on the right, we value the items on the left more. So Agile does NOT mean no documentation, no planning, no processes, etc. That is what we see the most when discussing Agile, but there is more…

The 12 principles defining Agile

There is a lot in those principles. To me, this is the real substance of Agile:

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
3. Deliver working software frequently.
4. Business people and developers must work together, daily, throughout the project.
5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
7. Working software is the primary measure of progress.
8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence and good design enhances agility.
10. Simplicity–the art of maximizing the amount of work not done–is essential.
11. The best architectures, requirements, and designs emerge from self-organizing teams.
12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.

Two very good images from Zen Ex Machina presenting The Agile mindset, differences between Doing Agile and Being Agile, and depicting the values, the principles, the practices and the frameworks, such as Scrum, using assembled groups of practices:

SCRUM – one of the Agile frameworks

Scrum is a probably the most known Agile framework permitting to adhere to the Agile values and principles via a set of practices, processes and roles. One good image presenting Scrum from Neon Rain is this one:

In which you can see all the Scrum Roles:

• Customers (internal and external), emitting their requests towards the Product Owner
• Product Owner, interfacing with those customers and putting their needs into a prioritized Product Backlog
• The Team, taking those items and creating sprint backlogs and operating iteratively on them, having daily meeting to keep the focus on delivering the sprint goals and delivering a finished work product after every iteration.
• Scrum Master, guardian of the Scrum process and ensuring that it is properly followed

You can also see the Scrum ceremonies:

• Backlog grooming done by the Product Owner, ensuring that items prioritized, from top to bottom of the Product Backlog, and with exit criteria clearly defined for the Teams.
• Sprint Planning, in which the top portion of the backlog is taken by the teams, broken down into tasks, and entering the next sprint for development.
• Daily Standup meeting, in which the team can communicate what was done yesterday, what is planned for today and if there are any impediments preventing proper work to be done (the Scrum Master should help remove the impediments as part of his role).
• The Sprint Review, or often called Demo, at the end of the iteration, to show the produced increment of the product, ideally to real users, in order to get their thumbs up/thumbs down so the team can adjust in future sprints.
• The Sprint Retrospective, for the team, by the team, to reflect on what went well in the last iteration and should be continued, what did not work and should be stopped and what could be started to get even better.

The others

• Kanban is essentially like Scrum, but with a sprint of one day; typically useful for teams more in a support mode than in a pure development mode.
• ScrumBan is a Kanban, but that wants to keep the Scrum end of sprint ceremonies (demos, retrospectives) after every couple of weeks, so the daily Kanban focus can be compensated with a more global focus every once in a while, permitting to also work toward a vision/mission.
• SAFe is a promise to scale Scrum to multiple Scrum teams; Spoiler alert: scaling Agile is not easy, there are no miracle recipes…

My advice is to start with Scrum if you do Development, start with KanBan if you mostly do support and migrate to their variants as you need to and as your teams mature in agility.

Lean Engineering concepts

Another way to have the proper mindset when entering a DevOps transformation, is to take the Lean Engineering mindset. Essentially, Lean is about:

• Accelerating Development from left to right (which is a lot about automating things and optimizing for the global system, not the local ones):

• Adjust by putting in place the proper feedback loops (which is about measures and metrics) :

• Creating a learning/efficient organization, by reducing the batch size, which essentially means working in as small iterations as possible:

If I want to put Lean in 1 sentence: The art of reducing waste, i.e. developing what needs to be delivered, the right and efficient way, not more, not less. I covered in more details Lean in this Lean Engineering and the DevOps movement post.

Source Code Management (SCM)

The mindset is very important as we discussed just above; we could consider this as “soft”/human related practices. More technical practices are however required to enable the subsequent DevOps related practices. The very first of those practices is certainly Source Code Management. The concept of keeping all source code under a revision control system such as Git, Mercurial, etc. and being able to track every single code change over time is crucial. Everything in the following posts of this series is based on strong Configuration Management (CM) practice. There are advanced techniques now bringing even more code into SCM, such as

• Infrastructure As Code, the idea to provision and orchestrate complete infrastructures using coded information into SCM,
• Data As Code, the idea to provision data or test data on demand, programmatically, are all based on SCM concepts.

The more you have everything under SCM, the easier crucial things like High Availability (HA) and Disaster Recovery (DR) will be. If all is under source code control, you can automatically recreate all the elements of your product stack. Nothing is hidden in the minds of one team members, required to be done manually, the time to re-create things is optimized, since all automated, etc. I like the Automate All the Things slogan, and this requires another one: Everything As Code.

The tools

Technically, you can apply the SCM best practices using pretty much any source code version control system (VCS). The old ones like Subversion, CVS, etc, still work. But in the last years, Distributed Version Control Systems (DVCS) appeared, making it easier to collaborate efficiently across different locations, time zones, facilitating forks, merges, etc. Git and its variant offerings, GitHub, BitBucket gained a lot of traction. Those tools are free, efficient and get the job done. Passed is the time where it made sense to pay for commercial SCM tools. The open source ones are excellent. It is up to you to leverage them!

Conclusion

With the basic things in place, Agile and Lean mindsets, source code control tools mastered with Everything As Code, you are now ready to take on the next step : Continuous Integration!