DevOps - Tools and Technology

This post is the continuation of the DevOps - Operational model post in this blog.

We have seen how DevOps processes and organization can help the agility of IT, enabling a huge value for the business.

Let’s investigate the tools that smart organizations use to implement DevOps in the real world.

And let’s try to understand how, in addition to code management, the lifecycle of a sw application can be optimized by managing the infrastructure as code.

At the end of the day, we want to apply the following picture to the infrastructure as well.

 

Usually different environments are created to run application, often cloned for each Tenant (customer, project...): development, integration test, QA test, production, Disaster Recovery

The infrastructure must provide similar topology and functions, with different scale and HA requirements.

Those environments are sometimes used for few days, then they are no longer needed and the resources could be reused for next project.

If we were able to generate a new environment "end to end" when it is required, and to release all the resources to a shared pool, this  would help a lot in the optimization of resources usage.

The economy of scale provided by shared infrastructure and resource pools will add to the simplicity and speed of the operations.

The following picture shows the cycle of the builds (for both the sw application and the infrastructure) that optimizes the time and the resources.

From http://www.rightscale.com/blog/cloud-management-best-practices/continuous-integration-and-delivery-cloud-how-rightscale-does-it:

There are a number of tools and solutions that can help automating this process.

Some apply to specific phases, other to the end to end DevOps.

You can find a good list at http://en.wikipedia.org/wiki/Comparison_of_open-source_configuration_management_software

Also collaboration tools help the team(s) to work together for their own and the entire company's benefit: from http://www.collab.net/solutions/devops

The most used DevOps tools, as far as I know from direct experience and investigation, are jenkins, vagrant, puppet, chef.

Here is another possible chain of tools that cover the entire process:

From http://pages.cloudbees.com/rs/cloudbees/images/Continuous-Delivery-Infographic.pdf

Stateless Infrastructure (also known as SDDC)

We understood that the maximum benefit comes from being able to create and destroy environment on demand, allocate resources just when needed (we can also consider Disaster Recovery as a important use case in this scenario, but in that case you should also ensure that data have been replicated before the event).

Infrastructure as code is a core capability of DevOps that allows organizations to manage the scale and the speed with which environments need to be provisioned and configured to enable continuous delivery.

Evolving around the notion of infrastructure as code is the notion of software-defined environments.

Whereas infrastructure as code deals with capturing node definitions and configurations as code, software-defined environments use technologies that define entire systems made up of multiple nodes — not just their configurations, but also their definitions, topologies, roles, relationships, workloads and workload policies, and behavior.

Stateless Computing and Stateless Networking are important innovations that some vendors (Cisco could be considered a leader here) brought to the market in last 5 years.

Policy based configuration and the availability of software controllers for all the components of the architecture allow the separation of the modeling from the physical topology.

Servers

As an example, UCS servers (up to 160 in one management domain, but domains can be joined to share resources and policies) are stateless.

You can imagine each server (either a blade or a rack-mount server) as a dumb piece of iron, before you push its identity, its features (e.g. number, type and configuration of the network interfaces) and its behavior as a piece of configuration.

It is like adding the soul to a body.

Later you can move the same soul to a different body (maybe more powerful, such as from a 2-CPU server to a 4-CPU one). The new machine will be restarted as if it was the same.

This can be useful to recover a faulty server, to do DR but also to repurpose a server farm in few minutes (and eventually restore the previous state the day after).

The state (identity, features and behavior) is defined by a XML document that can be stored, versioned and managed as code in a repository (other than in the embedded UCS Manager).

This abstraction of the server from the actual machine makes the management easier and was the main factor for the incredible success of UCS as a server platform.

Networks

Similarly, in the networking domain, we have had a quantum leap in network management with Cisco ACI (Application Centric Infrastructure).

For those that have not met ACI yet, I have published a “ACI for Dummies” post.

In few words, ACI brings the management of physical and virtual networks together.

It has a very performant and scalable fabric, made of spine and leaf switches, that are managed by a software controller called APIC.

APIC also integrates the virtual switches in the different hypervisors, so that its policy model can be extended to the virtual end points.

A GUI is provided to manage APIC, but essentially you would drive it through the excellent open API offered to orchestration systems and - of course - DevOps tools.

XML (or JSON) artifacts can be stored in a repository as code, and pushing them to APIC will create your new Data Center on the fly.

You can create new Tenants with dedicated resources, or deploy the infrastructure for a new application in such a way that it is isolated (in terms of security, performances and stability) from others, though running on a shared infrastructure.

It would take just the time of a REST call, where you push the new policy to the controller.

And of course you could use the same templates in the different environments: development, integration test, QA test, production, Disaster Recovery

The previous generation of network devices (e.g. the Nexus family) can be managed in a DevOps scenario as well.

They offer API and have puppet agents onboard. And a version of the APIC controller has been created also for networks outside ACI (APIC-EM - https://developer.cisco.com/site/apic-em/discover/overview/).

The Cisco DevNet community prodives a lot of information and samples at https://developer.cisco.com/site/devnet/home/index.gsp

I wrote a short post on Ansible here: http://lucarelandini.blogspot.com/2015/05/a-powerful-devops-tool-ansible.html where a great recorded session from the Openstack Summit is linked.