Figuring Out The Messy Middle (DevOps in the modern age)

Here’s the problem often described by clients:

we’ve figured out the lower end (the application development lifecycle), and the higher-end (delivering applications to our internal or external customers) – the issue we NOW  face is the manual work we have to do to stitch it all together.

Starting from the low-end, R&D teams are developing applications in silos, each team is using tools best suited for the job. Some, for example, would choose Google Cloud for Google analytics features; and some would choose AWS Lambda, etc. In addition, each team is leveraging a different automation framework like Terraform, Redhat Ansible, Kubernetes, Azure Arm …and the list goes on. The benefits from this method are obvious- R&D teams can leverage any tool to perfect the end result, but at the same time, the more apps businesses have, the more environments become an operational headache. Here’s why:

Let’s take a look at the higher-end to better understand the challenge. At this level we’ll have our customers – (either internal departments, or external users who consume the services developed). The customer requirement is a simple one: to have the service up and running in a click of a button. The behind-the-scenes implications of this is that the operations team (AKA the infrastructure team or the cloud automation team) will have to deal with the complexity of integrating between the customers’ portal and each and every application, including its tools, infrastructure, scripts, and CI/CD pipelines. The image below depicts this complexity. 

To deal with the manual stitching done on CI/CD, there’s a great need for sophisticated automation. In a recent blog post, Gartner defined such tools as follows: 

Infrastructure automation tools provide DevOps teams with on-demand, self-service access to standardized environments.

Notice how it highlights self-service access to standardized environments. This, in my opinion, is today’s messy middle. CI/CD pipelines should not be manually stitched. Infrastructure, networking, applications, and even functions are service components that should be automated from within CI/CD tools (like Jenkins or spinnaker). The image below depicts this concept:

Cloudify’s open source technology is addressing these problems uniquely from within the CI/CD in a way that masks the complexity of integrating various tools into pipelines, enabling DevOps teams to simply choose an environment from a dropdown list:

For a deeper dive into this approach, refer to this blog post.

In summary, to deal with the complexity of a modern Dev / DevOps environment, there’s a growing need to connect the dots between various private and public clouds, as well as automation tools. The more complex the environment, the more manual jobs will need to be placed in this “messy middle” – i.e, the stitching of pipelines into CI/CD. The modern approach to this issue must include a tool that will integrate the various tools across the stack, into a unified framework without a long migration process or a complete re-write of processes. 

Ariel Dan is the CEO of Cloudify