As the pandemic drives enterprise digital transformation, an abundance of funding is being thrown at startups with a plan to support the process. It’s ushering in a new wave of DevOps automation, and in this post I’ll share a few predictions that might shape the market in the year ahead. Let’s start with the current needs and challenges facing enterprises along the road toward transformation:

1. Scale is (still) King: SaaS vendors need to scale fast in response to demand gyrations.
2. Agility Fragility: Increasing complexity of the DevOps environment is resulting in less agility at a time when more is needed.
3. Tyranny of the xOps: The rise of DevOps automation concepts like DevSecOps, MLOps, FInOps, GitOps has complicated the shift to highly distributed architectures of microservices, serverless, machine learning, and multi-cloud. 
4. Death by glue code: DevOps teams are constantly dealing with integration tasks and writing custom glue code to close the gaps, and this doesn’t scale. Teams are spending less time automating their development environments and actually writing code to improve the product – skilled DevOps engineers are so hard to find, their time needs to be optimized.

What’s needed is a new class of DevOps solutions to address these four challenges and manage the highly distributed environments that make transformation possible. They must have greater out-of-the-box integrations aimed at improving velocity, boosting productivity, and simplifying IaaC complexity. 

Here are my predictions on what’s happening to address this in 2022.

HCL Wins. Sorry. (Not Sorry)

Infrastructure automation is a specific domain, and it requires a domain-specific language. I believe that HCL, a domain-specific language for Terraform designed by HashiCorp, is here to stay as a part of an IaaC solution for developer productivity.

HCL can be frustrating for developers to use because of its limitations in expressing more complex business logic, so in 2022 we will see a growing set of frameworks (Cloudify included) that augment Terraform rather than attempting to replace it with a new IaaS solution written in native languages that were not designed to handle infrastructure management domains. There will always be multiple orchestration languages in managing IaaC environments that will drive demand for better interoperability and integration among orchestration domains. I expect we’ll see new DevOps frameworks geared specifically to saving all the custom glue code in today’s IaaC environments.

Non-Developers Automating Infrastructure

In 2022 we’ll see new low/no code tools that combine the no code and IaaC approaches to simplify DevOps automation, allowing users to move from one model to the other interchangeably during development cycles. This will allow non-developers to take part in infrastructure automation. Canned integration and graphical interfaces, an approach we first saw used in workflow automation, is an approach Cloudify and other vendors will be using. This will also lead to a closer integration of the IaaC tools into IT and business workflow management. 

Democratization of the Development Environment From Production

In 2022 we’re going to see a democratization of the development environment from production which will result in faster release cycles. Production tools are frequently used to run development teams because it’s a complex and costly task to maintain multiple views of the same environment. But this has an adverse effect on agility. The longer it takes to create a development and testing environment, the fewer tests can be run on a given day, and the longer it takes to release a new feature. Software quality is also at risk when using a production tool in development, as development environments need to be optimized for quick troubleshooting without the heavy lift of SLAs and security that production environments require. In 2022, we’ll see tools emerge that make a single environment accessible in meaningful ways to development teams as well as production teams. 

ITSM Shifts Left

The IT and DevOps automation change management process has evolved in almost opposite directions. IT change management is driven through ITSM such as ServiceNow, and it tends to be more human-driven through ticketing and approval systems. DevOps change management, by contrast, was designed to allow full automation (continuous deployment) with minimal human intervention and fit into the current software development cycle. The most notable trend here is GitOps, which aims to streamline the release cycle process by integrating it with the software development cycles through Git.

In 2022, we’ll see ITSM take a major shift left towards better alignment with DevOps automation processes like GitOps. And as with the rest of the Shift Left movement, I believe that 2022 will bring new tools for closer integration between ITSM and DevOps automation to allow more streamlined change management processes between the two worlds. This will address IT needs for better visibility and governance automation processes without adding friction to the development and release cycles. This in turn will speed up delivery of certified environments. 

Cloud Wars… They’re Just Getting Warmed Up

Cloud wars 2.0 will make cost and efficiency a major focus as companies start to scale in 2022.  Up until now, success for companies has been measured by how fast they can grow and not by how much they can save. I expect that 2022 will see a shift left happening in this area also. As companies reach a certain scale, the cost of running their services on top of a public cloud infrastructure becomes a major contribution to cost, in excess of 50% of their overall revenue according to Martin Cassado’s report on the Trillion Dollar paradox. He referred to this occurrence as Cloud Wars 1.0. 

What we know now is that when the developer is exposed to the costs during the development cycle, they have the opportunity to take closer control of their infrastructure and minimize their dependency on managed cloud services before it gets to production. Some are also taking a more extreme measure toward complete repatriation of their hyper-scale workloads from the cloud infrastructure. Kubernetes is an important enabler in this task as it allows more granular optimization between the workload and the infrastructure by enabling portability between different infrastructure environments. I expect that in 2022 we will see more tools and frameworks that will help make the right infrastructure choice for the job simple and seamless.

Our current toolchain has been mostly centered around CI/CD, IaaC, and Ansible Kubernetes as the main layer of our stack.  Those tools were not built to deal with many of the new challenges simply because those challenges didn’t exist at the time they were written.

This is where I believe we’ll see the focus of the new wave of DevOps automation tools, and Environment as a Service (EaaS) fits right in. It is time to think beyond the current toolchain and start to move up the stack, literally speaking, in order to address modern challenges, such as:

• How can we speed up the time it takes to deliver an end-to-end certified environment to our development team?
• How can we ensure that developers can optimize those environments for cost while they’re developing those environments?
• How can we ensure that we can handle continuous deployment across many distributed services in a reliable fashion?
• How can we enable better reuse of pre-templatized environments both internally and externally?
• How can we streamline and govern our change management across our IT operation and development teams?

We are already starting to see a growing trend of users that are hitting the wall with the current approach and are looking for a new set of tools to address these challenges. 

Final notes: The Speed Paradox

Hyperscale companies such as Spotify are also recognizing their developers’ productivity challenges from the existing tool sets. 

Spotify chose to develop a set of developer productivity tools and launched a new project (backstage) to provide a self-service experience and simplify the way developers access infrastructure services.

I believe that enterprises and software companies that are not nearly the size of Spotify will look for COTS-based offerings to solve this problem, as they don’t have the luxury of building such a tool themselves, and tools like backstage still require them to operate the frameworks. Cloudify is definitely planning to be there to help push in this new wave of DevOps automation.

Nati Shalom, CTO, Cloudify

A version of this article originally ran in VMblog.