DevOps primer: The past, present and future

Introduction

After more than a decade in the enterprise, the DevOps trend, which blends software development and IT operations, has undergone dramatic changes as it evolved from a grassroots, developer-driven movement to a top priority of management and leadership. DevOps has moved beyond the benefits of faster releases and more efficient IT management toward the goals of organizational agility, resiliency and business outcomes, as well as integrating security, artificial intelligence and machine learning into software releases. As DevOps continues to evolve and expand in the industry and within organizations, we expect it will increasingly cross over with other disciplines and stakeholders, creating the opportunity for more synergy but also the challenge of continued silos.

DevOps has gone mainstream, now adopted at the team level or across all of IT for most enterprise organizations that deploy software to production. The trend, which is based on collaboration between developers and IT operations teams, has come a long way since it emerged more than a decade ago. DevOps was a mostly grassroots, developer-driven initiative that was left primarily to the practitioners and champions. Today, it is driven primarily by top-down adoption, with management and leadership leveraging it for digital transformation. Another difference is the DevOps mentality. In 2011, the mantra was: "If you aren't embarrassed about your code, you waited too long to release it" — the idea being that code should be shipped first and refined later. A decade later, DevOps is about managing IT efficiently and moving rapidly, but doing so in a way that is sanctioned and secure, aka "DevSecOps." Another dramatic shift in DevOps is the end user, who was generally less important than elegant code and scripts 10 years ago but today is a priority for teams deploying apps and services. The introduction of more data analytics, artificial intelligence and machine learning will likely contribute to more data- and business-driven DevOps guided by faster releases and optimized IT management, as well as organizational agility and customer satisfaction.

DevOps of the past

DevOps emerged from enterprise software development and IT operations teams that were eager to take advantage of agile development, cloud computing and automation but did not have an end-to-end perspective on applications and operations, since these were typically silos in the organization. Teams began cross-discipline DevOps, whereby developers included IT in infrastructure and tooling choices and IT departments became more responsive to developers to avoid getting bypassed, resulting in shadow-IT deployments. While it was limited to champions and cutting-edge companies at its start, DevOps has now become mainstream.

While 59% of organizations in a recent 451 Research Voice of the Enterprise: DevOps, Workloads and Key Projects survey say that half or more of their applications are released in a DevOps manner today, an impressive 82% expect half or more of their applications will be released via DevOps in two years. Additionally, of organizations that have released applications to production in the last year, 95% are implementing DevOps, either fully across their IT organization (51%) or across some teams (44%).

At its beginning, DevOps was mostly about going faster and more efficiently managing IT infrastructure, even at scale. Over years of evolution in the enterprise, DevOps has come to mean much more to organizations seeking to digitally transform both their internal workflows and end-user experiences. While DevOps is still about faster software deployments, which have become a baseline of modern software teams, it has come to mean much more in terms of enterprise digital transformation. When we consider the top advantages of DevOps, companies cite flexibility to respond to changes nearly equally to faster software releases.

There are benefits to moving faster, but organizations realize the value in being able to deploy software in a way that is sanctioned, secure and compliant — indicative of the maturity DevOps has achieved for many companies. Enterprises have also, over time, leveraged DevOps for not only faster software releases and more efficient IT management, but also for organizational flexibility and resiliency that helps teams effectively deal with changes in the market, whether it is new technology (such as cloud native), security issues or a global pandemic. In this sense, DevOps embodies a next generation of agility that applies to business operations and objectives, as well as technical ones.

DevOps has also marked an increasingly faster pace of enterprise software releases over the years. There were far more quarterly and monthly software releases 10 years ago, but today it is just as common to see weekly, daily, hourly and faster software releases for enterprises across a wide range of verticals. The difference with today's DevOps releases is that those faster software sprints must be sanctioned by the organization, secure and compliant. While these objectives beyond speed had been inhibitors of DevOps in the past, today they are incorporated into an approach that balances developer flexibility with policy guardrails to ensure releases are secure and risk is reduced.

We have also seen changes in the types of organizations undertaking a DevOps strategy. Companies implementing DevOps a decade ago tended to be cutting-edge players in financial services, insurance, telecom or media. Today, we see DevOps broadly deployed across many industries including healthcare, manufacturing and government/education. In addition to broader deployment across industries, our research indicates that the broader the DevOps deployment, the more likely objectives extend beyond speed and efficiency to include security and business objectives.

DevOps of the present

DevOps has always been about effectively leveraging cloud computing resources, which were starting to gain adoption in the enterprise at the start of the trend. Today, cloud continues to play a prominent role in DevOps. Public clouds have become increasingly important for enterprise DevOps teams, ranking second in our surveys only to infrastructure automation, which has been in the market and part of DevOps for a longer period. This is driven by organizations seeking cloud benefits, including cost, performance and agility, by putting more application development and deployment in public clouds. Public cloud providers are also more formally and aggressively catering to enterprise DevOps teams.

Today's DevOps has also extended beyond just the developers and IT operations teams to include a number of other stakeholders. Our research indicates that DevOps teams are now including and interacting with traditional administrators, security teams, management and leadership, and data science teams. This is indicative of the stakeholder spread that should happen in organizations as they apply DevOps technology, practices and teams to more of their application portfolios.

The fact that management and leadership are among the top additional stakeholders in DevOps highlights how it has transformed from a skunk works type of deployment to a top-down-driven priority, particularly when company leadership views it as a key step or building block to digital transformation. We have observed a sped-up digital transformation timeline as a result of the COVID-19 pandemic, and our research shows far more acceleration and initiation of DevOps deployments than slowdowns or stoppages amid the ongoing pandemic.

In short, DevOps is no longer viewed as strictly the realm of software developers and IT operations professionals. We believe this stakeholder spread, which means DevOps teams are now collaborating with security, data science, line-of-business and product teams, is a healthy sign that DevOps is working and helping to improve both internal processes and customer experiences.

The present state of DevOps also includes the ongoing COVID-19 pandemic, during which our survey research indicates there has been far more acceleration (34%), continuation (21%) and initiation (18%) of DevOps deployments than delays (24%) and cancellations (2%). Due to its close connection to digital transformation and business outcomes, DevOps is still generally growing inside organizations and throughout the industry amid COVID-19.

DevOps yet to come

The DevOps of the future will be increasingly secure, as well as driven by data analytics, AI and machine learning to help promote continuous improvement and realization of business objectives. In terms of security, we have charted the continued growth of security integration over the last three years. The number of organizations in 451 Voice of the Enterprise surveys reporting that half or more of their DevOps workflows include security elements has grown from 52% in 2019 to 64% in 2020 and to 69% in 2021.

Organizations are including vulnerability scanning, different types of application security testing, API firewalls and other security technology into their releases. However, challenges remain as nearly one-third of organizations report that security and DevOps teams work on DevSecOps requirements independently, highlighting how silos can persist. Companies will have to enable both DevOps teams and security teams to address challenges together and drive success, just as developers and IT operators had to collaborate at the beginning of DevOps.

The focus must also be about more than breaking or stopping software builds, as this simply reinforces the idea that security will slow teams down. Instead, by leveraging integration, automation and abstraction, a productive DevSecOps approach can provide policy and guardrails to ensure secure releases and reduce risk without holding everything up.

Another big part of DevOps going forward will be the inclusion of data analytics, correlation, AI and machine learning to improve software development and deployment, as well as business outcomes. Analytics, AI and machine learning are also among the most important tools for DevOps, and we see growing adoption of AIOps initiatives that leverage analysis of operational data such as logs or traces, with 30% of surveyed enterprises using tools today, while 26% are in discovery or proof of concept, and another 32% are planning deployment in the next two years. We believe this is indicative of more analytics- and business-driven DevOps to come.

Organizations are also increasingly interested in measuring DevOps success not only by technical metrics, such as quality or application performance but also by business metrics such as customer satisfaction. DevOps has already come to mean much more than speed and efficiency, and we believe it will continue to evolve with a market that is increasingly in step with users of applications and services and their needs.