From the Blogosphere
The State of #DevOps Today | @DevOpsSummit #APM #SDN #AI #DevSecOps
DevOps is an ever-changing, sometimes complicated term
By: AppDynamics Blog
Mar. 13, 2017 12:30 PM
The State of DevOps at the Start of 2017
The word—or rather, the contraction—is simple: DevOps. But DevOps, which refers to the increased communication and collaboration between development and IT operations, is an ever-changing, sometimes complicated term. While “dev” and “ops” used to be siloed into separate philosophies, practices, tools, and work flows, they’re merging into one. The result? A more efficient, reliable process and product.
Though the basics aren’t exactly new, the DevOps world is still trying to figure out how it will evolve and integrate into already established infrastructures. Whether you head a company that relies heavily on DevOps or you’re just now dipping your toes into the DevOps pond, there’s plenty to learn—because this movement and culture are anything but set in stone. To get a better feel for what’s in store for DevOps in 2017, we reached out to the pros. Read on for challenges and changes you may encounter in the coming year.
Justin Vaughan-Brown: Director of Product Marketing at AppDynamics
Kevin Gillis: Software Product Management Leader (former DevOps leader at SmartBear)
Grant Smith: DevOps consultant at Just Eat and author of The Next Gen DevOps Transformation Framework
Stephen Thair: Co-Founder and CTO of DevOpsGuys
How has DevOps changed the IT landscape in the last few years?
DevOps has completely changed how the technology industry tackles engineering problems. DevOps turned the conversation away from a narrow focus on software and SDLCs to services and engineering. 2009 was a landmark year; more than a decade after we began running online services we finally started considering how we build, launch, manage, and support online services as a whole rather than just how we build the software. —Grant Smith
Dramatically. Departments, budgets, roles responsibilities, and above all—expectations. Annual product releases gave way to quarterly, then monthly, weekly, daily, and now several times a day. Now there are micro code check-ins, micro builds, micro testing, micro releases, and micro updates. Customers, the market, and executive management want everything now.
The lines have been completely blurred and that creates uncertainty and confusion as responsibilities go from Ops to Developers. To enable this shift in responsibilities and dramatic speed to release, it means automation of all stages in the development, testing, and production process, and that dev teams lead the charge and decision-making on tools and technology. My front row seat in particular showed how QA departments are driving toward more and more automation—some 100 percent—which means no manual/UX testing. This creates new risks around quality and user experience and has led to some serious philosophical debates in the industry. —Kevin Gillis
I think DevOps has fundamentally changed the IT landscape, and that change was long overdue. When you look at the stats on project delivery, you can see that most large enterprise IT departments are struggling to deliver. At best, 25 percent of the IT spend is wasted on failed projects and rework (some estimates have that as high as 50 percent or more). When you add the huge changes required across the organization to deal with digital disruption and digital transformation, it’s obvious something has to change. We need a new operating model for IT, and that new model is DevOps. Organizations are seeking to restructure their IT delivery models away from the short-term, cost-focused, project-based models to full lifecycle, innovation-focused, product-centric models based around cross-functional DevOps-enabled teams, using the newest cloud-hosted automated tools and technologies. This change isn’t easy. It is a cultural shift from command and control management to true leadership, surrendering centralized control to autonomous teams, as well as learning a raft of new technical skills in order to stay relevant in this new world. —Stephen Thair
What are some challenges you think the DevOps world will face in 2017?
This past year was dominated by data breaches, hacking stories, and a general sense of disquiet at how vulnerable our personal information is to a multitude of threats. Going into 2017, security needs to be an in-built consideration in any new product release, not an afterthought. There is huge partnership potential between DevOps and Security, which is largely unrealized. —Justin Vaughan-Brown
Availability of good engineers is the single biggest challenge facing the technology industry today—and it shows no sign of improving. Many businesses are investing in bootstrapping graduates where five years ago they would have demanded three or five years experience. Some universities have started offering DevOps courses or modules, but many computer science graduates still leave university having never configured the environments that run their code. —Grant Smith
There’s a notion, particularly with the ease of updating SaaS and Mobile-first applications, that speed wins—but at what expense? Quality, customer experience, brand erosion? Companies are struggling with this. At one conference, I heard Uber’s Denali Lumma talk about having thousands of code repos and getting build cycles in an intense CI/CD environment means everyone—even developers—are on call. Defects in production may go straight to the developer at 3AM! Still, fox-in-the-hen-house debates persist about developers picking up more and more of the testing. Companies like Facebook and Expedia are moving away from formal QA and more toward testing by developers and tools like Jenkins. I suspect that outside of really talented engineers, commercial-grade testing is beyond most developers. Automated testing is good regardless of dev methodology, but with DevOps, it’s a requirement and not an elective. —Kevin Gillis
I think it’s inevitable that in 2017 we’ll start to see a DevOps backlash. As we move through the hype cycle from the peak of inflated expectations into the trough of disillusionment—as happens with any new technology or methodology—there will be articles saying “DevOps is Dead,” and that’s fine. What doesn’t change is that the old, siloed, hierarchical models are still broken. We need new ways to deliver IT products and services, particularly as cloud SaaS and PaaS services evolve, which in turn means software architecture evolves (e.g. the current hype over server-less computing). The other biggest challenge the DevOps world will face is a massive skills shortage. Currently there is somewhat of a zero-sum game going on with experienced DevOps people in high demand, which means people jump from company to company. This isn’t sustainable in the long run, which is why we (DevOpsGuys) are investing heavily in 2017 to expand our education arm and set up our own DevOps Academy to train people in the theory and cultural aspects of DevOps as well as practical hands-on tools and technology. —Stephen Thair
How will DevOps advance in 2017?
There are also a ton of events for DevOps practitioners to gather and share: MeetUps, summits, conferences, virtual shows, and more. Learning from people who have initiated change and had a measurable impact reassures people at the start of the DevOps journey.
And finally, DevOps is having a direct impact on businesses in terms of customer satisfaction, retention, and acquisition. This gives Development and Operations heads the mandate to engage business colleagues and understand where they can focus efforts next, and be truly aligned. AppSphere saw the launch of Business iQ, which shows the correlation between the customer online experience and the bottom line. This is just one example of how IT teams can share actionable insights with business unit colleagues and foster deeper collaboration. —Justin Vaughan-Brown
Server-less is going to be part of every progressive organization operating model. The benefits are too huge to ignore given how easy it is to implement. —Grant Smith
The entire Software Development Lifecycle (SDLC) is shrinking. And this requires more controls to measure and track effectiveness and productivity. I love the idea of Facebook using a scoring system where developers lose points for bad releases/defects that make it to production. Test automation and test parallelization will receive more attention. Until code automation/AI is more prevalent, the bottleneck is testing and deployment—and that can be addressed with automation (e.g. testing, code check-ins, builds, integration), APIs, and deployment. —Kevin Gillis
DevOps will advance as we expand the practitioner knowledgebase. As more and more organizations adopt DevOps, we’ll see more case studies and real-world examples of how to effectively manage the DevOps transition—especially for large enterprise organizations. There is a difference between organic and transformational DevOps, and the challenges a globally-distributed enterprise with thousands of staff faces are completely different than those faced by a small start-up implementing DevOps as they grow. That said, the goal is to get all organizations to communicate more effectively, better manage their products, and cope with the rates of change we’ll see over the next five years. And for good or bad, we will also see more DevOps certification and standardization efforts in 2017. We’ve already seen at least three organizations claim they’re producing a “Certified DevOps” curriculum. Even Axelos, the commercial owners of the ITIL standard, are getting in on the act. —Stephen Thair
What are some DevOps practices you think will become mainstream in 2017?
“Shadow IT” and “citizen developers” are just two terms being shared to describe the growing impact the business will have on tech spend decisions, and IT teams need to have a plan for this. Equally, each employee’s field of vision is becoming extended, as their touch points with the wider organization increase. We will see a parallel loss of direct control and increase in influence further afield as the extended IT ecosystem grows. The days of empire building are fast receding.
And expect to see a new role emerge in the coming years called the Site Reliability Engineer (SRE). A term initially created by Benjamin Treynor Sloss, who leads global ops, networking, and engineering within the Google team, the SRE is sometimes a developer who now sit in an Ops role and is focused on availability, stability, latency, and performance. —Justin Vaughan-Brown
Anomaly detection. Innovations in logging and monitoring capabilities have now got us to the point where we can apply machine learning algorithms to metrics directly. You still have to identify your primary constraint metrics, but once that’s done you can have the computers stare at the graphs for you. —Grant Smith
Many larger, more mature companies are still transforming their product delivery teams to Agile and Scrum, and DevOps is still just a glimmer. For those currently running DevOps, expect CI/CD and daily micro releases to become more and more mainstream. Waterfall was measured in quarters, agile in weeks. But DevOps is now measured in days—even hours. —Kevin Gillis
The low hanging fruit of DevOps automation practices (continuous integration, test automation, automated provisioning, configuration management, application release automation, etc.) will be increasingly adopted. They can buy you some time and breathing space and help you pay back enough technical debt so you can focus on the wider digital and DevOps transformation agenda. Measurement-driven approaches to DevOps will be huge in 2017. We are already seeing organizations collecting and analyzing huge amounts of data about their customers, applications, and infrastructure. We’re also seeing more organizations use this information to guide decision making and product strategies, and where they invest their time and effort. Both open source and vendor solutions, such as AppDynamics, will play a role here. —Stephen Thair
Do you think DevOps will become more standardized?
DevOps is actually highly standardized. By my reckoning, DevOps is in its third generation now. For those early adopters, standardization is based around their own internal code bases. Facebook, Netflix, Google, and Etsy have rich, publicly available code repositories, which show how they’ve developed. The second generation of DevOps organizations all standardized around AWS and CI tools like Jenkins and Teamcity, and have invested heavily in ELK and visualization tools. The third generation of DevOps organizations have standardized around containers, microservices, and server-less functions. And they’ve fully embraced SAAS offerings for non-strategic requirements like log and metric analysis. —Grant Smith
Yes, but it will take one or two key, industry-wide initiatives to drive this. For instance, Sonatype helps companies with the quality, security, and speed of their software supply chains, and SmartBear’s SwaggerHub helps standardize the building, documentation, and deploying of APIs. —Kevin Gillis
The IEEE are already working on a DevOps Standard 2675 to “… specify practices for groups including development, operations, and other key stakeholders to collaborate and communicate effectively to build, package, and deploy software and systems in a secure and reliable way.” Note that the cultural aspects of DevOps, which are harder to measure and define, are out of scope for 2675. ISO are also working closely with IEEE on the DevOps standard. —Stephen Thair
Why should organizations who haven’t yet adopted DevOps do so in 2017?
The reason for adopting DevOps working practices are the same now as they were when the term was first coined. It leads to better solutions that offer customers better products and delivers innovation faster than the alternative. If your organization doesn’t need to improve how it delivers online services, then you can continue as you have been. —Grant Smith
Because probably most of their competitors have! More seriously, I see this shift being very similar to what I saw with Agile/Scrum. About 10 years ago, my entire product management and R&D team was trained on Scrum by Mountain Goat Software. It was a company-wide initiative and there was great resistance from the one high-performing team that was shipping on time with the right quality and scope.
I asked Ken Schwaber (Co-Founder of Scrum) to speak to my team and he told me “don’t switch to Scrum, because what you have is working!” It’s very similar for DevOps—if your customers, competitors, and the market are not demanding it, don’t switch. But if you need a more responsive methodology and quicker time to market, your organization should start with small, focused, and non-critical projects in 2017 before making company-wide institutional changes. —Kevin Gillis
If you don’t significantly improve your ability to deliver digital products in the next two to three years, your organization will go out of business. The IDC predicts that by 2018, 66 percent of global 2000 enterprises will have digital transformation at the center of their corporate strategy. Doing things the old way just isn’t good enough anymore. —Stephen Thair
Do you have any additional thoughts on the state of DevOps in the coming year?
We in DevOps need to get better at delivering the basics quickly. The exciting and challenging problems aren’t building infrastructure, deploying containers, or exposing metrics and logs through query engines. Data is the next great frontier. We can manage software with software, infrastructure with software, and networks with software—and yet when it comes to managing data we’re still scrambling around for flint to make fire. We either need to find very sophisticated solutions to version, deploy, roll back, and delete records in large data structures, or we need to get much smarter about how we store and manipulate customer data. (My bet is figure out the latter before we get the former.) —Grant Smith
Watch for the “Ops” movement to become contagion: DevOpos, DevSecOps, BizOps, and for you beer lovers, even HopOps! As with the shift from waterfall to modified waterfall to various flavors of Agile, expect to see many DevOps variants. As your company assesses, don’t be too prescriptive or pedantic, but instead match up with what you need to accomplish your company’s business goals and then figure out where DevOps fits in your organization. —Kevin Gillis
Tech Companies already lead the Fortune 500 by market capitalization, and the speed at which unicorns like Uber, AirBnB, WhatsApp, Nest, and Beats reached multi-billion dollar valuations is phenomenal. A new entrant can disrupt established companies just as quickly, so you need strong digital capabilities to be able to react to changes in your market. DevOps is the new operating model for IT that sits at the heart of your digital capabilities. —Stephen Thair
More About Our Experts
The post The State of DevOps at the Start of 2017 appeared first on Application Performance Monitoring Blog | AppDynamics.
Latest AJAXWorld RIA Stories
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
SYS-CON Featured Whitepapers
Most Read This Week