From the Blogosphere
Integrating APM into Testing | @DevOpsSummit @Stackify #DevOps #APM #AI
Does your team have a testing strategy?
By: Stackify Blog
Jul. 13, 2017 01:00 PM
Integrating APM into Your Testing Strategy
Does your team have a testing strategy? In 2017, I have a hard time imagining that wouldn't at least have some strategy, however rudimentary. Unlike a couple of decades ago, you hear less and less about people just changing code on the production server and hoping for the best.
At the very least, you probably have a QA group or at least someone who serves in that role before shipping your software. You write the code, do something to test it, and then ship it once the testers bless it (or at least notate "known issues").
From there, things probably run the gamut among those of you reading. Some of you probably do what I've described and little more. Some of you probably have multiple pre-production environments to which a continuous integration setup automatically deploys builds. Of course, it only deploys those builds assuming all automated unit, integration, and smoke tests pass and assuming that your static analysis doesn't flag any show stopper issues. Once deployed, a team of highly skilled testers performs exploratory testing. Or, maybe, you do something somewhere in between.
But, whatever you do, you can always do more. In fact, I encourage you always to look for new ways to test. And today I'd like to talk about an idea for just such a thing. Specifically, I think you can leverage application performance management (APM) software to help your testing efforts. I say this in spite of the fact that most shops have traditionally taken advantage of these tools only in production.
What You Need to Get Started
You're going to need a non-production environment in which to test your application. Generally speaking, this means a server (or multiple servers) in which you simulate your production environment. You don't need to simulate all production conditions, necessarily. But you can't reasonably do this on Steve's development box because he has the only desktop with 16 gigs of memory. You're going to need to up your game a little beyond that.
You'll also need for this environment to include reasonable facsimiles of externalities to your application. Most commonly, this means a database, but it also might involve things like (non-production) endpoints web service calls. You need to recreate production, both regarding the deployed environment and the things it depends on.
What You Really Ought to Have
The same reasoning applies to exercising the application. You could deploy it to the test environment and then do something like having a whole bunch of people log in and start banging around at things. But you'd do a lot better to automate this to simulate traffic. You can think of this as an automated smoke test. And doing it will put stress on the system in a way that manual testers probably can't and certainly won't. It will also ensure a consistent and methodical approach that you can repeat to troubleshoot.
With all of that squared away, you can start to take advantage of APM in your testing strategy.
First Steps and Getting a Feel for Things
Get an APM tool and set it up in your testing environment. Play with it and get a feel for its features and how it works. See what sorts of data it furnishes and how that changes under different circumstances.
You're just seeing what it will tell you about your application at this point. If you've never really profiled your code before, I can almost promise you that it will find things that astonish you. Sure, you knew the call to get all customer records was slow, but you just assumed it was a network issue or something. You had no idea that Customer data transfer object had a Thread.Sleep(1000) in its constructor. (Okay, you probably won't find yourself quite that astonished, but you never know.)
Your manual efforts alone will likely result in a robust set of technical debt backlog items to address. You'll have queries to improve, modules that spew swallowed exceptions to clean up, and a good bit of general housekeeping.
Move On to Automation
Did some kind of rogue query result in an order of magnitude delay? By all means, fix the query, but carry that learning forward. Set a threshold a bit above the proper time for the operation, and instrument your build pipeline to notify you if performance exceeds the threshold. Better yet, create some kind of failure that the team can't ignore.
Assuming you've taken my advice, you now have a pretty powerful end to end setup. Your commits trigger automated builds. These, in turn, trigger deployments to a testing environment and subject the application to all manner of testing - including smoke tests. These smoke tests stimulate the application into situations where APM will detect issues. When it does, you'll become aware of them with a significantly tighter feedback loop than a user calling the help desk to ask why things are so slow.
Keep the Pressure On
And that's just a single example. If you've set goals for other concerns or forms of research usage, keep poking at those for improvement as well. And keep dialing up the volume on the smoke tests if you can. The closer you can run to a stressed production environment, the better you can sleep at night when you actually ship.
Keep Things in Perspective
But take care that you don't allow that confidence to slide into overconfidence. Using APM can expose issues other forms of testing can't easily detect. It lets you simulate the vagaries of production and then take steps to guard against them. But note that I said simulate. It affects production, but it doesn't recreate it.
So as you incorporate APM into your testing strategy bear two things in mind. First, you also definitely need to monitor production itself. And secondly, no testing strategy, however sophisticated, can give you 100% confidence. Incorporate APM into your strategy, but never stop looking for other ways to improve it.
@DevOpsSummit at Cloud Expo taking place June 6-8, 2017, at Javits Center, New York City, and is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
DevOps at Cloud Expo / @ThingsExpo 2017 New York
DevOps at Cloud Expo / @ThingsExpo 2017 Silicon Valley
Download Show Prospectus ▸ Here
The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.
@DevOpsSummit will expand the DevOps community, enable a wide sharing of knowledge, and educate delegates and technology providers alike. Recent research has shown that DevOps dramatically reduces development time, the amount of enterprise IT professionals put out fires, and support time generally. Time spent on infrastructure development is significantly increased, and DevOps practitioners report more software releases and higher quality. Sponsors of @DevOpsSummit will benefit from unmatched branding, profile building and lead generation opportunities through:
For more information on sponsorship, exhibit, and keynote opportunities, contact Carmen Gonzalez by email at events (at) sys-con.com, or by phone 201 802-3021.
The World's Largest "Cloud Digital Transformation" Event
@CloudExpo / @ThingsExpo 2017 New York
@CloudExpo / @ThingsExpo 2017 Silicon Valley
Full Conference Registration Gold Pass and Exhibit Hall ▸ Here
Register For @CloudExpo ▸ Here via EventBrite
Register For @ThingsExpo ▸ Here via EventBrite
Register For @DevOpsSummit ▸ Here via EventBrite
Sponsors of Cloud Expo / @ThingsExpo will benefit from unmatched branding, profile building and lead generation opportunities through:
For more information on sponsorship, exhibit, and keynote opportunities, contact Carmen Gonzalez (@GonzalezCarmen) today by email at events (at) sys-con.com, or by phone 201 802-3021.
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Track 1. FinTech
Delegates to Cloud Expo / @ThingsExpo will be able to attend 8 simultaneous, information-packed education tracks.
There are over 120 breakout sessions in all, with Keynotes, General Sessions, and Power Panels adding to three days of incredibly rich presentations and content.
Join Cloud Expo / @ThingsExpo conference chair Roger Strukhoff (@IoT2040), June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA for three days of intense Enterprise Cloud and 'Digital Transformation' discussion and focus, including Big Data's indispensable role in IoT, Smart Grids and (IIoT) Industrial Internet of Things, Wearables and Consumer IoT, as well as (new) Digital Transformation in Vertical Markets.
Financial Technology - or FinTech - Is Now Part of the @CloudExpo Program!
Accordingly, attendees at the upcoming 20th Cloud Expo / @ThingsExpo June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA will find fresh new content in a new track called FinTech, which will incorporate machine learning, artificial intelligence, deep learning, and blockchain into one track.
Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expensive intermediate processes from their businesses.
FinTech brings efficiency as well as the ability to deliver new services and a much improved customer experience throughout the global financial services industry. FinTech is a natural fit with cloud computing, as new services are quickly developed, deployed, and scaled on public, private, and hybrid clouds.
More than US$20 billion in venture capital is being invested in FinTech this year. @CloudExpo is pleased to bring you the latest FinTech developments as an integral part of our program, starting at the 20th International Cloud Expo June 6-8, 2017 in New York City and October 31 - November 2, 2017 in Silicon Valley.
The upcoming 20th International @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA announces that its Call For Papers for speaking opportunities is open.
Submit your speaking proposal today! ▸ Here
Our Top 100 Sponsors and the Leading "Digital Transformation" Companies
(ISC)2, 24Notion (Bronze Sponsor), 910Telecom, Accelertite (Gold Sponsor), Addteq, Adobe (Bronze Sponsor), Aeroybyte, Alert Logic, Anexia, AppNeta, Avere Systems, BMC Software (Silver Sponsor), Bsquare Corporation (Silver Sponsor), BZ Media (Media Sponsor), Catchpoint Systems (Silver Sponsor), CDS Global Cloud, Cemware, Chetu Inc., China Unicom, Cloud Raxak, CloudBerry (Media Sponsor), Cloudbric, Coalfire Systems, CollabNet, Inc. (Silver Sponsor), Column Technologies, Commvault (Bronze Sponsor), Connect2.me, ContentMX (Bronze Sponsor), CrowdReviews (Media Sponsor) CyberTrend (Media Sponsor), DataCenterDynamics (Media Sponsor), Delaplex, DICE (Bronze Sponsor), EastBanc Technologies, eCube Systems, Embotics, Enzu Inc., Ericsson (Gold Sponsor), FalconStor, Formation Data Systems, Fusion, Hanu Software, HGST, Inc. (Bronze Sponsor), Hitrons Solutions, IBM BlueBox, IBM Bluemix, IBM Cloud (Platinum Sponsor), IBM Cloud Data Services/Cloudant (Platinum Sponsor), IBM DevOps (Platinum Sponsor), iDevices, Industrial Internet of Things Consortium (Association Sponsor), Impinger Technologies, Interface Masters, Intel (Keynote Sponsor), Interoute (Bronze Sponsor), IQP Corporation, Isomorphic Software, Japan IoT Consortium, Kintone Corporation (Bronze Sponsor), LeaseWeb USA, LinearHub, MangoApps, MathFreeOn, Men & Mice, MobiDev, New Relic, Inc. (Bronze Sponsor), New York Times, Niagara Networks, Numerex, NVIDIA Corporation (AI Session Sponsor), Object Management Group (Association Sponsor), On The Avenue Marketing, Oracle MySQL, Peak10, Inc., Penta Security, Plasma Corporation, Pulzze Systems, Pythian (Bronze Sponsor), Cosmos, RackN, ReadyTalk (Silver Sponsor), Roma Software, Roundee.io, Secure Channels Inc., SD Times (Media Sponsor), SoftLayer (Platinum Sponsor), SoftNet Solutions, Solinea Inc., SpeedyCloud, SSLGURU LLC, StarNet, Stratoscale, Streamliner, SuperAdmins, TechTarget (Media Sponsor), TelecomReseller (Media Sponsor), Tintri (Welcome Reception Sponsor), TMCnet (Media Sponsor), Transparent Cloud Computing Consortium, Veeam, Venafi, Violin Memory, VAI Software, Zerto
About SYS-CON Media & Events
Cloud Expo®, Big Data Expo® and @ThingsExpo® are registered trademarks of Cloud Expo, Inc., a SYS-CON Events company.
Latest AJAXWorld RIA Stories
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
SYS-CON Featured Whitepapers
Most Read This Week