From the Blogosphere
Taming the API Sprawl | @DevOpsSummit #API #DevOps #Microservices
One of the much-touted advantages of web services is reusability
By: SmartBear Blog
Oct. 20, 2015 03:00 PM
Taming the API Sprawl
Ten years ago, there may have been only a single application that talked directly to the database and spit out HTML; customer service, sales - most of the organizations I work with have been moving toward a design philosophy more like unix, where each application consists of a series of small tools stitched together. In web example above, that likely means a login service combines with webpages that call other services - like enter and update record. That allows the customer service team to write their own tools using the web, the command line, scheduled, or any other interface.
Sound too good to be true, doesn't it? It is true, but it comes at a cost. For example, I never defined a mechanism to manage the explosion of APIs that will result under this approach. Consider this: uncontrolled growth is one definition of cancer.
Since the rapid creation of APIs is not quite so deadly, I will call this the API Sprawl; I've seen it across every client that moved to web-services approach, typically two to four years into conversion.
Today I'll tell you how to control it, avoid it, or, if you have to, tame the API Sprawl.
Starting from a Mess
All wonderful so far.
Sadly, in order to use this function, we need to find it. Without controls, the web services for an organization start to feel a bit like a junk drawer. First we have to go looking for what we seek, either talking to teams or searching through intranet pages. Then, eventually, we find three tools that almost do what we want (we need to know pharmacy coverage, or if the user has hit their deductible, or some other detail). The existing services don't really work, so we write a fourth service, must_pay(date, memberid, coverage_type).
After going through this trouble only a few times, we've created three new services. The whole notion of reusing these services at all becomes suspect.
This cycle helps us more efficiently destroy the spirit of the next well-meaning person who optimistically thinks someone must already have built this....
Taming this API sprawl certainly means putting services where programmers can find them, but I want to be careful. Shiny exiting promises got us into this mess; a shiny new service repository and a mass email won't be enough to get us out.
Think about it this way: The world's most meticulously-curated library of terribly-written, unedited novels is a joke; it would provide almost no practical value to its community. You can't think "If we build it, they will come." Yes, you have to build it, but if they don't trust it, they will stay away.
A Service Directory
That means that the number one issue with the yellow pages was getting people to find what they need, by subject or name. A company service directory should do the same - make services discoverable, and make the registration and change process not overly painful. You can probably find a team member whose eyes light up at this kind of challenge - the idea of hunting down the best tool for the job.
You might also invest time, energy, and money in all this, look up two years from now, and find that not much has changed. Without improving trust among the people who build and use the APIs, you'll find yourself circling back, looking for ways to deal with the situation, and you might even need to read this article again.
Beyond setting up a services repository and publishing it to your group, getting your APIs under control involves improving discipline, a fair amount of thankless work, and considerable patience. It means weeding out duplication among your services, making those APIs self-documenting through better and more consistent naming. There is a need to document and in some cases reverse-engineer the contract of each and every service. This has to happen, not because of some abstract sense of "right" and "wrong", but in order to address the key reason that the APIs keep sprawling under your feet: people just don't trust the services enough to risk their projects on them.
What Builds Trust?
The services themselves need to work, because if they don't, then I can't depend on them. Sadly, "it works" ranks as one of the most ambiguous sentences in English, so let me clarify what I mean: the services need to do what they claim to do. This almost certainly means executable examples - what some might call integration tests. I define "it works" to mean "passes these tests". Put the other way, I define "what the services does" in terms of the tests it passes.
Now Add Tests
So write tests. Not just when creating a service, not just when documenting it for the repository - even when teams use a service they can write tests for it, leaving behind a record of what they've learned, their use cases for the service and their expected results. If you run out of time to write tests, then at least document your assumptions and expectations about what a service does. Provide examples, even if you can't run those examples right away. (By writing examples, even as bullet points in documentation, we have an option to turn them into running tests when we have time and energy.) Make those tests easy to find and easy to run. Make those examples easy to find and easy to read. Do this over and over, even when it seems like nobody cares. This illustrates one of the key points of trust: you have to give a lot of it away before you start getting it back from others.
As the suite of tests for a given service grows, so does our confidence in the service, and we can move it some place where others can find it and start using it. In this way, we use a kind of internal open-source model of incubator projects that feed the prestige, published Service Repository. In the incubator projects we find the services that "aren't ready for prime time". These are the ones we're slowly-but-surely testing and documenting. Maybe we're refactoring them to improve the public interface. Maybe we're incrementally rewriting some of them (don't go overboard on this), intending to publish the new versions and phase out the old ones. Over time, we move services from these incubators into The Services Repository. This builds even more trust, because it clearly delineates which services we believe are ready and which ones need more work. It acts as a kind of service level agreement for our APIs: you should expect these ones to work and to be able to understand them, but those ones there-that's still beta territory. Maybe alpha. Be careful. We're working on it.
This is very similar to what we do in programming, when we write some code once, they turn it into a method for internal use, then only eventually, and with some hesitancy, do we add it to a library for some other programmer to use.
With the strategy above, by the time a service is added to the repository, it is discoverable, documented, and has examples.
This is sort of the opposite of our Sprawl Scenario - we are avoiding the temptation so publish a service until we are confident that it is ready for prime-time. As you build trust with the programmers, they'll slowly accept your mistakes more graciously, but it takes time. The more meticulously you go about this, the sooner you'll build the critical mass of trust you need for the system to sustain itself. And it all starts by just writing tests.
Get Your APIs Flowing
Make the next book you read about trust, and you'll find it easier to get this sprawl under control. Trust me.
Where to go for more
Refactoring Databases. It seems strange, but APIs act a lot like database schemas, at least in the sense of being difficult to change. Reading this book will give you some ideas how to deal with changing published APIs.
The Five Dysfunctions of a Team. The next level of depth in understanding the role that trust has in any team situation, including but definitely not limited to API sprawl.
The Speed of Trust. When you're ready to dive deeply into trust as an ingredient in organizational success, you can start here.
Latest AJAXWorld RIA Stories
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
SYS-CON Featured Whitepapers
Most Read This Week