Episode 95

Today’s episode is sponsored by the rain in Amsterdam and werewolves. It’s my second trip here within last two weeks with the initial goal of attending AWS trainings and some sightseeing, but AWS canceled one of these a day after my company bought non-refundable plane tickets and booked the hotel, so… more emphasis on sightseeing, or rather sitting in the coffee shop waiting for better weather. Why werewolves? Well, because in this installment of our long-running series on API design, we will talk about versioning and I find the werewolf a decent allegory to represent breaking (or tearing) changes in API that leads to a dilemma: which versioning strategy should be employed. And because I like fantasy themes for my articles.

Image by Aleksandr Nikonov

There are many approaches to versioning with pros and cons and it’s difficult to definitively choose one or the other. We will explore possibilities of how to version and,  where to actually put the version information, including URI, parameters, content negotiation, custom headers, or… nowhere at all. We are going to talk about breaking and non-breaking changes and various considerations and hints relevant to helping clients of your API deal with the evolution of our system in a bearable way.

Shapeshifting

Before we start to multiply API version, we should consider whether we really need it. Backward compatible, or non-breaking changes should not break existing clients. Such changes include:

• Introduction of new URLs
• Introduction of new optional query parameters and object fields
• Removal of access restriction on existing resources
• Introduction of new media types
• Introduction of new error responses or depreciation of existing ones

On the other hand, the non-backward compatible or breaking changes would be:

• Changes in URL path structure or removal of existing endpoints
• Introduction of new required query parameter or object fields
• Change in behavior in the absence of new fields
• Change in the interpretation of previously existing fields
• Change in value type of format in existing fields
• Adding access restrictions to existing resources
• Rejection of previously recognized query parameters or media types
• Redefinition of existing error responses and codes

This list might be a little tricky as what exactly happens and the definition of break depends on client implementation. Adding a new error code may or may not break the client depending on how it handles errors for example.

Versioning Schema

If we decide to version the API we should think of how to name the version. The most common approach seems to just use consecutive numbers with “v”, “ver” or “version” suffix. Some opt to go for semantic versioning in the form of major.minor.patch numbers. This is fine if we version a system behind, but for API itself we should rather stick to single major version. We care for breaking changes here, not for minor additions and depending on where we put the version, too fine-grained versioning will probably run us into trouble. This might be useful for development purposes in internal APIs, but in general, if we want to know a version of the system, we should have a separate endpoint for that. The third approach, used by Twilio is using timestamps instead of version numbers, which might be informative, but can be seen as an unnecessary noise. People will probably remember “v3” more than a particular date, especially if there are a lot such dates to remember. Let’s now move to where to actually put the version indicator. There are several strategies for that.

Image by Bob Kehl

The URI

By far the most common solution is to put a version number in URI, for example: api.exampe.com/books/v2. This is straightforward and easy to understand, let’s us version specific resource branches but has a number of disadvantages. First of all URIs in REST should represent the resource only and version of API is not an attribute of the resource. We may end up in the same resource being available under several URIs which may lead to weird bugs and problems with cache performance. We can’t easily use URI to compare identity. Depending on whether we use HATEOAS or not and whether our clients take advantage of it, they might be forced to change a client in many places. The somehow similar concept is to use a subdomain to represent version, for example: v2.api.example.com/books.

The URI Parameter

To better satisfy REST ideas and keep URIs clean we can move the version to a query parameter, like that: api.example.com/books?v=2. This is used by Amazon, Google, and Netflix but is not very popular in general. The new version will not break existing hyperlinks. We have an option how to handle a request without parameter – either use the newest version, the oldest or the one that was in use when the particular client registered. This can be done with the URI approach too but might be more difficult as the number of elements in path changes. We can take better advantage of caching as well. On the downsides – it’s less obvious which version is actually in use and it might be more difficult to handle the request on server-side as we need to parse the entire request to route the request correctly.

The Accept Header

Content negotiation can be used for versioning and there are actually three ways to do it. We can extend a vendor-specific format: Accept: application/vnd.example.v1+json. We can add a version indicator to the list of accepted formats: Accept: application/vnd.example+json; v=1. Or, less known style, we can drop the standard application prefix and use a fully custom value starting, by convention, with x: Accept: x.example.v1. This approach relieves us from some problems with URIs and URI parameters, like routing, caching, comparison difficulties and general URI mess. It also lets us maintain a fine-grained control over particular resources versioning. On the other hand, it feels a bit like a hack and is a bit hidden. Such things tend to be harder to work with. You can’t hand somebody a simple URI and fire it in a browser, you need to think about headers. One might also say that it complicates and pollutes content negotiation.

Image by quarridors

The Custom Header

Going further down the hidden path, we may opt for putting the version in a custom header by convention prefixed with “x”, for example: X-example-version: 1. It is used in Microsoft APIs for instance. This approach unclutters our content negotiation and has advantages similar to using Accept header for versioning, however, suffers even more from headers disadvantages – is more difficult to work with. With standard headers, our tools can help us a bit and, for example, find a typo or just suggest a header from the list. With the custom header, we have to do it ourselves. Also, some older proxies might drop non-standard headers when routing requests and such problems might be very difficult to debug.

No Explicit Versioning

A difficult choice so far. We can look at moving away from URI versioning from two perspectives. Either it is more and more hidden and unclear, or on contrary: more separated and clean. How you see it depends on your API design philosophy and various practical considerations that may vary from one environment to another. But if we were to move even further on the hidden/separated scale? There is no more hidden versioning than lack of thereof. Roy Fielding, the creator of REST advises against API versioning and calls it a polite way to kill deployed applications as you may read here. Alternative solutions are tricky to enumerate as they depend on particular cases, but HATOAS, described in the previous part of my API design series, might be helpful in dealing with changes. The problem is that not many clients really take advantage of it.

The Continuous Versioning

So ideally we don’t want to have explicit versioning, yet we need changes but our clients don’t use HATEOAS (which is also not the ideal solution itself, as you might read in the previous article in the series). Seems like a stalemate situation. What we can do however is to try to avoid breaking changes by introducing new things while still keeping old things and perform feature negotiation on requests. Say we had a Boolean field in the object and now it was extended with more options and it becomes an enum. Old clients may use the old field, new clients the new field. New features may be released on the spot, without bundling them in large versions.  We don’t need to spend that much effort on communicating changes. This approach might seem a bit chaotic, but is used successfully by Badoo and described in this article. It’s originally intended for internal APIs but might work to some extent in external ones too.

Image by Sephirothart

Lycanthropy is Hard

So once again we are approaching the end of an article describing possible solutions to handle certain API aspect, once again there are a lot of choices and once again it’s difficult to tell which one to follow. Dealing with changes is especially difficult as a number of our clients grow. The choice depends on many factors, in general, I’m leaning towards URI parameters or the Accept header as they seem to be somewhere in the middle of our hidden/separate scale. A tremendous amount of API stick to URI versioning, but I’ll leave the decision whether “popular” and “familiar” correlates with “good”. Feel free to reach me if you have any comments/insights regarding versioning strategy. And stay tuned for the next episode!