Hypermedia-driven RESTful web APIs

Recently, We started working on a project, which involved the development of a RESTful API, so I decided to first do a research to better understand exactly what it means to be REST compliant (RESTful).

Recently, I started working on a project, which involved the development of a RESTful API, so I decided to first do a research to better understand exactly what it means to be REST compliant (RESTful). It didn't take long to notice that many people out there were arguing about whether their APIs should be RESTful or not and I thought it had to be something either very important or very confusing, since a lot of people could not get to a definite agreement on whether a RESTful API really matters.

Typical RESTful meme

First I found a document, written by Dr. Roy Fielding, who introduced the REST to the world. He wrote a doctoral dissertation in which he explained in detail what REST is and why it is important, so I'll try to summarize it here.

So, what is REST? And what is it used for?

Dr. Roy Fielding

REST is an architectural style, used to create scalable, performant and maintainable distributed web services. In simple words, REST gives you a good platform to build a web service that will last for a long time. All these features (scalability, performance, user-perceived performance, maintainability,etc.) are something that Dr. Roy Fielding considered important for the evolution of the world-wide-web services. But that doesn't mean REST is a silver bullet or the one-and-only solution to any web service you need to develop.

It simply means that, for a given set of requirements (features, goals), Dr. Fielding has found and documented all the constraints that need to be satisfied in order to achieve these goals. Nothing more, nothing less. If your goals match with his, then you should also consider using REST to build your web services, but if your goals are different, REST will not help you much. It can even make things worse because each of those constraints introduces certain trade-offs, which might collide with some important features of your web service. Shortly, don't use REST for just any web service you want to build, because everybody is talking about REST. It's simply wrong.

You're doing it wrong

That being said, let's see what are all the goals, their constraints and trade-offs introduced in Dr. Fielding's REST architecture. The following table briefly summarizes that:

The important constraint here is the one named "Uniform interface" and the most important of those 4 interface constraints is the one named "hypermedia as the engine of application state" (aka HATEOAS). This constraint is the main reason behind those widespread disagreements between people. Since the majority of the web APIs, that exist these days, don't implement this constraint, they can't really be characterized as being RESTful. Maybe "RESTish" at most.

And why is that HATEOAS constraint so important?

The HATEOAS constraint is important because it provides an important feature called "independent evolvability". This means that the development of the client and server code is decoupled and that different teams can develop the client and the server separately. Take a typical web browser and a web server, for example. The Firefox/Chrome development team needs to know nothing about the Apache/IIS development in order to successfully improve their product. Their product only needs to conform to that "uniform interface" constraint which guarantees those products can continue to communicate successfully, even though they are developed by different teams.

Now, take a look at a typical web API client implemented these days. It is usually very tightly coupled with its server through the extensive knowledge of server's internals. In the client source code there is usually a list of all the server endpoints (resource URIs with RPC-style web calls), plus it contains the business-level logic (like workflows) and it usually generates the URIs for subsequent requests to the API server.

$('#poster').html("Loading...");
$.getJSON("http://api.example.org/2.1/foobar.search/en/json/31866/" + film + "?callback=?", function(json) {
  if (json != "Nothing found.") {
    $('#poster').html('We found you a poster: <img src=' + json[0].posters[0].image.url + ' />');
  } else {
    $.getJSON("http://api.example.org/2.1/foobar.search/en/json/31866/goonies?callback=?", function(json) {
      console.log(json);
      $('#poster').html('Nothing was found. Perhaps you were looking for The Goonies:
        <img src=' + json[0].posters[0].image.url + ' />');
    });
  }
});

An example of business logic and manual URI generation in a web API client source code

Of course, it also doesn't implement the HATEOAS constraint, which means it doesn't use hypermedia formats (like XML, XHTML, HTML, ATOM, etc.) in its client-server communication. Mostly a JSON (JavaScript object notation) media format is used, but it's not a hypermedia format at its core and it doesn't have a standard way to represent basic hypermedia elements like links, embedded resources and relations. Instead, JSON is a format created primarily for the transmission of data objects in a human-readable format and all those JSON extensions (like Hal or Siren) are just trying to adapt that "object notation" format to a hypermedia-like format. On the other hand, using native hypermedia formats like XML, ATOM, XHTML or just HTML already gives you a basic set of hypermedia features out of the box.

If those API clients were using the hypermedia for client-server communication, the server could embed links into a response so the client would just have to follow links, without the need to keep the list of all the resource URIs. The server would be able to provide the list of possible actions/links a client can perform at a given state, following those links. That also means the server could manage the workflow through the use of hypermedia, so the client wouldn't have to worry about it.

The lack of hypermedia makes decoupling of the client/server code a bit difficult which eventually prevents the "independent evolvability". What would really help here is if we could move all the business logic from the client side to the server side, because we could then reuse our client for other web APIs, too. Or even better, just like a standard web browser, a standard API client would be able to emerge, handling just any kind of a hypermedia API. Wouldn't it be awesome if you wouldn't have to write an API client at all, each time you create a new hypermedia API?

Imagine now how would it look like if Facebook would provide its own web browser, that only knows how to browse the Facebook's website, then imagine Twitter, Microsoft, Google and all the other popular websites doing the same thing. That's what would have happened if today's world-wide-web was implemented as RESTish instead of RESTful.

Why so many people ignored this constraint and considered their APIs RESTful anyway?

After reading about this HATEOAS constraint, it was clear to me why it was an important feature of a RESTful API. But, to answer this question, I had to read a lot of discussions online in order to understand the reasoning of each side. So, my first impression was that other people maybe didn't care or didn't have time to read Dr. Fielding's dissertation (or at least a Wikipedia article on it), basically blaming it on the people's ignorance. But once I tried to improve one of my RESTish APIs to be truly RESTful, I realized the horrible truth. The amount of work that needs to be done to upgrade your RESTish API to be RESTful is just huge. It is probably easier to start everything from scratch.

A meme about fixing bugs

That explained why so many people didn't care about their APIs not being RESTful. And, to be honest, I really don't blame them. If it works, don't touch it, sure. But, with time, more RESTful APIs will emerge, which will be very easy to extend and will mutate quickly, following the market's demand and that will introduce a lot of trouble for your RESTish API which you won't be able to extend that fast, to be able to keep up with your competition, so guess what happens. All of a sudden, you're out of business.

So, in order for today's APIs to really be RESTful, they need to implement their client-server communication through the use of hypermedia formats. That doesn't mean they only need to wrap their communication in HTML, XML or any other native hypermedia format, but instead it means to make a full use of all the features hypermedia formats provide. That will help them evolve their APIs into something that is very scalable, maintainable and portable, at least.

If you think of a Hypermedia API as a concept of a web browser and a web server, you'll be as close to a RESTful (hypermedia) API as you possibly can. Considering that the hypermedia has been out there for quite some time now, it is very safe to assume that there are some very good things (constraints) present in its design, which are worth implementing. But not every single web service should be implemented as a RESTful service either. Remember what Dr. Roy Fielding said about that:

The REST interface is designed to be efficient for large-grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction.

Dr. Roy Fielding

That being said, it is clear that the hypermedia APIs are very convenient in cases where we need an ease of maintenance, rapid development and scalable deployment. On the other hand, hypermedia APIs will perform poorly if we need to build a time-critical API or a network-efficient service, without too much client/server chatter. These services should be implemented using some other, probably binary-based, form of communication, without caching and layering, to improve the response time and the information accuracy, etc. Obviously, we first need to get our requirements on the table and compare them with the REST requirements, before we decide if the RESTful API service is what we really need. It doesn't mean that, if our API is RESTful, that it's automagically the best possible choice for any use case.

Isn't it strange to use HTML for the web API?

Well, it depends on your goals. As previously said, if your goals match those of the REST then no, it's not strange. You should consider using HTML for your REST API (also XML, ATOM, etc.), because it will give you all the hypermedia elements out of the box. If you are worried about the overhead of the HTML communication (even though HTML can be gzipped with a very high compression ratio) and you want to use something that is smaller and faster, then you should also consider creating a stateful web service (take a look at the table of constraints to see the trade-offs for the "stateless" constraint). What I'm trying to say here is that you probably don't have the same set of goals as the REST web services do. In other words, are you sure you really need a RESTful web service? To properly answer this question, take your requirements (goals) and match them with those of REST. If they don't match, then REST might not be the best thing for you and you are probably better off using some other software architecture style for your web service.

Any interesting links on Hypermedia APIs?

Yes, there are many interesting videos and articles on this topic, but I'll only recommend a few for now, because these authors really understand the bigger picture and you can learn a lot from them. So, here they are:

A great video lecture by Mike Amundsen, recorded at Google in Mountain View, 22 April 2014: REST, Hypermedia, and the Semantic Gap Amundsen has authored numerous books and papers on programming over the last 15 years. His most recent book is a collaboration with Leonard Richardson titled "RESTful Web APIs" published in 2013. His 2011 book, "Building Hypermedia APIs with HTML5 and Node", is an oft-cited reference on building adaptable Web applications.

Mike Amundsen, Director of API Architecture, API Academy

A great video lecture by Jon Moore, recorded at ØREDEV Developer Conference, Malmö, Sweden, 10 Nov 2010: Session about Hypermedia APIs Dr. Moore is the Chief Engineer at Comcast Interactive Media (CIM), a division of Comcast Corporation dedicated to developing and operating online and cross-platform entertainment and media businesses. He guides technical choices that allow CIM to bring innovative products to our customers ever more quickly.

Jon Moore, Chief Engineer at Comcast Interactive Media

An article about Roy Fielding on " Versioning, Hypermedia, and REST"
An article by Martin Fowler on "Richardson Maturity Model (Glory of REST)"

"If you can't explain it simply, you don't understand it well enough"