REST Versioning: The Ripple Effect

A few weeks ago, I posted an illustration of the SOA service + schema versioning ripple effect for incompatible or sematic changes. Stepping out on a limb, I thought I should make a similar illustration for the artifacts in a RESTful service.

The artifacts in such a service is based on the four REST principles:

1. Identification of resources
2. Manipulation of resources through representations
3. Self-descriptive messages
4. Hypermedia as the engine of application state (HATEOAS)

Even REST needs a versioning and compatibility policy. The only thing that is not subject to change is the uniform interface. All other artifacts are subject to semantic or incompatible changes as the service evolves over time. Changes to the flow of the state machine are changed semantics, while changes to the decisions (allowable next state transitions/hyperlinks) need not be.


The ripple effect is a bottom-up effect, and an incompatible change on e.g. a customer address resource will cause an explosion of new versions of all affected representation artifacts. In the end, such a change should propagate even to the service consumers. If not, the consumers would operate on representations that have different real-world effects than they are expected to have.

Semantic changes should always be explicitly communicated to the consumers. Incompatible changes should be treated the same way for consistency, i.e. enforce a uniform versioning policy that allows consumers to be standardized.

Having a versioning strategy let you control the effects of the inevitable changes. Using a compatibility policy will help you alleviate some of the negative effects of versioning.

REST: Decision Services Needed More Than Ever

You can switch from classic SOA using service interfaces expressing business capabilities and domain specific messages and data types, to REST exchanging resource representations in standardized formats over the uniform interface with hypermedia expressing the next allowable state transitions of the processes operating on the resources. But how to you put those next state transitions (links) into the representation in the first place? Where is the semantics of the business events interpreted?

The classic SOA domain specific service interfaces and schemas try to express semantics that can be used when composing processes, not just a common format and protocol. Yes, you get more technical loose coupling in your systems with REST, but still have to do the semantic interpretation of the business messages somewhere. The huge advantage of REST is that the semantic decisions or compositions of processes are not part of the consumers, allowing applications like mashup clients to be standardized and (re)use a wide range of shared RESTful services. The only decisions consumers make is which of the transitions to follow next.

The application state machine using hypermedia as the engine needs to have a mechanism for deducing which are the next allowable state transitions to emit as links into the resource representation. This is where I think that the classical decision services/business rule engine is a good fit even for REST. Granted, it is a bit unusual to calculate a set of possible decisions - allowable state transitions - on beforehand, kind of a dynamic state-driven workflow. But it should be no stranger than adjusting to accepting the uniform interface. In addition, nothing is stopping you from inspecting the incoming resource representation as part of the decision making for controlling the flow of your business processes.

As I’ve said before, to me REST is just another architectural approach to service-orientation, as is EDA.

Service Compatibility: Backwards, Forwards

The definitions for backwards compatible and forwards compatible contracts are straightforward:

• A new version of a contract that continues to support consumers designed to work with the old version of the contract is considered to be Backwards Compatible


• A contract that is designed to support unknown future consumers is considered to be Forwards Compatible

Backwards compatibility is typically achieved by using optional schema components, while forwards compatibility is typically achieved by using schema wildcards. What can be confusing is that schema compatibility is strictly defined as being between message sender and receiver, while it now is more common to talk about service consumers and service providers.


The correct definition of forwards compatible schemas as defined by David Orchard is this: "In schema terms, this is when a schema processor with an older schema can process and validate an instance that is valid against a newer schema". Backwards compatibility means that a new version of a receiver can be rolled out so it does not break existing senders. Forwards compatibility means that an older version of a receiver can consume newer messages and not break.

The confusion is caused when applying this message based definition to services, as service operations typically are both receivers and senders of messages. Most services should be designed to support both backwards and forwards compatible messages. But are the services themselves backwards or forwards compatible?

I define service compatibility this way:

• A new version of a service that continues to support consumers designed to work with the old version of the service is considered to be Backwards Compatible


• A service that is designed to support unknown future consumers is considered to be Forwards Compatible

Backwards and forwards compatible services use a combined “ignore unknown/missing” strategy, that is a combination of forwards and backwards contract (schema) compatibility. The following figures illustrates the definition of forwards and backwards services.


As can be seen from the above figure, service backwards compatibility depends on the provider being able to validate the on-the-wire request XML against a newer schema version and the consumer being able to validate the response XML against an older schema version. The provider must be able to "ignore missing", while the consumer must be able to "ignore unknown".

As can be seen from the above figure, service forwards compatibility depends on the provider being able to validate the on-the-wire request XML against an older schema version and the consumer being able to validate the response XML against a newer schema version. The provider must be able to "ignore unknown", while the consumer must be able to "ignore missing".

So my advice when talking about compatibility is, always make it clear if you're focusing on the contracts (message) or the services (provider). You can even talk about compatibility from the consumer perspective if you're bold enough. But please: never, ever talk about the service provider as the message consumer...

To add to the complexity of forwards compatibility, there are three types of forward:

• Schema forward compatibilty
  
• Service forward compatibilty
• Routing forward compatibilty, a.k.a implicit versioning
  

In service version routing, the service endpoint accepts multiple versions of the contract (service virtualization) and then applies one of two routing policies:

• Implicit version routing: forwards compatible service routing, where a message automatically is routed to the newest compatible version
  
• Explicit version routing: traditional service routing, where each message is routed based on explicit version information in the message, typically a namespace
  

The implicit version routing policy is what our InfoQ article refers to as forwards compatible service versioning.

[UPDATE] More details on service vs schema compatibility.

Published on InfoQ: SOA Versioning

An article about "Contract Versioning, Compatibility & Composability" in service-oriented solutions that I've written together with Jean-Jaques Dubray has been published on InfoQ. It covers a lot of themes that I've written about in this blog, and focuses on the need for having a versioned common information model for the enterprise data that comprise the messages used in your business processes.