Service Compatibility - A Primer

In a comment on the InfoQ article Contract Versioning, Compatibility and Composability about my definition of service forwards and backwards compatibility, the problem of talking about compatibility of services compared to the definition of schema compatibility is acknowledged.

The "problem" is that a service version that is compatible with the specifications of older versions of the service, can be achieved using both backwards and forwards compatible schemas. That is correct, but doesn't preclude having a definition of forwards and backwards compatibility for service providers, a.k.a "services". Service compatibility is based on ability to validate a message, it is not based on using wildcards in the schema definition. For a definition of the three types of forward compatibility, see my post on schema, service and routing compatibility.

Seen only from the service provider perspective, how it handles incoming messages is what defines if a service is forwards or backwards compatible (or both). How consumers handle messages sent by the service is really not of any concern for the provider - wait, read on.

Thinking about this within SOAP 1.x constraints, where a WSDL operation has a request message and a response message with fixed schema definitions/version (unilateral contracts), will lead to the conclusion that operations cannot be classified as forwards or backwards compatible, only the message schemas. This is a limitation of SOAP, but not of messaging in general.

In the following examples, the v1.2 service provider is backwards compatible and interacts with a v1.1 consumer. However, the schemas are not designed to be forwards compatible - they do not support XML extensibility (schema wildcards). In this scenario, the consumer can do either XSD validation of response messages against the v1.1 schema, or do 'validation by projection' of response messages - i.e. do "ignore unkown" validation. Doing 'validation by projection' is a recommended practice for compatibility and really simplifies building SOA solutions - this is also how WCF validates messages. So how to handle consumers that only do XSD validation, without relying on schema wildcards?

In REST, the consumer can put an "accept formats" header in the v1.1 request message, and the service provider can then respond with a v1.1 schema even if the service version is v1.2. The service provider adheres to it's obligations by being backwards compatible, and the consumer is allowed to express it's version expectation - the service has bilateral contracts.

Service Virtualization is a mechanism that can help with service versioning. The task of such an abstract endpoint intermediary is to handle versioning through both service compatibility and schema compatibility. A virtual service supports multiple versions of the service on the same endpoint, and must be capable of processing older requests. The virtual endpoint must have a mechanism that allows for the latest major v1.2 provider to handle v1.1 consumers. The intermediary mediates between the schema versions by transforming or projecting/enriching the messages as needed.

Back to the example, the service provider v1.2 response message can be stripped down to a v1.1 message by the intermediary as it is sent back to the consumer. The net effect is that the service has virtual bilateral contracts.

In messaging in general, by definition there are no duplex channels, only one-way channels (see Enterprise Integration Patterns by Hohpe/Woolf). On top of this, you can have a logical two-way channel for message exchange patterns such as request-response, specified using a "reply-to" address and a "reply-format" (bilateral contracts). The message compatibility is defined by the schema constructs, but just as in REST, the version of the incoming message does not dictate the version of the response message. It is the implementation of the endpoint that processes the messages that defines the compatibility policy of the endpoint, not the schemas.

So, service compatibility do not require using forwards compatible schemas in addition to backwards compatible schemas. The message validation policy is what defines service compatibility.

It is of course much simpler to just have a service compatibility policy based on that the schemas used in the services must be both forwards and backwards compatible - as shown in the "SOAP-style unilateral contracts" service compatibility figures.



Click figures to enlarge.

This way, the service provider or consumer platform need not handle "request-format" and "reply-format" that have different versions. In such a unilateral schema compatibility policy world, services are just intrinsically compatible through schema compatibility.

REST Versioning: The Ripple Effect

A few weeks ago, I posted an illustration of the SOA service + schema versioning ripple effect for incompatible or sematic changes. Stepping out on a limb, I thought I should make a similar illustration for the artifacts in a RESTful service.

The artifacts in such a service is based on the four REST principles:

1. Identification of resources
2. Manipulation of resources through representations
3. Self-descriptive messages
4. Hypermedia as the engine of application state (HATEOAS)

Even REST needs a versioning and compatibility policy. The only thing that is not subject to change is the uniform interface. All other artifacts are subject to semantic or incompatible changes as the service evolves over time. Changes to the flow of the state machine are changed semantics, while changes to the decisions (allowable next state transitions/hyperlinks) need not be.


The ripple effect is a bottom-up effect, and an incompatible change on e.g. a customer address resource will cause an explosion of new versions of all affected representation artifacts. In the end, such a change should propagate even to the service consumers. If not, the consumers would operate on representations that have different real-world effects than they are expected to have.

Semantic changes should always be explicitly communicated to the consumers. Incompatible changes should be treated the same way for consistency, i.e. enforce a uniform versioning policy that allows consumers to be standardized.

Having a versioning strategy let you control the effects of the inevitable changes. Using a compatibility policy will help you alleviate some of the negative effects of versioning.

REST: Decision Services Needed More Than Ever

You can switch from classic SOA using service interfaces expressing business capabilities and domain specific messages and data types, to REST exchanging resource representations in standardized formats over the uniform interface with hypermedia expressing the next allowable state transitions of the processes operating on the resources. But how to you put those next state transitions (links) into the representation in the first place? Where is the semantics of the business events interpreted?

The classic SOA domain specific service interfaces and schemas try to express semantics that can be used when composing processes, not just a common format and protocol. Yes, you get more technical loose coupling in your systems with REST, but still have to do the semantic interpretation of the business messages somewhere. The huge advantage of REST is that the semantic decisions or compositions of processes are not part of the consumers, allowing applications like mashup clients to be standardized and (re)use a wide range of shared RESTful services. The only decisions consumers make is which of the transitions to follow next.

The application state machine using hypermedia as the engine needs to have a mechanism for deducing which are the next allowable state transitions to emit as links into the resource representation. This is where I think that the classical decision services/business rule engine is a good fit even for REST. Granted, it is a bit unusual to calculate a set of possible decisions - allowable state transitions - on beforehand, kind of a dynamic state-driven workflow. But it should be no stranger than adjusting to accepting the uniform interface. In addition, nothing is stopping you from inspecting the incoming resource representation as part of the decision making for controlling the flow of your business processes.

As I’ve said before, to me REST is just another architectural approach to service-orientation, as is EDA.

Guerilla SOA, MEST

InfoQ has posted an interview with Jim Webber of ThoughtWorks about "Guerilla SOA", where he talks about how to design scalable SOA solutions that are evolvable and allow innovation in the different services that make up the solution.

The advice he offers covers topics that I have written about in my posts these last months:

• Share business messages owned by the business people (business process information model)
• SOA has no operations, only business messages delivered to letter boxes - services exchange messages to fulfill business processes
• Services cannot be invoked, they just receive event+payload messages and decide if and how to process it: "Could you possibly have a look at this message and maybe if it suits you do some processing on it" (semantic covenant: the service is always right)
• Business processes are workflows; you need to model long-lived conversations beyond request/response (SSDL, check out the Soya WCF SSDL toolkit at SourceForge)

This architectual style is coined MEST, as it resembles REST: it is based on message exchange through a uniform interface - the letterbox. As you will see, MEST is a decoupled scalable design pattern along the lines of EDA.

Listen to the interview and read the transcript: Jim Webber on "Guerilla SOA"

Service Architecture: JBOWS, SOA or WOA

A few days ago, I got into a discussion of whether the Amazon Web Services are true SOA or just a programming model exposed using web-services. The discussion started when I said that CRUD style operations are not according to SOA best practices; and that operating on parts of a domain object of type aggregate root should be avoided, favoring actions on the complete entity to be exposed as service operations.

Before making my SOA-vs-JBOWS case, a short introduction to WOA: as adhering to all the SOA service design principles and the different WS-* technologies can be quite daunting and complex, a lot of service-oriented solutions have emerged that take a simpler approach (REST, SOAP, POX, etc). Gartner has coined the acronym WOA (Web-Oriented Architecture) for these kinds of solutions that implement and expose services using more pragmatic techniques following the WOA tenets.

First the "what's in an operation name" issue:

It is not best practice to use CRUDy style operation names that conveys only that the domain object state is going to change in the repository (e.g. UpdateCustomerAddress). You should rather use operation names that reveals the event in the business process that caused the action (e.g. CustomerHasMoved). The point is that your service will most likely perform some business logic on the domain object in conjunction with storing it in the database. The operation name should convey the fact that the state of the real-world entity represented by the domain object has changed.

Note that you need not change the operation names very much, sometimes renaming UpdateXxx to ChangeXxx can be sufficient to convey the correct semantics. Focus on creating business process driven services rather than data-driven services.

The discussion I had was about if names such as AddXxx, ModifyXxx and DeleteXxx are CRUDy style names or not. I think they are to closely named after what the code is going to do with the domain object repository rather than reflecting changes to the real-life entity, i.e. it is more a programming model (WOA) than a SOA operation.

Note that CRUDy operations and names can be quite ok, especially for information services, which will still be needed to manage data in your repositories. There will still be a need for creating new customers, even in SOA.

Then on to the operations on different types of domain objects issue:

A common metaphor for designing SOA operation and data contracts is to think of paper forms being passed around to clerks to fulfil a business process. Another metaphor is mail orders, the point is that the 'document' contains all data needed to perform the business process.

Documents can be simple domain objects or aggregate root objects. An order is an example of an aggregate root object - it contains general order data and a set of order items. The order items belong to the order, i.e. it is an identifying relationship and not just a relation. As a general rule, an operation should process a whole document and never mess directly with 'identified items' within the document. Such operations would be very CRUDy style, and breaks the "boundaries are explicit" tenet.


Note that it is perfectly legal to operate on normal relations, such as adding orders to a customer. It is the nature of the domain and its business processes that decide if a relation is identifying or not, thus there is no hard rule to help you decide whether incremental operations are ok or not.

Using the 'document' metaphor, you would implement a 'RegisterOrder' operation to add new orders. You should, however, not implement operations for changing a subset of the order items; you must rather implement a 'ChangeOrder' or a 'CancelOrder' operation.

In my opinion, the Amazon shopping cart is analogous to the above order domain object. Thus, according to SOA best practices, they should not expose operations such as ChartModify that allows you to do incremental changes to the items in the chart. I understand that they do this for simplicity and performance reasons, and that the shopping cart is really not registered until it is submitted; but some developers use the AWS as general best practices for SOA and apply the same programming model blindly.

I don't propose that Amazon Web Services are not properly designed; they are just WOA rather than SOA.
A closing note: there is no "one or the other" between SOA and WOA, in fact they are both central in web 2.0. Read more about it in this report from the Web 2.0 Summit.