It's been five years since Eric Evans' classic book Domain-driven design: tackling complexity in the heart of software was published. Last week two interesting articles came out; What I've learned about DDD since the book by Eric himself and Domain Models: Employing the Domain Model Pattern by Udi Dahan. They both talk of lessons learnt about some of the concepts of domain-driven design (DDD).
Udi talks about a very common misconception about domain Entity objects, what I like to call the Fat Domain Model: using the persistent object model as the Domain Model. I've done this myself for many years, trying to avoid Martin Fowler's Anemic Domain Model and Fat Service Layer anti-patterns by adding the domain entity's behavior to the related data entity object.
Microsoft does a variant of the Fat Service Layer in their Three-Layered Services Application guidance: they use anemic Business Entity objects and keeps the business logic in separate Business Component objects, exposed by Service Interface objects. So coming from this, it seemed natural to merge the entity and logic into one when I started practicing DDD.
The description of the Entity pattern from Eric's book is clear on what goes into the entity object:
"It is natural to think about the attributes when modeling an object, and it is quite important to think about its behavior. [...] strip the Entity object's definition down to the most intrinsic characteristics [...]. Add only behavior that is essential to the [domain] concept and attributes that are required by that behavior."
So, only the data that is required by the domain logic should be in the domain object. Eric use the term "information about the business situation" in the book when explaining the Domain Layer, another hint that this is not traditional data entities. Querying and viewing entities is not domain logic, so there is no need to add all attributes of the ORM data entities used in the presentation layer to the DDD Entity object. Querying and viewing entities belongs to the Service Layer. I suggest using two separate object models (Command and Query Responsibility Segregation, CQRS) as shown in Udi's article, but think twice before implementing your forms-over-data application using the Domain Model pattern.
In his QCon 2009 presentation, Eric talks about the DDD Aggregate pattern and how it can be hard to model an Aggregate Boundary and decide to which Entity object a specific behavior belongs. Here is the definition of Aggregate from the book:
"First we need an abstraction for encapsulating references within the [domain] model. An Aggregate is a cluster of associated [domain] objects that we treat as a unit for the purpose of [ensuring consistency of] data changes. Each Aggregate has a root and a boundary. The root is a single, specific Entity contained in the Aggregate."
I think the Aggregate pattern is easy to misunderstand as it focuses on modeling the relationships and boundary of the root and its associated collection of domain objects - immediately making people think of the classical "purchase order with items collection" OO data encapsulation design or even ER-design. In addition, the strong focus the Aggregate being an abstract consistency mechanism leads to the misconception that Aggregate domain logic must belong to the root Entity object and that logic that involves associated Aggregate data will require that data to be modeled into the collection of domain objects.
Eric uses the example of calculating the weight of a grape cluster Aggregate. This logic doesn't belong on the grape Entity, so the only place to put this is on the stem Entity - because the Aggregate itself is not an object, just an abstract concept. He also talks about how this kind of logic typically is designed as an iteration of the associated object collection, while lessons learnt has shown that this OO legacy mindset is not mandated by the domain business rules. Udi also touches the same problem in the "UnpaidOrdersAmount" example in his domain model article.
The fact is that the domain aggregate is not just an abstract concept, it should be modeled as what I like to call a Concrete Aggregate. This alleviates the root Entity from getting responsibilities that doesn't naturally belong to it. This kind of non-entity domain logic goes into the Concrete Aggregate as its behavior and prevents leaking domain logic into the Service Layer.
Using Concrete Aggregate objects might also lessen the need for fully modeling objects within Aggregate boundaries. E.g. having a GrapeCluster object that can calculate its weight directly without loading a Stem and its collection of Grape objects makes the domain model much simpler. Again note the focus on business rules behavior rather than OO entity-relationship modeling.
I admit struggling with naming the Concrete Aggregate pattern, maybe Aggregate Entity would be a more self-explanatory name.
Showing posts with label SOA. Show all posts
Showing posts with label SOA. Show all posts
Monday, August 17, 2009
Thursday, February 12, 2009
SOA CIM: Common Information Model
As I've written about before, there is a lot of confusion and different interpretations of what a "Common Information Model" (CIM) is. The most common reification of CIM is to imply that it is the same concept as in chapter 4 "XML: The Foundation for Business Data Integration" in David Chappell's seminal book Enterprise Service Bus from 2004. Dave describes the need for having a common XML data format for "expressing data in messages as it flows through an enterprise across the ESB", but never uses the term CIM or describe any modeling approach. This common format will naturally comprise any kind of XML data used in the service bus, including messages, business entities and event data.
The CIM concept in relation to SOA was described by Mike Rosen and Eric Roch in 2006: it defines a common representation of business entity objects that provides a canonical format and unified semantics for a business domain.
The above figure is from Mike Rosen's article Business Architecture and SOA. As can be seen in the figure, CIM comprises a shared information model that is the basis for the documents that flows through the business processes built using the capabilities provided by the services.
Mike explains the role of CIM like this:
[The SOA Business Model] has to manage the sharing of services and information across processes. In other words, it needs to eliminate redundancy, overlap, and gaps between services so that each business capability is implemented once, by the organizational unit that is responsible for that capability. And that those services are used by all the different processes needing those capabilities. In addition, all of the information shared between services must be identified in the common information model. In other words, all services that are related to the same business concepts must use the same information to describe those concepts. Finally, the SOA Business Model must ensure that all of the information passed into and out of the business services (mostly in the form of documents) is defined in the [shared semantic] information model.
I hope this clarifies the definition of what a CIM is: it is a model that comprises shared information entities such as "Customer", but not actual messages like "CustomerHasMoved" defined in the business process information model (BPIM). The message types such as "AddressChange" (document) are part of this logical model as projections of the shared entities.
The "Canonical Schemas" used in messages are part of the service model's standardized service contracts. Note that these schemas are generated from, but are not part of CIM. Generating the canonical schemas from CIM makes enforcing "Schema Centralization" unnecessary, as they already share a common basis. This helps alleviate the negative effects of a one true schema approach; e.g. the versioning ripple effect through the canonical data model that affects all related service contracts - even if not intended.
Do not confuse CIM with the centralized SOA logical data model (LDM) approach. The services speak "CIM" natively. Service messages are not transformed to/from CIM by a message broker as in the LDM schema bus approach. Still, using the message broker approach is handy to include non-conformant services in your SOA solutions. Enforcing a LDM for your information will cause the same problems as the traditional EAI hub-and-spoke canonical data model (CDM).
I recommend reading chapter 5 "Service Context and Common Semantics" in the "Applied SOA: Service-Oriented Architecture and Design Strategies" book by Michael Rosen et al. Figure 5-12, 5-14 and 5-15 in the book shows how a grapical projection DSL for documents (message types) might look.
The CIM concept in relation to SOA was described by Mike Rosen and Eric Roch in 2006: it defines a common representation of business entity objects that provides a canonical format and unified semantics for a business domain.
The above figure is from Mike Rosen's article Business Architecture and SOA. As can be seen in the figure, CIM comprises a shared information model that is the basis for the documents that flows through the business processes built using the capabilities provided by the services.Mike explains the role of CIM like this:
[The SOA Business Model] has to manage the sharing of services and information across processes. In other words, it needs to eliminate redundancy, overlap, and gaps between services so that each business capability is implemented once, by the organizational unit that is responsible for that capability. And that those services are used by all the different processes needing those capabilities. In addition, all of the information shared between services must be identified in the common information model. In other words, all services that are related to the same business concepts must use the same information to describe those concepts. Finally, the SOA Business Model must ensure that all of the information passed into and out of the business services (mostly in the form of documents) is defined in the [shared semantic] information model.
I hope this clarifies the definition of what a CIM is: it is a model that comprises shared information entities such as "Customer", but not actual messages like "CustomerHasMoved" defined in the business process information model (BPIM). The message types such as "AddressChange" (document) are part of this logical model as projections of the shared entities.
The "Canonical Schemas" used in messages are part of the service model's standardized service contracts. Note that these schemas are generated from, but are not part of CIM. Generating the canonical schemas from CIM makes enforcing "Schema Centralization" unnecessary, as they already share a common basis. This helps alleviate the negative effects of a one true schema approach; e.g. the versioning ripple effect through the canonical data model that affects all related service contracts - even if not intended.
Do not confuse CIM with the centralized SOA logical data model (LDM) approach. The services speak "CIM" natively. Service messages are not transformed to/from CIM by a message broker as in the LDM schema bus approach. Still, using the message broker approach is handy to include non-conformant services in your SOA solutions. Enforcing a LDM for your information will cause the same problems as the traditional EAI hub-and-spoke canonical data model (CDM).
I recommend reading chapter 5 "Service Context and Common Semantics" in the "Applied SOA: Service-Oriented Architecture and Design Strategies" book by Michael Rosen et al. Figure 5-12, 5-14 and 5-15 in the book shows how a grapical projection DSL for documents (message types) might look.
Tuesday, February 10, 2009
SOA: Business Event Message Models
InfoQ has published an article about SOA Message Type Architecture by Jean-Jacques Dubray. The article shows how to model 'message type' artifacts based on an enterprise common data model using a DSL, and also outlines how the modeled artifacts can be used to generate XML schemas for use in your service contracts. The message type DSL is not for modeling messages, it is just for modeling the types used as message payloads. In WCF terms, a message type is a [DataContract].
Note that even if the message type model contains a set of standardized verbs, the model does not cover business process aspects such as flow and actions. You will still need to analyze the business processes that pass those message types around to model the action, query and notification events that drive the processes. I've written several times about creating such a model that comprise both the business events and the message types, a business process information model (BPIM). I like JJ's approach; I just think that we need to model also the business capabilities and interactions that utilize the message types to get a complete set of artifacts for service contracts.
One difference is that I prefer using a common information model (CIM) as the basis for modeling the message types, rather than an enterprise data model (EDM). It is a lot of effort to create an EDM that covers all information in all systems-of-record in a company; and the moment you have completed the all-encompassing model, your CxO will inform you that parts of the business have been outsourced or that a new business will have to be incorporated, or even just that the CRM system is to be replaced. Change is the only constant. Thus I prefer starting small by creating a CIM that covers only the business entities comprised by the business processes that are about to be service-oriented. As there will be multiple resource domains in your architecture, there will be multiple CIM models as your SOA grows. Federate these domain models by creating context maps for cross-domain capabilities and logic only.
Create a CIM according to the Domain-Driven Design (DDD) principles and use the domain entity objects when implementing the business logic underlying your services. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain. As the BPIM is a projection of the CIM, the service interface messages will have an unambiguous mapping to/from the underlying domain resources (entities and aggregates). Mapping data between the message types and objects will be required in the service implementation (provider container).
The 'message type' artifacts should also be partitioned into bounded contexts according to business area as in Domain-Driven Design; where each service domain comprise a set of cohesive services based on the same underlying CIM, delivering an BPIM for the specific business domain. Having clear domain boundaries make it easier to analyze, model and design, and version the artifacts - it also aids the discoverability by the consumers by providing a clear business context for the provided services.
Your SOA solution will, as it grows, encompass multiple service domains each with its own BPIM. Composite services that compose business capabilities across two or more domains will require mediation between the message models. Even if "transformation avoidance" is considered a SOA best practice, it is unrealistic that you will be able to avoid mediation completely in your service bus (composition container). In addition, you cannot expect to enforce your model upon the extended enterprise, think of e.g. third-party and outsourced capabilities.
The message types exposed by a service domain implicitly become the "canonical schema". They enable better service discoverability, reuse and composition as they all share the same underlying data model - which also ensures that the message payloads have common semantics within the service domain. In combination with business functions, the schemas provide a complete standardized service contract.
As you have seen, the BPIM and CIM approach fits well with DDD. It also fits well with the middle-out SOA approach - including the recommendation to start small and think big picture, not big bang.
Note that even if the message type model contains a set of standardized verbs, the model does not cover business process aspects such as flow and actions. You will still need to analyze the business processes that pass those message types around to model the action, query and notification events that drive the processes. I've written several times about creating such a model that comprise both the business events and the message types, a business process information model (BPIM). I like JJ's approach; I just think that we need to model also the business capabilities and interactions that utilize the message types to get a complete set of artifacts for service contracts.
One difference is that I prefer using a common information model (CIM) as the basis for modeling the message types, rather than an enterprise data model (EDM). It is a lot of effort to create an EDM that covers all information in all systems-of-record in a company; and the moment you have completed the all-encompassing model, your CxO will inform you that parts of the business have been outsourced or that a new business will have to be incorporated, or even just that the CRM system is to be replaced. Change is the only constant. Thus I prefer starting small by creating a CIM that covers only the business entities comprised by the business processes that are about to be service-oriented. As there will be multiple resource domains in your architecture, there will be multiple CIM models as your SOA grows. Federate these domain models by creating context maps for cross-domain capabilities and logic only.
Create a CIM according to the Domain-Driven Design (DDD) principles and use the domain entity objects when implementing the business logic underlying your services. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain. As the BPIM is a projection of the CIM, the service interface messages will have an unambiguous mapping to/from the underlying domain resources (entities and aggregates). Mapping data between the message types and objects will be required in the service implementation (provider container).
The 'message type' artifacts should also be partitioned into bounded contexts according to business area as in Domain-Driven Design; where each service domain comprise a set of cohesive services based on the same underlying CIM, delivering an BPIM for the specific business domain. Having clear domain boundaries make it easier to analyze, model and design, and version the artifacts - it also aids the discoverability by the consumers by providing a clear business context for the provided services.
Your SOA solution will, as it grows, encompass multiple service domains each with its own BPIM. Composite services that compose business capabilities across two or more domains will require mediation between the message models. Even if "transformation avoidance" is considered a SOA best practice, it is unrealistic that you will be able to avoid mediation completely in your service bus (composition container). In addition, you cannot expect to enforce your model upon the extended enterprise, think of e.g. third-party and outsourced capabilities.
The message types exposed by a service domain implicitly become the "canonical schema". They enable better service discoverability, reuse and composition as they all share the same underlying data model - which also ensures that the message payloads have common semantics within the service domain. In combination with business functions, the schemas provide a complete standardized service contract.
As you have seen, the BPIM and CIM approach fits well with DDD. It also fits well with the middle-out SOA approach - including the recommendation to start small and think big picture, not big bang.
Sunday, February 01, 2009
Service Compatibility - A Primer
In a comment on the InfoQ article Contract Versioning, Compatibility and Composability about my definition of service forwards and backwards compatibility, the problem of talking about compatibility of services compared to the definition of schema compatibility is acknowledged.
The "problem" is that a service version that is compatible with the specifications of older versions of the service, can be achieved using both backwards and forwards compatible schemas. That is correct, but doesn't preclude having a definition of forwards and backwards compatibility for service providers, a.k.a "services". Service compatibility is based on ability to validate a message, it is not based on using wildcards in the schema definition. For a definition of the three types of forward compatibility, see my post on schema, service and routing compatibility.
Seen only from the service provider perspective, how it handles incoming messages is what defines if a service is forwards or backwards compatible (or both). How consumers handle messages sent by the service is really not of any concern for the provider - wait, read on.
Thinking about this within SOAP 1.x constraints, where a WSDL operation has a request message and a response message with fixed schema definitions/version (unilateral contracts), will lead to the conclusion that operations cannot be classified as forwards or backwards compatible, only the message schemas. This is a limitation of SOAP, but not of messaging in general.
In the following examples, the v1.2 service provider is backwards compatible and interacts with a v1.1 consumer. However, the schemas are not designed to be forwards compatible - they do not support XML extensibility (schema wildcards). In this scenario, the consumer can do either XSD validation of response messages against the v1.1 schema, or do 'validation by projection' of response messages - i.e. do "ignore unkown" validation. Doing 'validation by projection' is a recommended practice for compatibility and really simplifies building SOA solutions - this is also how WCF validates messages. So how to handle consumers that only do XSD validation, without relying on schema wildcards?
In REST, the consumer can put an "accept formats" header in the v1.1 request message, and the service provider can then respond with a v1.1 schema even if the service version is v1.2. The service provider adheres to it's obligations by being backwards compatible, and the consumer is allowed to express it's version expectation - the service has bilateral contracts.
Service Virtualization is a mechanism that can help with service versioning. The task of such an abstract endpoint intermediary is to handle versioning through both service compatibility and schema compatibility. A virtual service supports multiple versions of the service on the same endpoint, and must be capable of processing older requests. The virtual endpoint must have a mechanism that allows for the latest major v1.2 provider to handle v1.1 consumers. The intermediary mediates between the schema versions by transforming or projecting/enriching the messages as needed.
Back to the example, the service provider v1.2 response message can be stripped down to a v1.1 message by the intermediary as it is sent back to the consumer. The net effect is that the service has virtual bilateral contracts.
In messaging in general, by definition there are no duplex channels, only one-way channels (see Enterprise Integration Patterns by Hohpe/Woolf). On top of this, you can have a logical two-way channel for message exchange patterns such as request-response, specified using a "reply-to" address and a "reply-format" (bilateral contracts). The message compatibility is defined by the schema constructs, but just as in REST, the version of the incoming message does not dictate the version of the response message. It is the implementation of the endpoint that processes the messages that defines the compatibility policy of the endpoint, not the schemas.
So, service compatibility do not require using forwards compatible schemas in addition to backwards compatible schemas. The message validation policy is what defines service compatibility.
It is of course much simpler to just have a service compatibility policy based on that the schemas used in the services must be both forwards and backwards compatible - as shown in the "SOAP-style unilateral contracts" service compatibility figures.
Click figures to enlarge.
This way, the service provider or consumer platform need not handle "request-format" and "reply-format" that have different versions. In such a unilateral schema compatibility policy world, services are just intrinsically compatible through schema compatibility.
The "problem" is that a service version that is compatible with the specifications of older versions of the service, can be achieved using both backwards and forwards compatible schemas. That is correct, but doesn't preclude having a definition of forwards and backwards compatibility for service providers, a.k.a "services". Service compatibility is based on ability to validate a message, it is not based on using wildcards in the schema definition. For a definition of the three types of forward compatibility, see my post on schema, service and routing compatibility.
Seen only from the service provider perspective, how it handles incoming messages is what defines if a service is forwards or backwards compatible (or both). How consumers handle messages sent by the service is really not of any concern for the provider - wait, read on.
Thinking about this within SOAP 1.x constraints, where a WSDL operation has a request message and a response message with fixed schema definitions/version (unilateral contracts), will lead to the conclusion that operations cannot be classified as forwards or backwards compatible, only the message schemas. This is a limitation of SOAP, but not of messaging in general.
In the following examples, the v1.2 service provider is backwards compatible and interacts with a v1.1 consumer. However, the schemas are not designed to be forwards compatible - they do not support XML extensibility (schema wildcards). In this scenario, the consumer can do either XSD validation of response messages against the v1.1 schema, or do 'validation by projection' of response messages - i.e. do "ignore unkown" validation. Doing 'validation by projection' is a recommended practice for compatibility and really simplifies building SOA solutions - this is also how WCF validates messages. So how to handle consumers that only do XSD validation, without relying on schema wildcards?
In REST, the consumer can put an "accept formats" header in the v1.1 request message, and the service provider can then respond with a v1.1 schema even if the service version is v1.2. The service provider adheres to it's obligations by being backwards compatible, and the consumer is allowed to express it's version expectation - the service has bilateral contracts.
Service Virtualization is a mechanism that can help with service versioning. The task of such an abstract endpoint intermediary is to handle versioning through both service compatibility and schema compatibility. A virtual service supports multiple versions of the service on the same endpoint, and must be capable of processing older requests. The virtual endpoint must have a mechanism that allows for the latest major v1.2 provider to handle v1.1 consumers. The intermediary mediates between the schema versions by transforming or projecting/enriching the messages as needed.
Back to the example, the service provider v1.2 response message can be stripped down to a v1.1 message by the intermediary as it is sent back to the consumer. The net effect is that the service has virtual bilateral contracts.
In messaging in general, by definition there are no duplex channels, only one-way channels (see Enterprise Integration Patterns by Hohpe/Woolf). On top of this, you can have a logical two-way channel for message exchange patterns such as request-response, specified using a "reply-to" address and a "reply-format" (bilateral contracts). The message compatibility is defined by the schema constructs, but just as in REST, the version of the incoming message does not dictate the version of the response message. It is the implementation of the endpoint that processes the messages that defines the compatibility policy of the endpoint, not the schemas.
So, service compatibility do not require using forwards compatible schemas in addition to backwards compatible schemas. The message validation policy is what defines service compatibility.
It is of course much simpler to just have a service compatibility policy based on that the schemas used in the services must be both forwards and backwards compatible - as shown in the "SOAP-style unilateral contracts" service compatibility figures.
Click figures to enlarge.
This way, the service provider or consumer platform need not handle "request-format" and "reply-format" that have different versions. In such a unilateral schema compatibility policy world, services are just intrinsically compatible through schema compatibility.
Tuesday, January 13, 2009
SOAMM: Published on InfoQ
A followup article to the article on "Contract Versioning, Compatibility & Composability" is now published at InfoQ. This new article is about showing how the recommended contract design policies from the versioning article relate to Microsoft's SOA Maturity Model (SOAMM), and at the same time suggest a roadmap for achieving these capabilities.
SOAMM Overview:
SOAMM Capabilities:
Click to enlarge pictures.
Related resources:
Unum Case Study
SOAMM web-cast
I recommend the web-cast for a good overview of the maturity model.
SOAMM Overview:
SOAMM Capabilities:
Click to enlarge pictures.Related resources:
Unum Case Study
SOAMM web-cast
I recommend the web-cast for a good overview of the maturity model.
Friday, December 19, 2008
REST Versioning: The Ripple Effect
A few weeks ago, I posted an illustration of the SOA service + schema versioning ripple effect for incompatible or sematic changes. Stepping out on a limb, I thought I should make a similar illustration for the artifacts in a RESTful service.
The artifacts in such a service is based on the four REST principles:
1. Identification of resources
2. Manipulation of resources through representations
3. Self-descriptive messages
4. Hypermedia as the engine of application state (HATEOAS)
Even REST needs a versioning and compatibility policy. The only thing that is not subject to change is the uniform interface. All other artifacts are subject to semantic or incompatible changes as the service evolves over time. Changes to the flow of the state machine are changed semantics, while changes to the decisions (allowable next state transitions/hyperlinks) need not be.
The ripple effect is a bottom-up effect, and an incompatible change on e.g. a customer address resource will cause an explosion of new versions of all affected representation artifacts. In the end, such a change should propagate even to the service consumers. If not, the consumers would operate on representations that have different real-world effects than they are expected to have.
Semantic changes should always be explicitly communicated to the consumers. Incompatible changes should be treated the same way for consistency, i.e. enforce a uniform versioning policy that allows consumers to be standardized.
Having a versioning strategy let you control the effects of the inevitable changes. Using a compatibility policy will help you alleviate some of the negative effects of versioning.
The artifacts in such a service is based on the four REST principles:
1. Identification of resources
2. Manipulation of resources through representations
3. Self-descriptive messages
4. Hypermedia as the engine of application state (HATEOAS)
Even REST needs a versioning and compatibility policy. The only thing that is not subject to change is the uniform interface. All other artifacts are subject to semantic or incompatible changes as the service evolves over time. Changes to the flow of the state machine are changed semantics, while changes to the decisions (allowable next state transitions/hyperlinks) need not be.
The ripple effect is a bottom-up effect, and an incompatible change on e.g. a customer address resource will cause an explosion of new versions of all affected representation artifacts. In the end, such a change should propagate even to the service consumers. If not, the consumers would operate on representations that have different real-world effects than they are expected to have.
Semantic changes should always be explicitly communicated to the consumers. Incompatible changes should be treated the same way for consistency, i.e. enforce a uniform versioning policy that allows consumers to be standardized.
Having a versioning strategy let you control the effects of the inevitable changes. Using a compatibility policy will help you alleviate some of the negative effects of versioning.
Tuesday, December 16, 2008
REST: Decision Services Needed More Than Ever
You can switch from classic SOA using service interfaces expressing business capabilities and domain specific messages and data types, to REST exchanging resource representations in standardized formats over the uniform interface with hypermedia expressing the next allowable state transitions of the processes operating on the resources. But how to you put those next state transitions (links) into the representation in the first place? Where is the semantics of the business events interpreted?
The classic SOA domain specific service interfaces and schemas try to express semantics that can be used when composing processes, not just a common format and protocol. Yes, you get more technical loose coupling in your systems with REST, but still have to do the semantic interpretation of the business messages somewhere. The huge advantage of REST is that the semantic decisions or compositions of processes are not part of the consumers, allowing applications like mashup clients to be standardized and (re)use a wide range of shared RESTful services. The only decisions consumers make is which of the transitions to follow next.
The application state machine using hypermedia as the engine needs to have a mechanism for deducing which are the next allowable state transitions to emit as links into the resource representation. This is where I think that the classical decision services/business rule engine is a good fit even for REST. Granted, it is a bit unusual to calculate a set of possible decisions - allowable state transitions - on beforehand, kind of a dynamic state-driven workflow. But it should be no stranger than adjusting to accepting the uniform interface. In addition, nothing is stopping you from inspecting the incoming resource representation as part of the decision making for controlling the flow of your business processes.
As I’ve said before, to me REST is just another architectural approach to service-orientation, as is EDA.
The classic SOA domain specific service interfaces and schemas try to express semantics that can be used when composing processes, not just a common format and protocol. Yes, you get more technical loose coupling in your systems with REST, but still have to do the semantic interpretation of the business messages somewhere. The huge advantage of REST is that the semantic decisions or compositions of processes are not part of the consumers, allowing applications like mashup clients to be standardized and (re)use a wide range of shared RESTful services. The only decisions consumers make is which of the transitions to follow next.
The application state machine using hypermedia as the engine needs to have a mechanism for deducing which are the next allowable state transitions to emit as links into the resource representation. This is where I think that the classical decision services/business rule engine is a good fit even for REST. Granted, it is a bit unusual to calculate a set of possible decisions - allowable state transitions - on beforehand, kind of a dynamic state-driven workflow. But it should be no stranger than adjusting to accepting the uniform interface. In addition, nothing is stopping you from inspecting the incoming resource representation as part of the decision making for controlling the flow of your business processes.
As I’ve said before, to me REST is just another architectural approach to service-orientation, as is EDA.
Thursday, December 11, 2008
Service Compatibility: Backwards, Forwards
The definitions for backwards compatible and forwards compatible contracts are straightforward:
The correct definition of forwards compatible schemas as defined by David Orchard is this: "In schema terms, this is when a schema processor with an older schema can process and validate an instance that is valid against a newer schema". Backwards compatibility means that a new version of a receiver can be rolled out so it does not break existing senders. Forwards compatibility means that an older version of a receiver can consume newer messages and not break.
The confusion is caused when applying this message based definition to services, as service operations typically are both receivers and senders of messages. Most services should be designed to support both backwards and forwards compatible messages. But are the services themselves backwards or forwards compatible?
I define service compatibility this way:
As can be seen from the above figure, service backwards compatibility depends on the provider being able to validate the on-the-wire request XML against a newer schema version and the consumer being able to validate the response XML against an older schema version. The provider must be able to "ignore missing", while the consumer must be able to "ignore unknown".
As can be seen from the above figure, service forwards compatibility depends on the provider being able to validate the on-the-wire request XML against an older schema version and the consumer being able to validate the response XML against a newer schema version. The provider must be able to "ignore unknown", while the consumer must be able to "ignore missing".
So my advice when talking about compatibility is, always make it clear if you're focusing on the contracts (message) or the services (provider). You can even talk about compatibility from the consumer perspective if you're bold enough. But please: never, ever talk about the service provider as the message consumer...
To add to the complexity of forwards compatibility, there are three types of forward:
- A new version of a contract that continues to support consumers designed to work with the old version of the contract is considered to be Backwards Compatible
- A contract that is designed to support unknown future consumers is considered to be Forwards Compatible
The correct definition of forwards compatible schemas as defined by David Orchard is this: "In schema terms, this is when a schema processor with an older schema can process and validate an instance that is valid against a newer schema". Backwards compatibility means that a new version of a receiver can be rolled out so it does not break existing senders. Forwards compatibility means that an older version of a receiver can consume newer messages and not break.
The confusion is caused when applying this message based definition to services, as service operations typically are both receivers and senders of messages. Most services should be designed to support both backwards and forwards compatible messages. But are the services themselves backwards or forwards compatible?
I define service compatibility this way:
- A new version of a service that continues to support consumers designed to work with the old version of the service is considered to be Backwards Compatible
- A service that is designed to support unknown future consumers is considered to be Forwards Compatible
As can be seen from the above figure, service backwards compatibility depends on the provider being able to validate the on-the-wire request XML against a newer schema version and the consumer being able to validate the response XML against an older schema version. The provider must be able to "ignore missing", while the consumer must be able to "ignore unknown".
As can be seen from the above figure, service forwards compatibility depends on the provider being able to validate the on-the-wire request XML against an older schema version and the consumer being able to validate the response XML against a newer schema version. The provider must be able to "ignore unknown", while the consumer must be able to "ignore missing".So my advice when talking about compatibility is, always make it clear if you're focusing on the contracts (message) or the services (provider). You can even talk about compatibility from the consumer perspective if you're bold enough. But please: never, ever talk about the service provider as the message consumer...
To add to the complexity of forwards compatibility, there are three types of forward:
- Schema forward compatibilty
- Service forward compatibilty
- Routing forward compatibilty, a.k.a implicit versioning
- Implicit version routing: forwards compatible service routing, where a message automatically is routed to the newest compatible version
- Explicit version routing: traditional service routing, where each message is routed based on explicit version information in the message, typically a namespace
The implicit version routing policy is what our InfoQ article refers to as forwards compatible service versioning.
Tuesday, December 09, 2008
Published on InfoQ: SOA Versioning
An article about "Contract Versioning, Compatibility & Composability" in service-oriented solutions that I've written together with Jean-Jaques Dubray has been published on InfoQ. It covers a lot of themes that I've written about in this blog, and focuses on the need for having a versioned common information model for the enterprise data that comprise the messages used in your business processes.
Thursday, November 27, 2008
Service+Schema Versioning: The Ripple Effect
In my SOA versioning, compatibility & composability session at NNUG this week, I stressed the importance of recognizing the ripple effect that incompatible or semantic changes to service contract artifacts will have. This illustration captures how versioning an artifact will affect upstream artifacts:
The ripple effect is a bottom-up effect, and an incompatible change on e.g. a customer address schema will cause an explosion of new versions of all affected contract artifacts. In the end, the change will propagate even to the service consumers.
Having a versioning strategy let you control the effects of the inevitable changes. Using a compatibility policy will help you alleviate some of the negative effects of versioning.
The ripple effect is a bottom-up effect, and an incompatible change on e.g. a customer address schema will cause an explosion of new versions of all affected contract artifacts. In the end, the change will propagate even to the service consumers.
Having a versioning strategy let you control the effects of the inevitable changes. Using a compatibility policy will help you alleviate some of the negative effects of versioning.
Tuesday, November 18, 2008
NNUG Oslo 25. November
I'll be giving a session on service+schema compatibility & composability at NNUG Oslo on 25. November: details here.
Among the topics I will cover is the "Flexible/Strict" strategy.
[UPDATE 12-DEC-2008] Download slidedeck here.
Wednesday, November 12, 2008
Service+Schema Versioning: Flexible/Strict Strategy
In a few weeks time I will be giving a session at NNUG on SOA service and schema versioning strategies and practices. A central topic will be schema compatibility rules, where I will recommend creating a policy based on the "Strict", "Flexible" and "Loose" versioning strategies described in chapter 20.4 in the latest Thomas Erl series book Web Service Contract Design and Versioning for SOA. I guess David Orchard is the author/editor of part III in the book.
I recommend using a “Flexible/Strict” compatibility policy:
Note that forwards compatible schemas is not required to have forwards compatible services, as service compatibility is defined by the ability to validate messages. WCF uses a variant of 'validation by projection' (ignore unknown) for forwards compatibility, but also supports schema wildcards.
As Nicolai M. Josuttis shows in the book SOA in Practice (chapter 12.2.1 Trivial Domain-Driven Versioning), even simple backwards compatible changes might cause unpredicted side effects such as response times breaking SLAs and causing problems for consumers. It is much safer to provide a new service version with the new schema version, as if there is a problem, only the upgraded consumers that required the change will be affected.
Note that even adding backwards compatible schema components can be risky, but adding is typically safe. Josuttis recommends using "Strict" as it is a very simple and explicit policy, but I prefer "Flexible/Strict" as this gives more flexibility and less service versions to govern.
Avoid trying to implement some smart automagical mechanism for handling schema version issues in the service logic. Rather use backwards compatibility, explicit schema versions and support multiple active service versions. In addition, consider applying a service virtualization mechanism.
I recommend using a “Flexible/Strict” compatibility policy:
- Flexible: Safe changes to schemas are backwards compatible and cause just a point version
- Strict: All unsafe schema changes must cause a new schema version and thus a new service version
- Do not require forwards compatible schemas (Loose, wildcard schemas) - schemas should be designed for extensibility, not to avoid versioning
- Service interfaces should also have a Flexible/Strict policy
Note that forwards compatible schemas is not required to have forwards compatible services, as service compatibility is defined by the ability to validate messages. WCF uses a variant of 'validation by projection' (ignore unknown) for forwards compatibility, but also supports schema wildcards.
As Nicolai M. Josuttis shows in the book SOA in Practice (chapter 12.2.1 Trivial Domain-Driven Versioning), even simple backwards compatible changes might cause unpredicted side effects such as response times breaking SLAs and causing problems for consumers. It is much safer to provide a new service version with the new schema version, as if there is a problem, only the upgraded consumers that required the change will be affected.
Note that even adding backwards compatible schema components can be risky, but adding is typically safe. Josuttis recommends using "Strict" as it is a very simple and explicit policy, but I prefer "Flexible/Strict" as this gives more flexibility and less service versions to govern.
Avoid trying to implement some smart automagical mechanism for handling schema version issues in the service logic. Rather use backwards compatibility, explicit schema versions and support multiple active service versions. In addition, consider applying a service virtualization mechanism.
Thursday, October 02, 2008
Versions and Schema Semantics in SOA Composite Services
As part of your SOA governance efforts, it is not sufficient to handle just the versioning of the services and the data schemas used by the services. You must also govern the semantics of the data to enable service composability. Composite services are not possible without semantic data integration.
There are two information models in play when composing services - just as there are two different, but related, process models involved in service-oriented solutions. One is the well known common information model (CIM), the other is the related business event message model used to enable semantic business process composition.
The process composition information should be partitioned according to the business process domains to allow for the business process information model (BPIM) to evolve independent of each other. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain. Your BPIM must be federated even if evolved and versioned separately, to enable semantic data mediation and semantic business process integration.
The BPIM contains metadata that models business event messages as projections of CIM data entities. A business process message typically contains a subset of one or more domain objects. E.g. in an order handling system, the real-world event "customer submits order" is a projection of the customer, address, credit and product entities from the CIM. Think classical paper-based mail order schemas or the Excel schemas you use to get your travel expences reimbursed, or rather the form used to apply for a vacation.
Creating an enterprise canonical data model might be feasible, but federated domain models are recommended. You would anyway need to mediate semantics between your system and third-party services that you involve, or on parts of your processes that you chose to outsource.Federated models are required for B2x integrations as you cannot easily enforce your model on the outside world. Using a CDM might work in a centrally controlled EAI hub, but most likely not across departments, organizational and partner boundaries in an extended enterprise.
There are two information models in play when composing services - just as there are two different, but related, process models involved in service-oriented solutions. One is the well known common information model (CIM), the other is the related business event message model used to enable semantic business process composition.
The process composition information should be partitioned according to the business process domains to allow for the business process information model (BPIM) to evolve independent of each other. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain. Your BPIM must be federated even if evolved and versioned separately, to enable semantic data mediation and semantic business process integration.
The BPIM contains metadata that models business event messages as projections of CIM data entities. A business process message typically contains a subset of one or more domain objects. E.g. in an order handling system, the real-world event "customer submits order" is a projection of the customer, address, credit and product entities from the CIM. Think classical paper-based mail order schemas or the Excel schemas you use to get your travel expences reimbursed, or rather the form used to apply for a vacation.
Creating an enterprise canonical data model might be feasible, but federated domain models are recommended. You would anyway need to mediate semantics between your system and third-party services that you involve, or on parts of your processes that you chose to outsource.Federated models are required for B2x integrations as you cannot easily enforce your model on the outside world. Using a CDM might work in a centrally controlled EAI hub, but most likely not across departments, organizational and partner boundaries in an extended enterprise.
Tuesday, September 30, 2008
Service Versions: Active, Published, Discoverable
As part of your SOA governance efforts, you should have a process for service lifecycle management. Many already use the policy that the three latest major versions of a service is supported, but at different endpoints due to the lack of service virtualization.
As using service virtualization becomes more common for doing lifecycle management, you need to categorize your services into lifecycle stages. Typical stages are
Existing consumers of deprecated versions should as soon as possible move to the a newer discoverable version (see figure), preferably the published version. Note how even if there is only one published version, the active major versions can still be discovered through a service registry.
As using service virtualization becomes more common for doing lifecycle management, you need to categorize your services into lifecycle stages. Typical stages are
- Provisioning
- Active - Published
- Active - Deprecated
- Decommissioned
Existing consumers of deprecated versions should as soon as possible move to the a newer discoverable version (see figure), preferably the published version. Note how even if there is only one published version, the active major versions can still be discovered through a service registry.
These service versioning lifecycle policies are illustrated in the above figure. Note how only the latest major version should allow multiple minor versions to be active. A recommended practice is to have max two such active minor versions. Still, only the latest version of the service should be discoverable.
It is important to minimize the number of active and discoverable versions of the services. This makes it easier to manage and communicate with the service consumers the force them to move on before their version becomes decommissioned. The virtualized endpoints must be monitored, so that you know how much a deprecated service is used - and who those consumers are.
Tuesday, September 23, 2008
Information Models: Business Process or Forms-over-Data
Joe McKendrick has a post about the need for doing data governance today:
If data governance is inadequate — information is outdated, out of sync, duplicated, or plain inaccurate — SOA-enabled services and applications will be delivering garbage. That’s a formula for SOA disaster.
He links to the XML to the rescue: Data governance in SOA article by Ed Tittel, which describes how having a common model and doing data governance is imperative for SOA success. The article conveys the same concepts as chapter 4 'XML: The Foundation for Business Data Integration' in David Chappell's seminal book Enterprise Service Bus. David describes the need for having a common XML data format for "expressing data in messages as it flows through an enterprise across the ESB". This common XML format is a specialization for the common data model (CDM) pattern from the classic book Enterprise Integration Patterns by Gregor Hohpe/Bobby Woolf.
I've written many times about the importance for having a common information model (CIM) for the domain objects (business entities) that your data services operates on, and the need for applying master data management (MDM) on those data. David Linthicum and I agreed on this last year and still do.
What I've written even more often about, and what most canonical models do not encompass; is that the data expressed in the messages conveying business events is not simply CIM entities, but are rather projections of one or more of the domain objects. In addition, the messages covers more than just activity data, they also contain queries, notifications (events) and commands.
E.g. an insurance clerk triggers a business process by registering the real world event "customer has moved" using a projection of customer and address data, not the complete customer aggregate. This action event message triggers a recalculation of the insurance premium due to the relocation of the customer, in addition to updating the customer record - i.e. an underlying business process is executed.
In a traditional forms-over-data application, the whole customer aggregate would be used - but this style would not make for services that can be easily composed to support business process management efforts. The business event data requirements must be simple enough to make composite services possible, including semantic data mediation and integration. In addition, process tailored data make it easier for any involved human operator that have to enter or act on the data. After all, BPM is not all about automation, and human tasks will be central in driving the processes for quite a while.
We need to move to an information model that allows for semantic business process integration, not just semantic business data integration. That is why a business process information model (BPIM) is required, modeling the messages for the action, query and notification events that drive the business processes. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain.
The BPIM is based on the CIM as each service domain model is just a projection of the common data model, and the projection metadata should be goverened as any other SOA artifact.
If data governance is inadequate — information is outdated, out of sync, duplicated, or plain inaccurate — SOA-enabled services and applications will be delivering garbage. That’s a formula for SOA disaster.
He links to the XML to the rescue: Data governance in SOA article by Ed Tittel, which describes how having a common model and doing data governance is imperative for SOA success. The article conveys the same concepts as chapter 4 'XML: The Foundation for Business Data Integration' in David Chappell's seminal book Enterprise Service Bus. David describes the need for having a common XML data format for "expressing data in messages as it flows through an enterprise across the ESB". This common XML format is a specialization for the common data model (CDM) pattern from the classic book Enterprise Integration Patterns by Gregor Hohpe/Bobby Woolf.
I've written many times about the importance for having a common information model (CIM) for the domain objects (business entities) that your data services operates on, and the need for applying master data management (MDM) on those data. David Linthicum and I agreed on this last year and still do.
What I've written even more often about, and what most canonical models do not encompass; is that the data expressed in the messages conveying business events is not simply CIM entities, but are rather projections of one or more of the domain objects. In addition, the messages covers more than just activity data, they also contain queries, notifications (events) and commands.
E.g. an insurance clerk triggers a business process by registering the real world event "customer has moved" using a projection of customer and address data, not the complete customer aggregate. This action event message triggers a recalculation of the insurance premium due to the relocation of the customer, in addition to updating the customer record - i.e. an underlying business process is executed.
In a traditional forms-over-data application, the whole customer aggregate would be used - but this style would not make for services that can be easily composed to support business process management efforts. The business event data requirements must be simple enough to make composite services possible, including semantic data mediation and integration. In addition, process tailored data make it easier for any involved human operator that have to enter or act on the data. After all, BPM is not all about automation, and human tasks will be central in driving the processes for quite a while.
We need to move to an information model that allows for semantic business process integration, not just semantic business data integration. That is why a business process information model (BPIM) is required, modeling the messages for the action, query and notification events that drive the business processes. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain.
The BPIM is based on the CIM as each service domain model is just a projection of the common data model, and the projection metadata should be goverened as any other SOA artifact.
Sunday, September 07, 2008
WS* Oriented Architecture
If a company asks for a service-oriented approach for exposing business functions to external consumers using web-services, but has no explicit requirements in the RFP for service virtualization or coordinated loose coupling, and neither any requirements for service and data contract versioning mechanisms; is it then sufficient to design a solution based on web-services adhering to the WS* standards, W3C MEPs and WS-I BasicProfile, plus a service registry; without any of the aforementioned mechanisms, and call it SOA ?
I think not. Adding a repository and applying a process for service lifecycle management will help, but you risk getting a just a glorified, governed JBOWS system. Then again, that's a start.
I think not. Adding a repository and applying a process for service lifecycle management will help, but you risk getting a just a glorified, governed JBOWS system. Then again, that's a start.
Saturday, September 06, 2008
How "Oslo" relates to BizTalk
The Microsoft "Oslo" initiative is to provide a modeling (UML), management, and hosting platform for delivering model-driven service-oriented composite applications and S+S solutions; plus cloud computing through modeling.
The recently published BizTalk 2009 roadmap shows how "Oslo" features can be utilized also from BizTalk:
In fact, you won’t need to upgrade BizTalk Server to take advantage of "Oslo" – current BizTalk Server 2006 R2 or BizTalk Server 2009 customers can benefit from "Oslo" by being able to leverage and compose existing services into new composite applications. BizTalk Server today provides the ability to service enable LOB systems or trading partners as web services (using WCF supported protocols), which can be composed with the "Oslo" modeling technologies.
As the roadmap shows, WCF is the central enabler for connecting services from different systems into service-oriented solutions. WCF and WF are central to the new "Oslo" modeling tool and repository, allowing users to define and execute business processes (much like BPEL, XLANG). "Oslo" has a strong focus on enabling automation of business processes with strong support for humans as central actors in the processes (like BPEL4People, WS-HumanTask). Add the BAM interceptors for WCF and WF provided by BizTalk, and you have a platform that gives repeatability, consistency and visibility to your business process management efforts.
The recently published BizTalk 2009 roadmap shows how "Oslo" features can be utilized also from BizTalk:
In fact, you won’t need to upgrade BizTalk Server to take advantage of "Oslo" – current BizTalk Server 2006 R2 or BizTalk Server 2009 customers can benefit from "Oslo" by being able to leverage and compose existing services into new composite applications. BizTalk Server today provides the ability to service enable LOB systems or trading partners as web services (using WCF supported protocols), which can be composed with the "Oslo" modeling technologies.
As the roadmap shows, WCF is the central enabler for connecting services from different systems into service-oriented solutions. WCF and WF are central to the new "Oslo" modeling tool and repository, allowing users to define and execute business processes (much like BPEL, XLANG). "Oslo" has a strong focus on enabling automation of business processes with strong support for humans as central actors in the processes (like BPEL4People, WS-HumanTask). Add the BAM interceptors for WCF and WF provided by BizTalk, and you have a platform that gives repeatability, consistency and visibility to your business process management efforts.
Monday, August 11, 2008
SOA Information Models and Services Compositions
I've written quite a lot about having a model for the information needed for semantic business process composition in service-oriented solutions, two of the most read and commented are SOA Canonical "Data" Model and Ontology for Business Processes, SOA and Information Models.
The process composition information model is focused on the business events, messages and documents and the semantics of the information that the model encompasses. The model is targeted at the orchestrations level in the ontology, not at the services level; as shown in the figure (click to enlarge):
The service classification scheme has the "hierarchy" taxomomy form. It is not a scientific hierarchy nor a tree form, thus there is no "is using" or "is dependent on" relation between the services - it is just a classification scheme. There can be no dependency between services in SOA except for composite services realizing a business process by consuming the actual services. Taking a dependency on another service will break the central "services are autonomous" tenet of SOA.
A note on the splitting of the service categories: this is a pure metadata ontology based on the taxonomy for categorizing services, it is not a split based on technolog or, teams, and surely not on layers. A service should not call on other services, making a service dependent on another service will lead to repeating the failure of DCOM/CORBA in distributed systems; using web-services and calling it SOA solves nothing. Use the active service pattern or EDA to ensure that your services are truly autonomous and independent of the availability of other services. Isolate service dependency to the composite services, or even better to the consumer mashups, only.
The "business process information model" (BPIM) is what models the business events and documents in the composite services level (orchestration / conversations / sagas). I like to use a domain model as it by definition focuses on the concepts of the problem domain (the business process information artifacts) rather than just the data of the domain. The information model must comprise the different message types for the action, query and notification events that drive the business processes. The model must be canonical for its domain to ensure that the format and semantics of the model is consitent within the domain.
There is some confusion on the difference between the process composition BPIM and the EAI "canonical data model" (CDM), plus the similar concept Common Information Model (CIM). As can be seen from the comments on the two postings and some blog reactions like SOA doesn’t need a Common Information Model, this has made it harder to convey the difference between the process composition information model and the more data-oriented models.
A CIM should be used for the resources data model - which the BPIM is related to. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain. The data in the event document for the real-life action "CustomerHasMoved" is not a complete "Customer" resource entity, it is a message including minimal "AddressChange" data. This data will typically contain both actual event data combined with reference keys to the resources such as customers and addresses in the different service domains that the composition comprises.
The message data is of course associated with the CIM of the underlying resource services. The BPIM message types are projections of CIM objects; in fact, they are projected compositions of the referenced resources - not just simple compositions of resource objects. These projections must be kept as small as possible, preferably just the resource keys, in addition to event details such as a booking reference number. Keeping the event document data model small has the nice side-effect of making the schema versioning a bit simpler.
The CIM should be the starting point of your modeling efforts, as the documents in the process compositions is based on the service domains. Having Business Process Modeling Notation (BPMN) diagrams can help in modeling, as BPMN encompass process, events, messages and business documents, covering actions, queries and notifications. Other process modeling diagrams can be used, just ensure that the events, messages and data are captured in the model.
It is still important to recognize that having a "one true schema" for process composition or SOA in general is not a recommended practice, hence the term "domain" in the naming. Rather than trying to enforce a common model across all service domains, federated models should be used. When composing services across the domains, mediation and transformation of messages will be required - here a service bus with such capabilities would come in handy.
A thing to recgonize is that the overall model incorporates several "flows": process flows, event flows, message flows, and data state flows (resource lifecycles). Typically, only the process flow is clearly shown in diagrams, while e.g. message flow is only represented by receive and send ports. See this interview with Gregor Hohpe on Conversation Patterns about how more than just the process need to be modeled.
For an excellent primer on architectural aspects of SOA and the proliferation of acronyms, I recommend reading Architecture requirements for Service-Oriented Business Applications by Johan den Haan. Note that he does the classical simplification of focusing on business processes without including a BPIM; it is important to separate the process composition model from the resource model, just as business processes are separated from resource lifecycle processes. The business process events, documents and messages are not just collaterals of the BPMN modeling process, they are important artifacts for semantic mediation in composite services.
[Business Process Integration: "e-Business: organizational and technical foundations", Papazoglou, M. P. & Ribbers, P. (2006)]
The process composition information model is focused on the business events, messages and documents and the semantics of the information that the model encompasses. The model is targeted at the orchestrations level in the ontology, not at the services level; as shown in the figure (click to enlarge):
The service classification scheme has the "hierarchy" taxomomy form. It is not a scientific hierarchy nor a tree form, thus there is no "is using" or "is dependent on" relation between the services - it is just a classification scheme. There can be no dependency between services in SOA except for composite services realizing a business process by consuming the actual services. Taking a dependency on another service will break the central "services are autonomous" tenet of SOA.
A note on the splitting of the service categories: this is a pure metadata ontology based on the taxonomy for categorizing services, it is not a split based on technolog or, teams, and surely not on layers. A service should not call on other services, making a service dependent on another service will lead to repeating the failure of DCOM/CORBA in distributed systems; using web-services and calling it SOA solves nothing. Use the active service pattern or EDA to ensure that your services are truly autonomous and independent of the availability of other services. Isolate service dependency to the composite services, or even better to the consumer mashups, only.
The "business process information model" (BPIM) is what models the business events and documents in the composite services level (orchestration / conversations / sagas). I like to use a domain model as it by definition focuses on the concepts of the problem domain (the business process information artifacts) rather than just the data of the domain. The information model must comprise the different message types for the action, query and notification events that drive the business processes. The model must be canonical for its domain to ensure that the format and semantics of the model is consitent within the domain.
There is some confusion on the difference between the process composition BPIM and the EAI "canonical data model" (CDM), plus the similar concept Common Information Model (CIM). As can be seen from the comments on the two postings and some blog reactions like SOA doesn’t need a Common Information Model, this has made it harder to convey the difference between the process composition information model and the more data-oriented models.
A CIM should be used for the resources data model - which the BPIM is related to. Design the BPIM based on the CIM, ensuring that the model is canonical for each process domain. The data in the event document for the real-life action "CustomerHasMoved" is not a complete "Customer" resource entity, it is a message including minimal "AddressChange" data. This data will typically contain both actual event data combined with reference keys to the resources such as customers and addresses in the different service domains that the composition comprises.
The message data is of course associated with the CIM of the underlying resource services. The BPIM message types are projections of CIM objects; in fact, they are projected compositions of the referenced resources - not just simple compositions of resource objects. These projections must be kept as small as possible, preferably just the resource keys, in addition to event details such as a booking reference number. Keeping the event document data model small has the nice side-effect of making the schema versioning a bit simpler.
The CIM should be the starting point of your modeling efforts, as the documents in the process compositions is based on the service domains. Having Business Process Modeling Notation (BPMN) diagrams can help in modeling, as BPMN encompass process, events, messages and business documents, covering actions, queries and notifications. Other process modeling diagrams can be used, just ensure that the events, messages and data are captured in the model.
It is still important to recognize that having a "one true schema" for process composition or SOA in general is not a recommended practice, hence the term "domain" in the naming. Rather than trying to enforce a common model across all service domains, federated models should be used. When composing services across the domains, mediation and transformation of messages will be required - here a service bus with such capabilities would come in handy.
A thing to recgonize is that the overall model incorporates several "flows": process flows, event flows, message flows, and data state flows (resource lifecycles). Typically, only the process flow is clearly shown in diagrams, while e.g. message flow is only represented by receive and send ports. See this interview with Gregor Hohpe on Conversation Patterns about how more than just the process need to be modeled.
For an excellent primer on architectural aspects of SOA and the proliferation of acronyms, I recommend reading Architecture requirements for Service-Oriented Business Applications by Johan den Haan. Note that he does the classical simplification of focusing on business processes without including a BPIM; it is important to separate the process composition model from the resource model, just as business processes are separated from resource lifecycle processes. The business process events, documents and messages are not just collaterals of the BPMN modeling process, they are important artifacts for semantic mediation in composite services.
[Business Process Integration: "e-Business: organizational and technical foundations", Papazoglou, M. P. & Ribbers, P. (2006)]
Wednesday, August 06, 2008
Service Virtualization: MSE on Channel9
Service virtualization can be an important architectual mechanism for your service-oriented solutions, and the Managed Services Engine is a free WCF-based tool available at CodePlex.
There are two videos at Channel9 about MSE that I recommend watching to learn about the capabilities of MSE:
There are two videos at Channel9 about MSE that I recommend watching to learn about the capabilities of MSE:
- Code to live: service virtualization, versioning, etc
- Talking about MSE: virtual services and endpoints, protocol adaption, policy enforcement, etc
Monday, August 04, 2008
How "Oslo" relates to SOA
The upcoming Microsoft "Oslo" offering might be a bit vague to grasp, which I can readily understand. It has been perceived as a SOA implementation toolkit, but is rather focused on being a modeling (UML), management, and hosting platform for delivering model-driven service-oriented composite applications and S+S solutions; plus cloud computing through modeling. Read Loraine Lawson's Best Explanation of Oslo So Far for a rather typical experience.
Also look at the Oslo PDC sessions posted by Matt Winkler, on how "Oslo" is a service repository and a platform for both lifecycle management and hosting of services and processes. The service and process implementation tools in "Oslo" will be Visual Studio, WF, WCF, BizTalk, Zermatt, etc - and "Oslo" will be the one initiative to rule them all.
Directions on Microsoft on "Oslo": Messaging, Workflow Roadmap Announced (PDF)
Also look at the Oslo PDC sessions posted by Matt Winkler, on how "Oslo" is a service repository and a platform for both lifecycle management and hosting of services and processes. The service and process implementation tools in "Oslo" will be Visual Studio, WF, WCF, BizTalk, Zermatt, etc - and "Oslo" will be the one initiative to rule them all.
Directions on Microsoft on "Oslo": Messaging, Workflow Roadmap Announced (PDF)
Subscribe to:
Posts (Atom)