Today was my last day as an MSFT FTE, tomorrow I'm co-founder and partner at Puzzlepart - as the second person in the company after my old colleague Mads Nissen.
At Puzzlepart we will continue our joint SharePoint adventure. Together we have delivered quite a few SharePoint and MSCRM solutions, such as this case study shown by Steve Ballmer himself at the 2005 business summit. As you can see from the video, we focus on delivering business value - and we're working on our design skills.
As they say at my old employer: I'm super excited to be here! And to my now former colleagues at MCS: Thank you for all that you do!
Monday, August 31, 2009
Monday, August 17, 2009
DDD Lessons Learned: CQRS, Fat Domain Model, Concrete Aggregate Entity
It's been five years since Eric Evans' classic book Domain-driven design: tackling complexity in the heart of software was published. Last week two interesting articles came out; What I've learned about DDD since the book by Eric himself and Domain Models: Employing the Domain Model Pattern by Udi Dahan. They both talk of lessons learnt about some of the concepts of domain-driven design (DDD).
Udi talks about a very common misconception about domain Entity objects, what I like to call the Fat Domain Model: using the persistent object model as the Domain Model. I've done this myself for many years, trying to avoid Martin Fowler's Anemic Domain Model and Fat Service Layer anti-patterns by adding the domain entity's behavior to the related data entity object.
Microsoft does a variant of the Fat Service Layer in their Three-Layered Services Application guidance: they use anemic Business Entity objects and keeps the business logic in separate Business Component objects, exposed by Service Interface objects. So coming from this, it seemed natural to merge the entity and logic into one when I started practicing DDD.
The description of the Entity pattern from Eric's book is clear on what goes into the entity object:
"It is natural to think about the attributes when modeling an object, and it is quite important to think about its behavior. [...] strip the Entity object's definition down to the most intrinsic characteristics [...]. Add only behavior that is essential to the [domain] concept and attributes that are required by that behavior."
So, only the data that is required by the domain logic should be in the domain object. Eric use the term "information about the business situation" in the book when explaining the Domain Layer, another hint that this is not traditional data entities. Querying and viewing entities is not domain logic, so there is no need to add all attributes of the ORM data entities used in the presentation layer to the DDD Entity object. Querying and viewing entities belongs to the Service Layer. I suggest using two separate object models (Command and Query Responsibility Segregation, CQRS) as shown in Udi's article, but think twice before implementing your forms-over-data application using the Domain Model pattern.
In his QCon 2009 presentation, Eric talks about the DDD Aggregate pattern and how it can be hard to model an Aggregate Boundary and decide to which Entity object a specific behavior belongs. Here is the definition of Aggregate from the book:
"First we need an abstraction for encapsulating references within the [domain] model. An Aggregate is a cluster of associated [domain] objects that we treat as a unit for the purpose of [ensuring consistency of] data changes. Each Aggregate has a root and a boundary. The root is a single, specific Entity contained in the Aggregate."
I think the Aggregate pattern is easy to misunderstand as it focuses on modeling the relationships and boundary of the root and its associated collection of domain objects - immediately making people think of the classical "purchase order with items collection" OO data encapsulation design or even ER-design. In addition, the strong focus the Aggregate being an abstract consistency mechanism leads to the misconception that Aggregate domain logic must belong to the root Entity object and that logic that involves associated Aggregate data will require that data to be modeled into the collection of domain objects.
Eric uses the example of calculating the weight of a grape cluster Aggregate. This logic doesn't belong on the grape Entity, so the only place to put this is on the stem Entity - because the Aggregate itself is not an object, just an abstract concept. He also talks about how this kind of logic typically is designed as an iteration of the associated object collection, while lessons learnt has shown that this OO legacy mindset is not mandated by the domain business rules. Udi also touches the same problem in the "UnpaidOrdersAmount" example in his domain model article.
The fact is that the domain aggregate is not just an abstract concept, it should be modeled as what I like to call a Concrete Aggregate. This alleviates the root Entity from getting responsibilities that doesn't naturally belong to it. This kind of non-entity domain logic goes into the Concrete Aggregate as its behavior and prevents leaking domain logic into the Service Layer.
Using Concrete Aggregate objects might also lessen the need for fully modeling objects within Aggregate boundaries. E.g. having a GrapeCluster object that can calculate its weight directly without loading a Stem and its collection of Grape objects makes the domain model much simpler. Again note the focus on business rules behavior rather than OO entity-relationship modeling.
I admit struggling with naming the Concrete Aggregate pattern, maybe Aggregate Entity would be a more self-explanatory name.
Udi talks about a very common misconception about domain Entity objects, what I like to call the Fat Domain Model: using the persistent object model as the Domain Model. I've done this myself for many years, trying to avoid Martin Fowler's Anemic Domain Model and Fat Service Layer anti-patterns by adding the domain entity's behavior to the related data entity object.
Microsoft does a variant of the Fat Service Layer in their Three-Layered Services Application guidance: they use anemic Business Entity objects and keeps the business logic in separate Business Component objects, exposed by Service Interface objects. So coming from this, it seemed natural to merge the entity and logic into one when I started practicing DDD.
The description of the Entity pattern from Eric's book is clear on what goes into the entity object:
"It is natural to think about the attributes when modeling an object, and it is quite important to think about its behavior. [...] strip the Entity object's definition down to the most intrinsic characteristics [...]. Add only behavior that is essential to the [domain] concept and attributes that are required by that behavior."
So, only the data that is required by the domain logic should be in the domain object. Eric use the term "information about the business situation" in the book when explaining the Domain Layer, another hint that this is not traditional data entities. Querying and viewing entities is not domain logic, so there is no need to add all attributes of the ORM data entities used in the presentation layer to the DDD Entity object. Querying and viewing entities belongs to the Service Layer. I suggest using two separate object models (Command and Query Responsibility Segregation, CQRS) as shown in Udi's article, but think twice before implementing your forms-over-data application using the Domain Model pattern.
In his QCon 2009 presentation, Eric talks about the DDD Aggregate pattern and how it can be hard to model an Aggregate Boundary and decide to which Entity object a specific behavior belongs. Here is the definition of Aggregate from the book:
"First we need an abstraction for encapsulating references within the [domain] model. An Aggregate is a cluster of associated [domain] objects that we treat as a unit for the purpose of [ensuring consistency of] data changes. Each Aggregate has a root and a boundary. The root is a single, specific Entity contained in the Aggregate."
I think the Aggregate pattern is easy to misunderstand as it focuses on modeling the relationships and boundary of the root and its associated collection of domain objects - immediately making people think of the classical "purchase order with items collection" OO data encapsulation design or even ER-design. In addition, the strong focus the Aggregate being an abstract consistency mechanism leads to the misconception that Aggregate domain logic must belong to the root Entity object and that logic that involves associated Aggregate data will require that data to be modeled into the collection of domain objects.
Eric uses the example of calculating the weight of a grape cluster Aggregate. This logic doesn't belong on the grape Entity, so the only place to put this is on the stem Entity - because the Aggregate itself is not an object, just an abstract concept. He also talks about how this kind of logic typically is designed as an iteration of the associated object collection, while lessons learnt has shown that this OO legacy mindset is not mandated by the domain business rules. Udi also touches the same problem in the "UnpaidOrdersAmount" example in his domain model article.
The fact is that the domain aggregate is not just an abstract concept, it should be modeled as what I like to call a Concrete Aggregate. This alleviates the root Entity from getting responsibilities that doesn't naturally belong to it. This kind of non-entity domain logic goes into the Concrete Aggregate as its behavior and prevents leaking domain logic into the Service Layer.
Using Concrete Aggregate objects might also lessen the need for fully modeling objects within Aggregate boundaries. E.g. having a GrapeCluster object that can calculate its weight directly without loading a Stem and its collection of Grape objects makes the domain model much simpler. Again note the focus on business rules behavior rather than OO entity-relationship modeling.
I admit struggling with naming the Concrete Aggregate pattern, maybe Aggregate Entity would be a more self-explanatory name.
Thursday, July 02, 2009
List Content Type Fields & Forms: CAML vs Code
In a feature based site definition I recently made, I had a site content type hierarchy with a base content type and two child content types defined in a feature. The child types inherits their site columns and display/edit/new form from their parent.
Another dependent feature contained a list definition and a provisioned list instance that would reference the two child site content types and thus snapshot inherit them as list content types.
The columns of the list are based on the inherited site columns, in reality a snapshot copy of them at the time the list was created. All provisioning of site columns, site content type, list definitions, list instances and list content types are defined in CAML in the features.
As you can see from the above figures, everything looked fine and dandy. Until I tried to create a new list item: all the custom fields where missing in the new item form. The same problem applied to the display and edit forms also.
Using the "List settings" UI to remove and then add the list content types manually proved to fix the problem, so it had to be an issue related to the way CAML <MetaData/ ContentTypes/ ContentTypeRef> makes the snapshot copy.
Knowing that the fields of a form are rendered using the <ListFieldIterator>, it was time to check the field collection of the list content types using SharePoint Manager 2007.
In the above figure site content types are to the left and list content types to the right. The "News article" content type added through the "List settings" UI has inherited a full snapshot, but the "Stand-alone article" added by CAML has inherited just the ootb BaseType "Item" fields. So the <RenderingTemplate> for NewForm finds only two fields to render. And the actual problem is in the CAML inheritance.
The problem is that the list columns do not inherit the site columns correctly. In fact, the only official way of making <ContentTypeRef> work correctly using CAML is to repeat the site column definition inside the list definition <Fields> element. This is not a good approach with regards to maintenance of your solution.
There is a similar problem with list content types related to site content types, which is easily rectified by forcing a ContentType Update(push down changes) during provisioning of the site content type feature (see exercise 13-5 in the Building the SharePoint User Experience book).
The list feature is activated after the site content type feature. You might think that you could force another site content type update to rectify the inheritance again. That isn't very feasible, as updating child content types depends on a having actual changes to push down; and there is none at this time - remember that the site content types have already been provisioned and pushed down. There are some "best" practices out there on Custom Fields in Schema.xml, but please read on before trying those.
Just adding the list content types from code will create correct snapshots with no fuzz:
"When you add a site content type to a list using the object model, Windows SharePoint Services automatically adds any columns that content type contains that are not already on the list. This is in contrast to provisioning a list schema with content types, in which case you must explicitly add the columns to the list schema for Windows SharePoint Services to provision them."
So instead of doing this using CAML, just add them when activating the list feature. The newly created list instance is empty, so the content types can easily be added:
public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
using (SPWeb web = (SPWeb)properties.Feature.Parent)
{
//enable management of content types
SPList list = web.Lists["News"];
list.ContentTypesEnabled = true;
list.Update();
//inherit the list content types from site content types
list.ContentTypes.Add(web.ContentTypes["Stand-alone article"]);
list.ContentTypes.Add(web.ContentTypes["Series article"]);
list.Update();
//remove item as list content type
list.ContentTypes["Item"].Delete();
list.Update();
}
}
NOTE: This will only work for list instances created by the feature. The code will not run for list instances added later on - and there is no ListAdding/ListAdded events in WSS3.0/MOSS. For lists added manually, the list content types must also be added manually. Or create an "associate [my] list content types" feature that you can re-activate at will to bind the content types to lists as applicable. Use a marker content type to look for to avoid hardcoding the list names. Some like to use a timer job to bind the list content types.
The code gets a reference to the list instance, turns on content type management / content types on the new menu, adds the list content types as new snapshot copies, and finally removes a dummy "Item" content type (that is there to make the CAML list definition valid).
My changed list definition Schema.xml looks like this:
As you can see, I have commented out the CAML <ContentTypeRef> for my two child site content types, and instead just referenced the standard "Item" content type - the dummy that is removed during feature activation. If you still would like to have those <ContentTypeRef> in the Schema.xml, then remove and re-add them from the code.
Note how you still can add your custom fields as <ViewFields> even if their content type is not referenced. Content type management for the list can also be turned on using the EnableContentTypes attribute on the root <List> element in Schema.xml.
That's it. I deleted the list and reactivated the list feature - and voila: the new, edit and display forms contains all fields of the content type selected from the new menu.
[UPDATE] SharePoint 2010 adds the Inherits attribute that makes CAML inheritance work as expected, and using FieldRefs unnecessary. See also the new 2010 feature upgrade mechanism UpgradeActions AddContentTypeField element with the PushDown attribute that force changes to a site content type and all derived types.
Another dependent feature contained a list definition and a provisioned list instance that would reference the two child site content types and thus snapshot inherit them as list content types.
As you can see from the above figures, everything looked fine and dandy. Until I tried to create a new list item: all the custom fields where missing in the new item form. The same problem applied to the display and edit forms also.
Using the "List settings" UI to remove and then add the list content types manually proved to fix the problem, so it had to be an issue related to the way CAML <MetaData/ ContentTypes/ ContentTypeRef> makes the snapshot copy.
Knowing that the fields of a form are rendered using the <ListFieldIterator>, it was time to check the field collection of the list content types using SharePoint Manager 2007.
In the above figure site content types are to the left and list content types to the right. The "News article" content type added through the "List settings" UI has inherited a full snapshot, but the "Stand-alone article" added by CAML has inherited just the ootb BaseType "Item" fields. So the <RenderingTemplate> for NewForm finds only two fields to render. And the actual problem is in the CAML inheritance.
The problem is that the list columns do not inherit the site columns correctly. In fact, the only official way of making <ContentTypeRef> work correctly using CAML is to repeat the site column definition inside the list definition <Fields> element. This is not a good approach with regards to maintenance of your solution.
There is a similar problem with list content types related to site content types, which is easily rectified by forcing a ContentType Update(push down changes) during provisioning of the site content type feature (see exercise 13-5 in the Building the SharePoint User Experience book).
The list feature is activated after the site content type feature. You might think that you could force another site content type update to rectify the inheritance again. That isn't very feasible, as updating child content types depends on a having actual changes to push down; and there is none at this time - remember that the site content types have already been provisioned and pushed down. There are some "best" practices out there on Custom Fields in Schema.xml, but please read on before trying those.
Just adding the list content types from code will create correct snapshots with no fuzz:
"When you add a site content type to a list using the object model, Windows SharePoint Services automatically adds any columns that content type contains that are not already on the list. This is in contrast to provisioning a list schema with content types, in which case you must explicitly add the columns to the list schema for Windows SharePoint Services to provision them."
So instead of doing this using CAML, just add them when activating the list feature. The newly created list instance is empty, so the content types can easily be added:
public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
using (SPWeb web = (SPWeb)properties.Feature.Parent)
{
//enable management of content types
SPList list = web.Lists["News"];
list.ContentTypesEnabled = true;
list.Update();
//inherit the list content types from site content types
list.ContentTypes.Add(web.ContentTypes["Stand-alone article"]);
list.ContentTypes.Add(web.ContentTypes["Series article"]);
list.Update();
//remove item as list content type
list.ContentTypes["Item"].Delete();
list.Update();
}
}
NOTE: This will only work for list instances created by the feature. The code will not run for list instances added later on - and there is no ListAdding/ListAdded events in WSS3.0/MOSS. For lists added manually, the list content types must also be added manually. Or create an "associate [my] list content types" feature that you can re-activate at will to bind the content types to lists as applicable. Use a marker content type to look for to avoid hardcoding the list names. Some like to use a timer job to bind the list content types.
The code gets a reference to the list instance, turns on content type management / content types on the new menu, adds the list content types as new snapshot copies, and finally removes a dummy "Item" content type (that is there to make the CAML list definition valid).
My changed list definition Schema.xml looks like this:
Note how you still can add your custom fields as <ViewFields> even if their content type is not referenced. Content type management for the list can also be turned on using the EnableContentTypes attribute on the root <List> element in Schema.xml.
That's it. I deleted the list and reactivated the list feature - and voila: the new, edit and display forms contains all fields of the content type selected from the new menu.
[UPDATE] SharePoint 2010 adds the Inherits attribute that makes CAML inheritance work as expected, and using FieldRefs unnecessary. See also the new 2010 feature upgrade mechanism UpgradeActions AddContentTypeField element with the PushDown attribute that force changes to a site content type and all derived types.
Wednesday, July 01, 2009
Review: Building the SharePoint User Experience
Lately I've been reading Building the SharePoint User Experience by fellow Norwegian Bjørn Furuknap, and it is an easy read even if it covers "under the hood" aspects of SharePoint UX such as list definitions, site definitions, content types, custom fields, features, stapling, onet.xml, and the beast itself: CAML. Don't be fooled by the term UX, this book is not about doing SharePoint design using Photoshop or design as in interaction design (master pages, navigation, CSS, etc). In fact, the book covers content types at great length without ever mentioning search and findability at all. This is indeed a hardcore book for SharePoint developers.
I absolutely recommend reading this book. Part 1 and 2 covers the basics of the SharePoint building blocks and a thorough walk-through of these artifacts. The nice thing in part 2 is that the text is based on actually using the stuff, as opposed to the documentation at MSDN which is rather poor when it comes to real life aspects.
What I liked the most is part 3, which is a series of exercises that builds a complete feature based site definition. Doing these exercises will help you really learn the quirks of the covered SharePoint aspects; things that seems easy when reading about it, but that has a lot of gotchas in practice. Don't just read the book, do the exercises! Get the companion eBook (PDF format) to avoid all the mindless typing.
Chapter 15 contains a rather lame way of configuring the site's navigation aspects using code. Your information architect and not least, your interaction designer, will cringe at the tought of such a central aspect being buried inside code instead of designer friendly XML.
Side note on feature based site definitions: have a look on the SharePoint Site Configurator at CodePlex made by the MCS IW-team in Norway.
There are some quirks in the exercises which can cause you some grief. The errata page for the book is rather thin, so here is an errata list from my walkthrough:
Now to the "More on this later" references: In exercise 13-3 you learned that list content types do not correctly relate to child site content types when defined in CAML. In exercise 13-5, this was corrected using code. However, the same problem applies to provisioning list columns, they do not correctly inherit the site columns if not repeated in the list's CAML definition; see the note in How to add a content type to a list.
This typically results in missing fields in new form, edit form, etc. See the comments on the MSDN How to create a custom list definition article for links to several blog posts, but don't blindly follow their advice of adding the fields explicitly to the list Schema.xml.
There is a much simpler solution to the NewForm/EditForm/ViewForm problem by using FeatureActivated code for the list. And that is the topic of my next post.
I absolutely recommend reading this book. Part 1 and 2 covers the basics of the SharePoint building blocks and a thorough walk-through of these artifacts. The nice thing in part 2 is that the text is based on actually using the stuff, as opposed to the documentation at MSDN which is rather poor when it comes to real life aspects.
What I liked the most is part 3, which is a series of exercises that builds a complete feature based site definition. Doing these exercises will help you really learn the quirks of the covered SharePoint aspects; things that seems easy when reading about it, but that has a lot of gotchas in practice. Don't just read the book, do the exercises! Get the companion eBook (PDF format) to avoid all the mindless typing.
Chapter 15 contains a rather lame way of configuring the site's navigation aspects using code. Your information architect and not least, your interaction designer, will cringe at the tought of such a central aspect being buried inside code instead of designer friendly XML.
Side note on feature based site definitions: have a look on the SharePoint Site Configurator at CodePlex made by the MCS IW-team in Norway.
There are some quirks in the exercises which can cause you some grief. The errata page for the book is rather thin, so here is an errata list from my walkthrough:
- Exercise 11-2, step 3: Note the TypeName here, it will bite you in exercise 11-7.
- Exercise 11-7, step 3: Here the @Type value of the site column is not the same as the TypeName in the custom field. This wil cause the general "the given key was not present in the dictionary" CreateSPField error when activating the feature. Use SPM2007 to diagonse the problem by clicking on the site's Fields node, this will tell you that “Field type ‘TimesFieldType’ is not installed properly” as there is no such custom field.
- Chapter 11, page 253: The feature Categories list will not be created until chapter 13, so manually create a Categories custom list for use in chapter 12.
- Exercise 11-9, step 1: The lookup column @List attribute must be removed before running the code in step 2, otherwise you will get the "Cannot change the lookup list of the lookup field" error activating the feature.
- Exercise 11-9, step 2: Out of the blue, a reference to TimesSiteColumns.cs and some FeatureActivated code that does not exist yet. To make this code snippet work for this site scoped feature, you need code from exercise 13-5 to set the "web" C# local member.
- Exercise 11-9, step 2: If you removed both @List and @ShowField in step 1, the you must add this code to set the show field: categoriesField.LookupField = "Title";
- Exercise 12-5, step 8: The 20002 download is empty.
- Exercise 12-6, step 4: You will only get the content type's defined NewForm because you manually added the list content types using the UI, this will not work in e.g. exercise 13-6. More on this later.
- Exercise13-1, step 1: Make this feature web scoped, not site scoped - or you will get an error in exercise 15-2.
- Exercise13-5, step 1: The file name should be TimesContentType.cs.
- Exercise13-5, step 2: The name of the content type shoud be "News Article" (with a space).
- Exercise13-5: The <MetaData> <ContentTypeRef> elements must be in the list template Schema.xml file to add the content types as list content types. In addition, the <List> element's EnableContentTypes attribute should be included and true. Note that this is not sufficient to get a fully working list instance. More on this later.
- Exercise 13-6, step 3: The 20003 download is empty.
- Exercise13-6, step 5: The NewForm of the "News article" child list content types will not show the custom columns added in exercise 13-5 by forcing updated inheritance. Only the standard "Item" fields are shown. More on this later.
- Exercise 14-2, step 2: There cannot be multiple <asp:Content> controls in an .ASPX page that target the same @ContentPlaceHolderID, here "PlaceHolderMain"
- Exercise 15-2, step 1: Using the ExecuteUrl element does not work when creating a site-collection from Central Admin.
- Exercise 15-2, step 2: Refers to a mystery FeatureAdded method and a TimesSetup.cs file, and the code refers to a missing "web" C# local member. Put this code after the code in step 4.
- Exercise 15-3, step 2: Using "/" as the URL will cause the "Cannot open /: no such file or folder" error. This applies to the rest of the navigation exercises.
- Exercise 15-5, step 1: Remember to add the <WebFeatures> <Feature> from the above text to the onet.xml file first.
Now to the "More on this later" references: In exercise 13-3 you learned that list content types do not correctly relate to child site content types when defined in CAML. In exercise 13-5, this was corrected using code. However, the same problem applies to provisioning list columns, they do not correctly inherit the site columns if not repeated in the list's CAML definition; see the note in How to add a content type to a list.
This typically results in missing fields in new form, edit form, etc. See the comments on the MSDN How to create a custom list definition article for links to several blog posts, but don't blindly follow their advice of adding the fields explicitly to the list Schema.xml.
There is a much simpler solution to the NewForm/EditForm/ViewForm problem by using FeatureActivated code for the list. And that is the topic of my next post.
Subscribe to:
Posts (Atom)