Showing posts with label ContentType. Show all posts
Showing posts with label ContentType. Show all posts

Wednesday, October 23, 2013

Roadmap for Responsive Design and Dynamic Content in SharePoint 2013

Responsive design combined with dynamic user-driven content and mobile first seems to be the main focus everywhere these days. The approach outlined in Optimizing SharePoint 2013 websites for mobile devices by Waldek Mastykarz and his How we did it series show how it can be achieved using SharePoint 2013.

But what if you have thousands of pages with good content that you want make resposive? What approach will you use to adapt all those articles after migrating the content over from SharePoint 2010? This post suggests a roadmap for gradually transforming your static content to become dynamic content.

First of all, the metadata of your content types must be classified so that you know the ranking of the metadata within a content type, so that it can be used in combination with device channel panels and resposive web design (RWD) techniques to prioritize what to show on what devices. This will most likely introduce more specialized content types with more granular metadata fields. All your content will need to be edited to fit the new content classification, at least the most important content you have. By edited, I mean that the content type of articles must be changed and the content adapted to the new metadata; in addition, selecting a new content type will also cause a new RWD page layout to be used for the content. These new RWD page layouts and master pages are also something you need to design and implement, as part of your new user experience (UX) concept.

While editing all the (most important) existing content, it is also a good time to ensure that the content gets high quality tagging according to your information architecture (IA), as tagging is the cornerstone of a good, dynamic user experience provided by term-driven navigation and search-driven content. Editing the content is the most important job here, tagging the content is not required to enable RWD for your pages.

Doing all of this at once as part of migrating to SP2013 is by experience too much to be realistic, so this is my recommended roadmap:

Phase 1
- Focus on RWD based on the new content types and their new prioritized metadata and new responsive master pages and page layouts
- Quick win: revise the search center to exploit the new search features, even if tagging is postponed to a later phase (IA: findability and search experience)
- Keep the existing information architecture structure, and thus the navigation as-is
- Keep the page content as-is, do not add search-driven content to the pages yet, focus on making the articles responsive
- Most time-consuming effort: adapting the content type of all articles, cutting and pasting content within each article to fit the new prioritized metadata structure

Phase 2
- Focus on your new concept for structure and navigation in SP2013 (IA: content classification and structure, browse and navigate UX)
- Tagging of the articles according to the new IA-concept for dynamic structuring of the content (IA: term sets for taxonomy)
- Keep the page content as-is, no new search-driven UX in this phase, just term-driven navigation
- Most time-consuming effort: tagging all of your articles, try scripting some auto-tagging based on the existing structure of the content

Phase 3
- Focus on search-driven content in the pages according to the new concept  (IA: discover and explore UX)
- New routines and processes for authors, approvers and publishers based on new SP2013 capabilities (IA: content contributor experience)
- Most time-consuming effort: tune and tag the content of all your articles to drive the ranking of the search-driven content according to the new concept

Phase 4
- Content targeting in the pages based on visitor profile segmentation, this kind of user-driven content is also search-driven content realized using query rules (and some code)

The IA aspects in the roadmap are taken from my SharePoint Information Architecture from the Field article.

Thursday, February 16, 2012

Reusable SPGridView with Multiple Filter and Sort Columns

The venerable SPGridView still has its use in SharePoint 2010 when your data is not stored in a list or accessible as an external content type through BCS. A typical example is when using a KeywordQuery to build a search-driven web-part feeding on results as DataTable as show in How to: Use the SharePoint 2010 Enterprise Search KeywordQuery Class by Corey Roth. Another example is cross-site queries using SPSiteDataQuery.

The SPGridView can use a DataTable as its data source, but several things from sorting arrows to filtering don't work as expected when not using an ObjectDataSource. As Shawn Kirby shows in SPGridView WebPart with Multiple Filter and Sort Columns it is quite easy to implement support for such features.

In this post, I show how to generalize Shawn's web-part code into a SPGridView derived class and a data source class wrapping a DataTable, isolating this functionality from the web-part code itself, for both better reusability and separation of concerns.

First the simple abstract data source class that you must implement to populate your data set:

namespace Puzzlepart.SharePoint.Core
{
    public abstract class SPGridViewDataSource
    {
        public abstract DataTable SelectData(string sortExpression);
 
        protected void Sort(DataTable dataSource, string sortExpression)
        {
            //clean up the sort expression if needed - the sort descending 
            //menu item causes the double in some cases 
            if (sortExpression.ToLowerInvariant().EndsWith("desc desc"))
                sortExpression = sortExpression.Substring(0, sortExpression.Length - 5);
 
            //need to handle the actual sorting of the data
            if (!string.IsNullOrEmpty(sortExpression))
            {
                DataView view = new DataView(dataSource);
                view.Sort = sortExpression;
                DataTable newTable = view.ToTable();
                dataSource.Clear();
                dataSource.Merge(newTable);
            }
        }
    }
}

The SPGridViewDataSource class provides the SelectData method that you must override, and a completed Sort method that allows the SPGridView to sort your DataTable. Note that this class must be stateless as required by any class used as an ObjectDataSource. Its logic cannot be combined with the grid view class, as it will get instantiated new every time the ObjectDataSource calls SelectData.

Then the derived grid view with support for filtering and sorting, including the arrows and filter images:

namespace Puzzlepart.SharePoint.Core
{
    public class SPGridViewMultiSortFilter : SPGridView
    {
        public SPGridViewMultiSortFilter()
        {
            this.FilteredDataSourcePropertyName = "FilterExpression";
            this.FilteredDataSourcePropertyFormat = "{1} = '{0}'";            
        }
 
        private ObjectDataSource _gridDS;
        private char[] _sortingSeparator = { ',' };
        private string[] _filterSeparator = { "AND" };
 
        public ObjectDataSource GridDataSource
        {
            get { return _gridDS; }
            private set
            {
                _gridDS = value;
                this.DataSourceID = _gridDS.ID;
            }
        }
 
        public bool AllowMultiSorting { get; set; }
        public bool AllowMultiFiltering { get; set; }
 
        string FilterExpression
        {
. . .
        }
 
        string SortExpression
        {
. . .
        }
 
        protected override void CreateChildControls()
        {
            base.CreateChildControls();
 
            this.Sorting += new GridViewSortEventHandler(GridView_Sorting);
            this.RowDataBound += new GridViewRowEventHandler(GridView_RowDataBound);
        }
 
        protected void GridView_Sorting(object sender, GridViewSortEventArgs e)
        {
            EnsureChildControls();
            string direction = e.SortDirection.ToString();
            direction = (direction == "Descending") ? " DESC" : "";
 
            SortExpression = e.SortExpression + direction;
            e.SortExpression = SortExpression;
 
            //keep the object dataset filter
            if (!string.IsNullOrEmpty(FilterExpression))
            {
                _gridDS.FilterExpression = FilterExpression;
            }
        }
 
        protected void GridView_RowDataBound(object sender, GridViewRowEventArgs e)
        {
            EnsureChildControls();
            if (sender == null || e.Row.RowType != DataControlRowType.Header)
            {
                return;
            }
 
            BuildFilterView(_gridDS.FilterExpression);
            SPGridView grid = sender as SPGridView;
 
            // Show icon on filtered and sorted columns 
            for (int i = 0; i < grid.Columns.Count; i++)
            {
. . .
            }
        }
 
        void BuildFilterView(string filterExp)
        {
. . .
 
            //update the filter
            if (!string.IsNullOrEmpty(lastExp))
            {
                FilterExpression = lastExp;
            }
 
            //reset object dataset filter
            if (!string.IsNullOrEmpty(FilterExpression))
            {
                _gridDS.FilterExpression = FilterExpression;
            }
        }
 
        public ObjectDataSource SetObjectDataSource(string dataSourceId, SPGridViewDataSource dataSource)
        {
            ObjectDataSource gridDS = new ObjectDataSource();
            gridDS.ID = dataSourceId;
            gridDS.SelectMethod = "SelectData";
            gridDS.TypeName = dataSource.GetType().AssemblyQualifiedName;
            gridDS.EnableViewState = false;
            gridDS.SortParameterName = "SortExpression";
            gridDS.FilterExpression = FilterExpression;
            this.GridDataSource = gridDS;
 
            return gridDS;
        }
    }
}

Only parts of the SPGridViewMultiSortFilter code is shown here, see download link below for the complete code. Note that I have added two properties that controls whether multi-column sorting and multi-column filtering are allowed or not.

This is an excerpt from a web-part that shows search results using the grid:

namespace Puzzlepart.SharePoint.Presentation
{
    [ToolboxItemAttribute(false)]
    public class JobPostingRollupWebPart : WebPart
    {
        protected SPGridViewMultiSortFilter GridView = null;
 
        protected override void CreateChildControls()
        {
            try
            {
                CreateJobPostingGrid();
            }
            catch (Exception ex)
            {
                Label error = new Label();
                error.Text = String.Format("An unexpected error occurred: {0}", ex.Message);
                error.ToolTip = ex.StackTrace;
                Controls.Add(error);
            }
        }
 
        private void CreateJobPostingGrid()
        {
            //add to control tree first is important for view state handling  
            Panel panel = new Panel();
            Controls.Add(panel);

            GridView = new SPGridViewMultiSortFilter();
 
  . . .
 
            GridView.AllowSorting = true;
            GridView.AllowMultiSorting = false;
            GridView.AllowFiltering = true;
            GridView.FilterDataFields = "Title,Author,Write,";
 
  . . .
 
            panel.Controls.Add(GridView) 

            //set PagerTemplate after adding grid to control tree

 
            PopulateGridDataSource();
 
            //must bind in OnPreRender
            //GridView.DataBind();  
        }
 
        protected override void OnPreRender(EventArgs e)
        {
            GridView.DataBind();
        }
 
        private void PopulateGridDataSource()
        {
            var dataSource = new ApprovedJobPostingDataSource();
            var gridDS = GridView.SetObjectDataSource("gridDS", dataSource);
            //add the data source
            Controls.Add(gridDS);
        }
    }
}

Note how the data source is created and assigned to the grid view, but also added to the control set of the web-part itself. This is required for the grid's DataSourceId binding to find the ObjectDataSource at run-time. Also note that data binding cannot be triggered from CreateChildControls as it is too early in the control's life cycle. The DataBind method must be called from OnPreRender to allow for view state and child controls to load before the sorting and filtering postback events

Finally, this is an example of how to implement a search-driven SPGridViewDataSource:

namespace Puzzlepart.SharePoint.Presentation
{
    public class ApprovedJobPostingDataSource : SPGridViewDataSource
    {
        private string _cacheKey = "Puzzlepart_Godkjente_Jobbannonser";
 
        public override DataTable SelectData(string sortExpression)
        {
            DataTable dataTable = (DataTable)HttpRuntime.Cache[_cacheKey];
            if (dataTable == null)
            {
                dataTable = GetJobPostingData();
                HttpRuntime.Cache.Insert(_cacheKey, dataTable, null
DateTime.Now.AddMinutes(1), Cache.NoSlidingExpiration);
            }
 
            this.Sort(dataTable, sortExpression);
 
            return dataTable;
        }
 
        private DataTable GetJobPostingData()
        {
            DataTable results = new DataTable();
            string jobPostingManager = "Puzzlepart Jobbannonse arbeidsleder";
            string jobPostingAssistant = "Puzzlepart Jobbannonse assistent";
            string approvedStatus = "0";
 
            SPSite site = SPContext.Current.Site;
            KeywordQuery query = new KeywordQuery(site);
            query.QueryText = String.Format(
"ContentType:\"{0}\" ContentType:\"{1}\" ModerationStatus:\"{2}\""
jobPostingManager, jobPostingAssistant, approvedStatus);
            query.ResultsProvider = SearchProvider.Default;
            query.ResultTypes = ResultType.RelevantResults;
 
            ResultTableCollection resultTables = query.Execute();
            if (resultTables.Count > 0)
            {
                ResultTable searchResults = resultTables[ResultType.RelevantResults];
                results.Load(searchResults, LoadOption.OverwriteChanges);
            }
 
            return results;
        }
    }
}

That was not too hard, was it? Note that SearchProvider Default work for both FAST (FS4SP) and a standard SharePoint 2010 Search Service Application (SSA).

All the code can be downloaded from here.

Thursday, May 19, 2011

New Sites and the SharePoint 2010 Content Type Hub

The SharePoint 2010 content type hub does quite a good job of managing and publishing a centrally controlled set of content types. There are a few quirks and limitations, some of them documented in Content Type Hub FAQ and Limitations by Chaks' SharePoint Corner; not to forget the content type publishing timer jobs that actually push the content types to the subscribers.

One of the less documented areas of using a content type hub (HUB) is what happens to new site-collections that are provisioned? What if I have list definitions in my features, how can I be sure that their referenced content types have been provisioned at feature activation time? Can I deploy my enterprise content types feature at both the hub site-collection and also at new site-collections that users create?


First, when a new site-collection is created, it will immediately have all the published content types from the parent SharePoint web-application's connected Managed Metadata Service (MMS) application's defined HUB, automatically provisioned into its local content type gallery. Note that this applies only to the published content types as configured in the source hub. Content types that are not published, will not exist in your new site-collection. Note that hub content types are not by default published; this must be configured for every single content type in the source hub.

So if your list definitions depend on global content types that have not been published in the HUB, your feature activation will fail. You can of course solve this by publishing the applicable global content types in the source hub and run the timer jobs first, as this will ensure that new site-collections will have the enterprise content types auto-provisioned from the MMS HUB.

However, you can also deploy your enterprise content types feature to both the content type hub and to any other site-collection that you create. This works fine as the site content type definitions are identical, including the content type ID structure - after all it is the same content type CAML feature. This won't affect subscribing to the content type hub, and publishing new, updated or derived content types from the hub works just as expected.

Activate your site content type feature before activating your list definition feature, or any other feature that depends on the site content types being provisioned, to ensure that they exists locally in the new site-collection even if not yet published in the HUB.

As your taxonomy is subject to change, so are your enterprise content types. Thus, your deployment strategy for enterprise content types needs to handle change. I strongly recommend using the Open-Closed Principle for modifying and extending the enterprise content types. The Open-Closed Principle is based on using a set of immutable base content types that you derive from to make new specialized content types, inheriting fields from the base. The immutable base of the Open-Closed Principle coincides nicely with provisioning global content types through both a feature and the content type hub, as by policy any changes are made by extending the former through the latter.

Even trivial stuff such as providing standardized company templates for Word and other Office applications, is best done by publishing new derived content types. Use the content type hub to inherit your base PzlDocument into PzlDocumentMemo and attach a template, go to "Manage publishing for this content type" to publish the content type. Wait for, or run, the two HUB timer jobs, and then add the Word template to the applicable document libraries.

Now you're in for a surprise later on. The next time you try to create a new site-collection after publishing modifications in the HUB, you might get this "content type is read only" error:


The ULS log typically contains an exception like this:

SPContentTypeReadOnlyException
Error code: -2146232832
The content type is read only or updateChildren is true and one of the child objects of the content type is read only.


The root cause for this is that published content types are by default read-only in the subscribers. What typically leads to this error is the need to use code when provisioning content types, e.g. when renaming and reordering fields, or when adding the enterprise keywords field to your content type. Another typical scenario where code is required is managed metadata fields; see How to provision SharePoint 2010 Managed Metadata columns by Wictor Wilén.

Making changes to the site content type definition in the FeatureActivated code and then calling SPContentType Update with updateChildren=true will work fine, until someone creates a new derived content type in the source hub and publish it. Your carefully tested code will suddenly crash, as the published child content type is read-only! Alas, what better proof that the deployed and the published global content types are the same?

Luckily, the change is isolated to the new inherited content type, thus it can safely be ignored when deploying the base content types. Use this overloaded Update method when modifying the global content types:

public void Update(
         bool updateChildren := true,
         bool throwOnSealedOrReadOnly := false
)

The HUB change did not affect your global content type due to using the Open-Closed governance policy for enterprise content types. See my SharePoint 2010 Open-Closed Taxonomy post to learn more about this recommended policy.

The content type hub and the Managed Metadata Service are perhaps the best new features in SharePoint 2010, still there are some uncharted areas that make developers reluctant at using the MMS HUB. There are a lot of articles at Technet and MSDN on the architecture, but way too little about deployment scenarios and issues such as those in this post.

Friday, July 02, 2010

Scalable SharePoint 2010 Farm & Services Architecture

This post describes a proposed solution design for providing an intranet collaboration farm capable to scale out to support a publishing portal and extranet collaboration with partners and suppliers. The design is based on a set of diverese sites types identified in an Information Architecture analysis, and the general planning recommendations for these types. In addition, several non-functional requirements is accounted for in the design, such as farm security, robustness and availability.

The SharePoint 2010 farm is designed with scale out support for different future solution areas. The initial farm architecture will be designed to support collaboration for internal users only, with an option to also add an intranet publishing solution. The long term goal of the farm is to eventually support collaboration with external users also, such as partners an suppliers.

All SharePoint 2010 solutions are built from a set of elements ranging from the farm hosting the solutions down to sites that contains the actual functionality and information of the solutions.


SharePoint solutions are deployed into SharePoint web-applications in a SharePoint farm. Web-applications are management containers for site collections. SharePoint uses site collections to structure the sites and subsites that implements the actual functionality of SharePoint solutions.

Site collections bridge logical architecture and information architecture (IA). The design goals for site collections in the model are to create logical divisions of content and functionality, in addition to satisfy requirements for URL design. To satisfy the requirements for URL design, each web application includes a single root-level site collection, such as ://portal/. In addition, a set of managed paths are used to incorporate a second tier of site collections, such as ://portal/HR/ and ://portal/sites/***.

All web-applications in a farm share a group of common SharePoint 2010 service applications that provide shared services such as indexing & search to the farm. I use the term “shared services provider” (SSP) in this post for such a group of service applications. A farm can contain multiple groups of shared services, such as an intranet SSP and an extranet SSP.

The overall SharePoint farm architecture for internal collaboration can be summarized like this (click to enlarge):


The initial SharePoint 2010 farm will be configured to support two solutions: providing team-sites for internal collaboration in groups or projects, and for providing My Sites for employees. These two web-applications will share a group of shared service applications (“default group”) for search, user profiles and enterprise taxonomy.

The User Profile service is required to provide My Sites. In addition, this service application is what provides all social features such as tagging, rating and social bookmarking.

Note that a search service application is needed for more than just crawling and queries, the social tagging functionality in SP2010 also requires search. The reason is that several web-parts such as Note Board, Tag Cloud and Tagged Items in some modes depends on the search service to do security trimming to provide their content. This includes the tag profile page, which use such web-parts. You can also use the search service and the content search, people search and social search capabilities when customizing the user experience of your solutions. A typical example is creating a tag cloud web-part for managed metadata and enterprise keywords columns for lists and libraries, not just for social bookmarking (it is not really social tagging that is implemented in SP2010) with the standard "Tags & Notes" ribbon.

If you use FAST Search for SharePoint 2010 (FS4SP) then the FAST Content SSA will provide the content sources that are crawled for search results, except for people search. The FAST Query SSA has a different set of content sources, including the one feeding the people scope. Only one Search Service Application (SSA) should be associated with your web-application, either FAST or standard SharePoint 2010 Server search. Note that FAST doesn't index the social tags, which affects those social tagging web-parts that depends on search (see above).

The intranet will use Active Directory as the identity provider for authentication in classic mode. The web-application authentication mode can later on be switched from classic to claims-based as needed. Note that it is not recommended to switch the mode from claims-based to classic.

The farm is also designed to scale-out to support future SharePoint 2010 solutions such as a publishing intranet (://puzzlepart). Note that other service applications not shown here might be required to support the functional solution design and future solutions.

The different web-applications in the farm runs in separate application pools to achieve process isolation. This way an error or crash in one application will be isolated and not affect the other web-applications in the farm.

Anywhere access for mobile employees to the intranet sites is provided using Forefront Unified Access Gateway (UAG). This supports a wide range of locations such as at home or on the road, and a wide range of devices from laptops to smart phones.

The overall SharePoint farm architecture target for collaboration with external partners and suppliers can be summarized like this (click to enlarge):


The extended 2011+ version of the farm is scaled-out to support two new solutions; one is the ://puzzlepart web-application for intranet publishing, the other is an extranet web-application for collaboration with external users such as partners and suppliers. The former connects to the intranet shared services provider (“default group”) and will as such reuse and share search, user profiles, and the enterprise taxonomy (metadata and content types) with the existing solutions for team-sites and my sites.

The extranet collaboration solution must not be connected to the intranet SSP for information security reasons. Using a separate group of shared services for all service applications that may contain or expose confidential information is strongly recommended – an extranet SSP (“custom group”). A typical service application of this kind is search. Using separate service applications prevents accidental information exposure due to e.g. misconfiguration of a service application.

The User Profile service application is included in the extranet SSP even if the extranet will not provide My Sites at all. This service is what enables user to do social tagging and bookmarking, and must thus be part of the extranet SSP to provide social features. It is not recommended to connect the extranet to the intranet User Profile service for the same reasons as for the Search service.

Still, some service applications are typically shared across both the intranet and the extranet web-applications. This includes the enterprise taxonomy – you need to provide for consistent classification and tagging of content across all solutions to drive findability. The managed metadata service, including content type hubs, is specifically built for the purpose of being both shared and syndicated across multiple solutions.

Another security aspect is access control; the extranet must use a separate claims-based authentication provider, that can be federated with external identity providers. The intranet and extranet web-applications must trust different providers for authorization. Using separate providers prevents accidental information exposure due to e.g. misconfiguration of site access control and group memberships. Note that multi-mode authentication is preferred over mixed mode for extranet collaboration.

A final aspect of the extended farm design is that the extranet web-application and shared services are run in separate application pools from the intranet. This ensures process isolation to prevent errors and crashes in the extranet processes to affect the intranet solutions. It also isolates malicious use of the extranet process resources from affecting the intranet processes, such as denial of service attacks.

Remote access for external users to the extranet collaboration web-application is provided using Forefront Unified Access Gateway (UAG) combined with ADFS for federated identity and access management (IAM). Using UAG saves you from setting up and managing a separate extranet farm in a DMZ perimeter zone; instead you give a controlled set of external users secure anywhere access to a controlled set of internal applications hosted on the intranet farm.

Friday, June 18, 2010

Open-Closed Term Sets & Terms in SharePoint 2010

The 'Move Term Set' action for term store management, in combination with Restricted and Full permissions settings for the Managed Metadata Service, can be used to enforce the Open-Closed principle for adding new term sets to your taxonomy. Allow local taxonomy managers to add column specific term sets (open for extension) in local site-collections, but do not give them full permissions on the core term store (closed for modification). Then have a policy for periodically reviewing local site-collection term sets to incorporate useful new ones into the shared core term store.


Note that each term set also have an open-closed setting for controlling if new terms can be added to the term set or not. Users with restricted or full permissions are allowed to add new terms to open term sets, users with read permissions are not. Use the combination of open + restricted to enforce the "open for extension, closed for modification" policy for term set terms.

Note that access to the term store is granted per web-application using the app-pool account, not per user or group.  In addition, you can control permissions on each term group using the "contribute" and "manage" settings for granting rights to users and groups. If users are not granted contribute rights, they will be restricted to read even if the web-application's MMS connection permissions allows for more than just read.

Tuesday, June 15, 2010

Content Type Hub Publishing and Column Specific Term Sets in SharePoint 2010

In SharePoint 2010, you realize your taxonomy using the content type hub and term store provided by the Managed Metadata Service (MMS). The hub contains site content types built from site columns, and the "Managed Metadata" column type is what you use to connect term sets to a field. When adding a managed metadata column, you must choose between "Use a managed term set" defined in the MMS using Central Admin, or "Customize your term set" which allows you to create a new column specific term set on the fly.

Column specific term sets that you create will by default be assigned to the site-collection hosting the site content type gallery in which you create the site column. Such term sets will not be visible in the Term Store Management Tool in Central Admin, even if the MMS connection is set as the default storage location for column specific term sets. This also applies to column specific term sets created in the content type hub.

Do not create column specific term sets in a content type hub, this will break the content type publishing. If you click "Manage publishing for this content type" you will get this error:
The current content type contains a managed metadata column that uses a customized term set that is not available outside the current site collection. Please change the column setting or remove the column and publish the content type again.
There is, however, a third option in addition to the two suggestions; moving the term set from the site collection term group to a shared term set group in the managed metadata service term store.


Move the term set, then verify that the content type publishing setting is Republish in the content type hub. In Central Admin, run the "Content Type Hub" job first, then all applicable "Content Type Subscriber" jobs to execute the actual publish-subscribe process. Finally, open the site content types inventory in a subscriber site-collection and verify the subscribed content type, site column and term set.

I also recommend updating the site column definition to use the "use a managed term set" setting after moving the term set into the core term store.

Wednesday, May 12, 2010

Minimal SharePoint Governance Plan - Part III

This is part III in a mini-series about the Minimal SharePoint Governance Plan needed to get you started with your SharePoint governance efforts.This part gives a more detailed overview of the mininal governance plan. The overview comprises operational and functional areas from SharePoint architecture, via site, user and information lifecycle management, to realization of SharePoint solutions. As you will see, there is a multitude of governance aspects, even if just focusing on the technical aspects. Executing on all governance aspects from day one is not viable, that is why I recommend to start with simple governance.

The is no one governance plan to rule them all. Governance is too multi-faceted for a single set of policies to fit all the different site types across diverse business areas. Governance for controlled publishing sites will be very different from Enterprise 2.0 pull-style situational solutions, as will it differ for management of project sites and team sites, and as for community, social and personal sites. Adapt your governance plan according to the targeted solution.


The site classification scheme shown here is from the Technet SharePoint Governance Checklist Guide, refer to page 20 for more details.

Architectural Governance

The governance plan for the SharePoint solution must define a logical architecture model based on your Information Architecture analysis, adhering to architectural components, farm deployment, capacity and planning recommendations on Technet.

• Farm design policies for a robust and flexible platform
• Sharing and isolation policies for applications and information
• Site-collection structure policies to drive overall solution architecture & governance
• Information asset structure policies to ensure classification, management and findability

The objective of having architectural policies is to create a workable farm and solution design considering hard and soft SharePoint limits.

Site Lifecycle Management

The governance plan for site lifecycle management (SLM) must specify policies for managing sites from creation to disposition. Define a classification scheme for site types and adapt your governance plan to each site type. It is recommended to develop timer jobs to automate and enforce SLM policies.

You need a site sweeper job that disposes expired, abandoned and useless sites from your solution to ensure that the overall effect produces useful business results. Make sure that knowledge captured in obsolete sites are retained through information management tasks before permanently disposing the sites. Alas, don't be afraid of self-service provisioning, after all more is different.

• SLM policies must be defined and enforced
• Site Provisioning
    o Implement custom provisioning if ootb functionality is not sufficient
    o Provisioning policy per site type defines level of automation and self-service
    o Use provisioning wizard to collection data related to SLM
    o Store SLM data in site properties or a site inventory list
• Site Retention
    o Do not rely on database backup for retention, backup retention might be shorter than site retention
    o Prepare to restore sites deleted by users
• Site Disposition
    o Implement custom site sweeper if ootb functionality is not sufficient
    o Standard site sweeper is only for site-collections (site use confirmation)
    o Define a procedure for information management when disposing a site

Note that there is no ootb site directory in SharePoint 2010. Still, just create a shared custom list and use it for inventory management of sites as part of your SLM implementation.

DocAve Backup & Recovery a recommended 3rd-party tool, it provides capabilities beyond ootb SharePoint 2010, such as item-level recovery. For site retention, the CodePlex MSIT Site Delete Capture tool is an option.

User Lifecycle Management / Identity & Access Management

The governance plan for user lifecycle management (ULM) must specify policies for managing users from onboarding to termination. ULM is directly related to information security (access and auditing), information management, and Identity & Access Management (IAM). Employees come and go, resulting in SharePoint data that nobody manages, or in worst case, lost knowledge.

Implementing good site disposition policies and good information management policies will reduce the efforts required for user lifecycle management, as obsolete sites and information then will be disposed of in a timely manner - keeping the quantity of orphaned data down.

• ULM policies must be defined and enforced
• Site memberships and permissions must be assigned for new users
• Site and information asset permissions & ownership must be handled when
    o Account is terminated
    o User transfers to another business role or department
• A policy for reassignment of ownership must be defined

Having tools for management of user permissions, ownership and lifespan is nice, but no prerequisite. LigthningTools DeliverPoint or Axceler ControlPoint are recommended partner solutions for user management.

Content Type Governance / Information Management

The governance plan must specify policies for content management according to your Information Architecture analysis and taxonomy. A taxonomy is realized in SharePoint using Site Content Types for information asset types and Term Sets for coherent tagging of information. Content types combined with metadata tagging is essential for information classification and for driving findability.

Content types defines the static classification hierarchy of the information managed in SharePoint. A content type is built from a set of fields defining the metadata of the content type, further detailing the classification of the information. Some metadata fields require the use of a controlled vocabulary for content tagging.

Information management policies can be assigned to content types. The most important are for retention and disposition of content, helping you manage e.g. outdated content to ensure the relevance and timeliness of your information.

• Reuse the Open-Closed Enterprise Taxonomy across web-applications and site-collections
• Always use a core Content Type Hub store for the enterprise taxonomy
• The core content type store defines company specific immutable base content types
• Ensure that all additional content types derives from the core content types, by extending the immutable base
• Use few required metadata fields, max 3-5 per content type
• Use sensible default values where possible
• Define polices for
    o Reusable content (document repository)
    o Retention of outdated content / historical archive (document repository)
    o Retention of expired content (records repository or disposition)
    o Regulatory compliance records (records management)
    o Disposition of content
• Retention policies are important for driving findability, use them to prevent irrelevant search results
• Define and enforce behavior using
    o Information Management policies
    o Workflows / Event receivers
• Use Information Management policies for
    o Retention, disposition, auditing, labeling / barcodes
• Ensure that content types are evolved according to best practices

The new SharePoint 2010 multi-stage retention support, combined with workflow and document /records repositories allows for implementing and enforcing sophisticated content management policies. Note that the new SharePoint 2010 Content Organizer only supports Document-based content types, in addition to e-mail messages.

Managed Metadata Governance

The governance plan must specify policies for management of the managed metadata used when tagging content in SharePoint. Managed metadata is a controlled vocabulary defined in the corporate taxonomy, realized in SharePoint 2010 as term sets defined in the Managed Metadata Service.

• Reuse the Open-Closed Enterprise Taxonomy across web-applications and site-collections
• Always use a core Managed Metadata Service term store for the enterprise taxonomy
• Allow local Managed Metadata Services for isolated, locally managed term stores
• Always use synonyms when defining terms, consistent content tagging is essential for content management and for driving findability
• Use term translation to support other languages for the term
• Avoid random or haphazard tagging due to unintelligible terms
• Enable managed keywords for user-driven freeform tagging of content
• Ensure that term sets are evolved according to best practices
• Define and enforce a policy for reviewing open term sets for improper usage

Note that search do not comprise term synonyms or translations when searching, it only finds the stored key term. The same applies to faceted search – or 'refinement panels' as they are called.

You can have multiple Term Set stores and Content Type Hub inventories in SharePoint 2010. This allows for combining both enterprise definitions and local definitions to support both shared and isolated taxonomy configurations. See Plan to share terminology and content types on Technet.

Social Tagging Governance

The governance plan must specify policies for management of the social tagging features

• Use managed keywords to enable folksonomy for content (list items)
• Use social tagging to enable folksonomy for "anything with an URL"
• Allow for managed metadata and managed keywords to be included in social tags
• Define and enforce a policy for reviewing the folksonomy tags for improper usage

Note that the social tagging of "anything with an URL" is provided by the SharePoint2010 User Profile Service application, not the Managed Metadata Service application. Thus, social tags have no explicit relation to the term store at all. The same applies to other SharePoint2010 social features such as ranking and social bookmarking.

Document Template Governance

The governance plan must specify policies for using Office templates in content types.

• Use a shared set of enterprise Office templates
• Manage and store templates in a SharePoint document library at a central location
• Do not store templates directly in content types, always reference the central shared templates
• Make use of the Office 2010 Backstage or the document information panel for managing metadata directly in Office

Office 2010 now has support for storing templates in a SharePoint repository. Use AD group policies to populate 'File > Save As' and to lock down storage locations such as file shares and local disk.

List & Library Definition Governance

The governance plan must specify policies for managing content in lists and libraries. It is strongly recommended to use only lists based on site content types, rather than directly customizing list definitions. Enforcement of consistent classification and information management policies depends on using site content types.

• List content
    o Use only a few content types per list
    o Content types in a list must be cohesive
    o Prefer list views over "dumb" folders
    o Use SharePoint 2010 folders when appropriate
• List permissions
    o Prefer using inherited permissions
    o Avoid user item level permissions
• Enforce content management policies using
    o Versioning, check-in/out, workflows / event receivers
• Information Rights Management (IRM)
    o Policies for document access and usage restrictions
    o Applies IRM policies when document is downloaded from library
    o Enable by installing Active Directory Rights Management Services (AD RMS)
• Information management policies
    o Prefer implementing IM policies on content types rather than on lists or libraries

Note that some of the new SharePoint 2010 features work only for document libraries, such as the Unique Document ID, Document Set and Content Organizer features.

Permissions Governance

The governance plan must specify policies for how to manage access to sites and information assets, including which permissions users and groups have. All experience shows that simple permission policies are more secure. The more intricate and fine-grained permissions assignments you have, the harder it is to know who has access to what – and the more likely it is that there will be information security breaches exposing confidential information.

• Use SP groups to manage user group memberships
• Build your SP groups from AD security groups
    o Management of AD group members is typically a bottleneck, thus avoid it
• Do not assign permissions to single users, always assign to SP groups
• Prefer inherited groups (role assignments)
• Prefer inherited permission levels (role definitions)
• Use unique permissions at site level (favored) or list/library level only when absolutely required
• Avoid assigning item level permissions
• Site-collections are preferred security management boundaries

The visibility into what a user has access to has improved a bit in SharePoint 2010, so has the usage reporting capabilities. Still, 3rd-party tools such as LigthningTools DeliverPoint or Axceler ControlPoint might be required for professional permissions management beyond the built-in SharePoint 2010 Permissions Tool.

Search Governance

The governance plan must specify policies for driving findability through indexing and search. The Information Architecture analysis defines the information taxonomy and organization blueprint realized in a SharePoint site structure capable of storing and managing your content. The site structure combined with content types enables findability through consistent classification and tagging of content.

• Ensure ease of adding information assets to correct location
    o Users should not have to enter a lot of required metadata
    o Users should not have to browse/navigate extensively to store content
    o Task context should deduce location, e.g. CRM client document store
• Metadata tagging through content types for all findable assets
• Use content type retention policies to prevent irrelevant, outdated search results
• Use search scopes to provide search context: people, tasks, articles, project, archive, etc.
• Use faceted search (refinement panel)
• Ensure and enforce information isolation
    o Farm design must prevent configuration mistakes from exposing confidential information by accident
    o Use separate service application groups or even separate shared services farms

The most valuable search is the one that connects a user to other people, as people are often the best sources of information and knowledge, especially tacit knowledge – know-how relating to new better business performance or to novel information that generate flow of new knowledge – in short, that ignites innovation. A former CEO of Hewlett Packard famously observed: "If HP knew what HP knows, we would be three times as profitable".

Findability is more than just search capabilities, it also includes the SharePoint 2010 social computing features such as “Tags & Notes” for tagging and social bookmarking. Tag clouds, metadata-based navigation and filtering, and even the My Site activity feed are all enablers of driving findability.

Note how I say "driving findability"; findability is not something you just enable, you have to actively manage and adapt the Search Service application settings according to your business needs. Just enabling search is just as bad as not managing your user's expectations for what to expect from enterprise search.

All parts of this mini-series:
Part I - SharePoint Governance - Eating an Elephant
Part II - Start with Simple Governance
Part III - Minimal Governance Plan (this post)

Tuesday, May 11, 2010

Start with Simple Governance - Part II

This is part II in a mini-series about the Minimal SharePoint Governance Plan needed to get you started with your SharePoint governance efforts. The objective is to create a simple and viable plan that covers the core governance aspects for operations and solution management. The plan comprises the functional areas of textbook enterprise governance plans, skipping organizational aspects to focus on technical aspects. The focus is on lifecycle management of SharePoint sites and users, classification and management of content, and on driving findability, all hosted on a robust SharePoint farm.

Required Operational Governance

The governance plan for the SharePoint environments must specify policies for both farm infrastructure and operations. The architecture of the farm must be adequate for hosting the planned solutions within defined service level agreements (SLA), and at the same time be positioned for future expansion into other solution areas.

Operational aspects that must be covered by the governance plan:

• Availability
    o Farm with redundancy
    o Monitoring
• Backup and Recovery
    o Retention and disposition policies must be defined and enforced
    o Restore specific information assets
    o Tested disaster recovery plan
    o Complete solution can be restored within allowed time limit

Usage of SCOM SharePoint management packs is recommended. For recovery is DocAve Backup & Recovery a recommended 3rd-party tool, it provides capabilities beyond ootb SharePoint 2010, such as item-level recovery.

Required Solution Governance

The governance plan for the SharePoint solution must specify policies for governing the sites and information assets that make up the solution. This especially applies to how to manage solutions over time as they evolve in response to changed business requirements.

Solution aspects that must be covered by the governance plan:

• Site Lifecycle Management (SLM)
    o Policies for provisioning, retention and disposition must be defined
    o Automation of SLM through site provisioning forms and timer jobs
    o Site delete capture for retention beyond database backups
• Content Type and metadata definitions (taxonomy)
    o Policies for retention and disposition, including archival and records, must be defined
    o Classification of all information assets, from sites to documents and items
    o Cover the core content types for your company (the immutable base content types)
    o Enable findability through consistent classification and tagging of content

SharePoint has little built-in support for SLM, but most SharePoint workflow tools provides support for user input forms and site creation and management. For site retention, the CodePlex MSIT Site Delete Capture tool is an option.

There is good taxonomy support in SharePoint 2010 provided by the Managed Metadata Service and the Content Type Hub mechanism. Note that neither managed metadata nor social tagging is included in SharePoint Foundation 2010, these are SharePoint Server 2010 service applications.

Optional Solution Governance

In addition to the above aspects, governance policies for how to manage users, their permissions and their ownership of sites and data should be defined. Employees come and go, resulting in SharePoint data that nobody manages, or in worst case, lost knowledge.

Solution aspects that should be covered by the governance plan:

• User Lifecycle Management (ULM)
    o Manage the lifecycle of accounts due to onboarding, transfering, and termination of users
    o Policies for permissions and ownership of sites and information assets
    o Automation of ULM though partner/open solutions
• Visibility into usage
• Visibility into permissions

The visibility into what a user has access to has improved a bit in SharePoint 2010, so has the usage reporting capabilities. Still, 3rd-party tools such as LigthningTools DeliverPoint or Axceler ControlPoint might be required dependent on needs.

SharePoint Governance Checklist

Microsoft provides a SharePoint governance checklist guide whitepaper that my customers find very useful. The checklist covers important governance and lifecycle management aspects that must be included in a governance plan. It is strongly recommended to review all areas of your governance plan against this checklist.

Aspects covered includes:
• Design-time and run-time governance
• Roles and ownership
• Information architecture, navigation and findability
• Branding
• Infrastructure and operations
• Testing and development

Each checklist has a related tips & information section, effectively making this a pocket governance guide.

All parts of this mini-series:
Part I - SharePoint Governance - Eating an Elephant
Part II - Start with Simple Governance (this post)
Part III - Minimal Governance Plan

Thursday, April 15, 2010

SharePoint 2010 Open-Closed Taxonomy

A taxonomy is realized in SharePoint using content types for information asset types and term sets for coherent tagging of information. Content types defines the static classification hierarchy of the information managed in SharePoint. A content type is built from a set of fields defining the metadata of the content type, further detailing the classification of the information. Some metadata fields require the use of a controlled vocabulary, realized in SharePoint 2010 as term sets defined in the managed metadata service.

A central design recommendation for both content types and term sets is the Open-Closed principle: open for extension, closed for modification. The Open-Closed principle states "software entities (content types, term sets) should be open for extension, but closed for modification"; that is, such an entity can allow its configuration to be modified through extension without altering the core base entity definition. This is especially valuable in a production environment, where changes to base definitions will necessitate code reviews, unit tests, and other such procedures to qualify those changes. The principle isolates the changes to the extension entity definitions, because the core base entities are immutable. This makes it simpler to evolve the entities in a controlled manner while keeping a common, consistent core base definition.

SharePoint is restrictive on what changes can be made when evolving or specializing content types, with very limited support for modifying existing content type definitions. The recommended practice is to use a base set of core content types (the closed part) and then perform all specialization and variations on content types inheriting from the base content types (the open part). Evolving term sets is less restrictive than content types and allows for much more flexibility in changes to their definition, including renaming and merging terms.


New in SharePoint 2010 is the capability to share content types and term sets across multiple site-collections and even across farms. These enterprise content types and term sets can be combined with local site content types and local term sets, the giving a combined syndication effect. The Open-Closed principle can be applied also to this syndicated taxonomy: use the enterprise content type inventory and the enterprise term set inventory as the core taxonomy base (the closed part), and use the site inventories to extend the taxonomy with local specializations (the open part).

In technical SharePoint 2010 service application terms: create a primary term store and a primary content type hub, then extend those locally with additional local term sets and local content type inventories subscribing to the primary core taxonomy.

Applying the Open-Closed principle both for content type and term set definitions, and for content type and term set syndication, allows for defining and enforcing governance policies that makes it possible to have centrally controlled lifecycle management of the core base definitions of your SharePoint taxonomy, while still allowing local extension and adaptation of the taxonomy.

See SharePoint 2010 Intra-Farm Term Store Syndication for details on realizing taxonomy syndication, including configuration of permissions per web-app, user or group for adding or modifying term sets.

Tuesday, January 19, 2010

SharePoint Content Types Best Practices

I posted some SharePoint site content type guidelines back in 2008 and this post adds some detailed recommendations on the content type ID attribute and on the SourceID and Name attributes for content types fields.

I often create content types and site columns using the UI because it is simple and straight forward, and then use SharePoint Manager to get a starting point for the feature XML from the SchemaXml panel in SPM. It is just a starting point as you need to modify the XML for it to become valid CAML for defining site content types and site columns. I won't describe this mechanism in details, you can find help on this elsewhere.

You should at least make these changes for site columns:
Creating a new field GUID will ensure that your packaged site columns will not collide with the jump start fields when you activate your feature later on. For the same reason, I recommend prefixing all your jump start names with e.g. "dev" such as in "devContosoWebIngress".

You should at least make these changes for content types:
  • Keep only the fields you added to the content type (it will contain all inherited fields also) 
  • Update the <FieldRef> IDs to reflect the new GUIDs you created for your site columns
  • Create a new <ContentType> ID according to recommended practices (see more below)
  • If your content type ID really requires a new GUID, make sure it does not start or end with zero
ContentType IDs have two main formats, one based on GUIDs for your core content types and one based on simple two-digit numbering for variations on those core types. The core content types should be your immutable base types, while the variations are where the customizations are done. See my "open for extension, closed for modification" recommended Open-Closed practice.

Creating content types using the UI will always create a "core style" ID, even if you just make a content type variant. That is why you always should create a new ID for your custom content types, limiting the number or core content types and use the shorter "variant style" ID whenever possible. Let me explain using an example to show how to inherit a new custom core type "Generic Section Page" from the standard MOSS WCM "Page" type and then create two variations of the core custom type.

The Page type has this ID, called [page] later on:

0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39

Our new "Generic Section Page" core content type requires a registry format GUID with the {} and - removed. To inherit the Page content type you append 00 and the custom, stripped GUID to the Page ID to create a new core ID:

[page]0042E113FF4B6C402B96F767B61591F882

This core content type ID is analogous to a namespace in .NET code, it is a unique grouping of all content types that have this as their base ID.

Our new section page variants "Section Page A" and "Section Page B" needs no GUIDs, just their own two-digit number appended to the core type, but not the reserved 00 combination:

[page]0042E113FF4B6C402B96F767B61591F88201
[page]0042E113FF4B6C402B96F767B61591F88202

Note that 00 is a marker for inheritance between content types when the IDs are concatenated. Thus, I recommend that your custom content type GUIDs never start or end with zero, as this just makes it harder to know where a specific type starts or ends in an inheritance tree ID as it gets longer and longer. Just a simple thing. Still, looking at content type source feels like looking at the green rain of Matrix code.

[UPDATE] See Content Type IDs on MSDN published in May 2010.

As a side note, research suggests that you need no more than about five to ten core content types to classify and organize your content - just think of the library card taxonomies such as the Dewey Decimal Classification that has been used for centuries.

Regarding naming of site columns; the age old recommendation for SharePoint sites and lists is to create them first without any spaces or special characters in the title to set the name, and then rename the site or list afterwards to set the display title. The same applies to site columns, make sure that the Name attribute is without any spaces or special characters - apply your user friendliness skills to the DisplayName attribute.

If you use the above SchemaXml trick and don't follow the naming best practice, for "Contoso Web Ingress" you get "Contoso_x0020_Web_x0020_Ingress" as the Name. If you think that looks ugly, just wait until you do a rollup using the content query web-part (CQWP) and need to select that field in XSL - then your laziness will result in this:

<xsl:value-of select="@Contoso_x005F_x0020_Web_x005F_x0020_Ingress" />

In this case it would be better to put lipstick on the pig using the CQWP DataColumnRenames setting to rename the encoded name to what it should have been in the first place.

Note that not all underscores will get encoded into _x005F_ such as for the description field of standard SharePoint lists and doc-libs, which has the internal name _Comments. So if you use CQWP to fetch data from e.g. a custom list, it will still have the name _Comments in the item style XSL.

Thursday, July 02, 2009

List Content Type Fields & Forms: CAML vs Code

In a feature based site definition I recently made, I had a site content type hierarchy with a base content type and two child content types defined in a feature. The child types inherits their site columns and display/edit/new form from their parent.

Another dependent feature contained a list definition and a provisioned list instance that would reference the two child site content types and thus snapshot inherit them as list content types.

The columns of the list are based on the inherited site columns, in reality a snapshot copy of them at the time the list was created. All provisioning of site columns, site content type, list definitions, list instances and list content types are defined in CAML in the features.

As you can see from the above figures, everything looked fine and dandy. Until I tried to create a new list item: all the custom fields where missing in the new item form. The same problem applied to the display and edit forms also.

Using the "List settings" UI to remove and then add the list content types manually proved to fix the problem, so it had to be an issue related to the way CAML <MetaData/ ContentTypes/ ContentTypeRef> makes the snapshot copy.

Knowing that the fields of a form are rendered using the <ListFieldIterator>, it was time to check the field collection of the list content types using SharePoint Manager 2007.


In the above figure site content types are to the left and list content types to the right. The "News article" content type added through the "List settings" UI has inherited a full snapshot, but the "Stand-alone article" added by CAML has inherited just the ootb BaseType "Item" fields. So the <RenderingTemplate> for NewForm finds only two fields to render. And the actual problem is in the CAML inheritance.

The problem is that the list columns do not inherit the site columns correctly. In fact, the only official way of making <ContentTypeRef> work correctly using CAML is to repeat the site column definition inside the list definition <Fields> element. This is not a good approach with regards to maintenance of your solution.

There is a similar problem with list content types related to site content types, which is easily rectified by forcing a ContentType Update(push down changes) during provisioning of the site content type feature (see exercise 13-5 in the Building the SharePoint User Experience book).

The list feature is activated after the site content type feature. You might think that you could force another site content type update to rectify the inheritance again. That isn't very feasible, as updating child content types depends on a having actual changes to push down; and there is none at this time - remember that the site content types have already been provisioned and pushed down. There are some "best" practices out there on Custom Fields in Schema.xml, but please read on before trying those.

Just adding the list content types from code will create correct snapshots with no fuzz:

"When you add a site content type to a list using the object model, Windows SharePoint Services automatically adds any columns that content type contains that are not already on the list. This is in contrast to provisioning a list schema with content types, in which case you must explicitly add the columns to the list schema for Windows SharePoint Services to provision them."

So instead of doing this using CAML, just add them when activating the list feature. The newly created list instance is empty, so the content types can easily be added:

public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
using (SPWeb web = (SPWeb)properties.Feature.Parent)

{
//enable management of content types
SPList list = web.Lists["News"];
list.ContentTypesEnabled = true;
list.Update();
//inherit the list content types from site content types
list.ContentTypes.Add(web.ContentTypes["Stand-alone article"]);
list.ContentTypes.Add(web.ContentTypes["Series article"]);
list.Update();
//remove item as list content type
list.ContentTypes["Item"].Delete();
list.Update();
}
}

NOTE: This will only work for list instances created by the feature. The code will not run for list instances added later on - and there is no ListAdding/ListAdded events in WSS3.0/MOSS. For lists added manually, the list content types must also be added manually. Or create an "associate [my] list content types" feature that you can re-activate at will to bind the content types to lists as applicable. Use a marker content type to look for to avoid hardcoding the list names. Some like to use a timer job to bind the list content types.

The code gets a reference to the list instance, turns on content type management / content types on the new menu, adds the list content types as new snapshot copies, and finally removes a dummy "Item" content type (that is there to make the CAML list definition valid).

My changed list definition Schema.xml looks like this:

As you can see, I have commented out the CAML <ContentTypeRef> for my two child site content types, and instead just referenced the standard "Item" content type - the dummy that is removed during feature activation. If you still would like to have those <ContentTypeRef> in the Schema.xml, then remove and re-add them from the code.

Note how you still can add your custom fields as <ViewFields> even if their content type is not referenced. Content type management for the list can also be turned on using the EnableContentTypes attribute on the root <List> element in Schema.xml.

That's it. I deleted the list and reactivated the list feature - and voila: the new, edit and display forms contains all fields of the content type selected from the new menu.

[UPDATE] SharePoint 2010 adds the Inherits attribute that makes CAML inheritance work as expected, and using FieldRefs unnecessary. See also the new 2010 feature upgrade mechanism UpgradeActions AddContentTypeField element with the PushDown attribute that force changes to a site content type and all derived types.