Controlling Content Database Size in SharePoint

A SharePoint content database can be up to 4TB with data (max 200GB is recommended). However, storage size is not the problem; it is the recovery time to restore all that data that is the availability problem. The recovery time decides for how long your business critical solution will be down. As SharePoint can spread its content across multiple databases, it is recommended that your architecture segments different content across different databases based on IA and other user experience aspects, plus business requirements for availability and recovery time. Plan for structuring your solutions with a strong focus on your information architecture (IA).

Here are some options for how to control the size of the content databases, without disposing and deleting content:

A) Use an ootb Record Center as an archive for old content: The users must manually send each document to the RC using e.g. move and leave a link; note that only the latest major version with metadata is kept – all version history is lost. The information management policies supported by SharePoint for retention and disposition can be used to automate the cleanup.
As the RC has its own content databases, the live collaboration databases will grow slower or even shrink as outdated information is moved to the archive. Keeping the live databases small ensures shorter recovery time; while the recovery time for the archived content can be considerable, but not business critical.
Search must be configured appropriately to cover both live and archived content.

B) Use a third-party archiving solution for SharePoint from e.g. MetaLogix or AvePoint. This has the same pros & cons as in option A, but the functionality is probably better in relation to keeping version history and batch management of outdated content.
Search must be configured appropriately to cover both live and archived content.

C) Use a third-party remote blob storage (RBS) solution for SharePoint, such as MetaLogix StoragePoint, so that documents are registered in the database, but not stored there. This gives smaller content databases, but more complicated backup and recovery as the content now resides both in databases and on disk. Provided that you don’t lose both at the same time, the recovery time should be shorter.
Search will work as before, as all content is still logically in the “database”.

D) Use powershell scripts or other code to implement the disposition of outdated content. The script can e.g. copy old documents to disk and delete old versions from the content database; the drawback being that all metadata will be lost and there is no link left in SharePoint.
The databases size will shrink as data is actually deleted, and backup and recovery is more complicated as content is now both in the database and on disk (same as for option C).
Search can be configured to also crawl and index the files on disk, but content ranking will suffer as the valuable metadata is lost.

My recommendation is to consider option A first, especially if you are able to define automated rules and exploit the built-in information management policies in SharePoint. The keyword is *able* - in my experience, everyone is positive to having automated retention and disposition, but noone even at large banks and law firms are able to come up with the policies.

Always consider using RBS for databases larger than 200GB, and note that RBS also helps you meet the disk IOPS requirements of SharePoint.

How to Debug SharePoint Solutions in a Multi-Server Farm

Some tips on deploying and debugging code in a multi-server SharePoint 2010 farm:

When debugging on a multi-server SharePoint farm, with Visual Studio on the app-server, then add AAM mapping for the app-server to the web-application to debug, otherwise VS can't attach to the local w3wp process.

For example, to debug the web-app hosted on

• http://azure-sp2010web:8383/ 

you must add the app-server URL first to AAM and then use it as Site URL when debugging

• http://azure-sp2010app:8383/

Make sure to browse the site using the added app-server URL to load the code in a local w3wp process.

If the breakpoints have yellow warning triangles, then VS could not load the correct code to the w3wp processes attached to the debugger. Solve by rebuild and deploy to get the latest bits into the [14] hive and GAC. Note that you can’t activate WSPs on deploy in a multi-server farm, set the "Active Deployment Configuration" to “no activation” in Visual Studio project properties. If VS still can’t deploy the WSP, then use powershell to first Add-SPSolution and then Install-SPSolution across the farm.

Validate the WSP deployment status in “Manage farm solutions” in Central Admin first, and make sure that your feature is activated in the site or subsite you try to debug.

Happy debugging :)

Dynamically Assign Approver for the Content Approval Workflow in SP2010

This post is a how-to guide for customizing the ootb SharePoint 2010 content approval workflow to automatically pick a user from the current list item such as the 'content resposible' field, and assign that user as the approver of the content, using SharePoint Designer. The customization of the content approval using SPD is quite straightforwards except for some less intuitive and misleading options for editing the workflow task process. It also involves publishing and editing the XML config of the workflow to enable using the "Start this workflow to approve publishing a major version of an item" option for automatically starting the approval workflow when the author check-in (submits) the page or document for approval.

To get started, follow the SharePoint Designer Walkthrough: Copy & Modify Publishing Workflow steps 1-4 to make a copy of the "Approval - SharePoint 2010" workflow in the site-collection root. Edit the workflow name to suit your needs and make sure to pick the content type that contains the user field that you will use to auto-assign as the approver in "Pick a base content type to limit this workflow to". Otherwise you won't be able to add a lookup for that content type field. Click OK and save.

Open the saved custom approval workflow and click "edit workflow". Change the name of the "Start Approval Workflow Task" action as you like. Then click on "Parameter: Approvers" to change the "with [these users]" for the workflow action into using an user from the current item that is pending approval. Now, to dynamically assign an approver, you need to click the "Enter participants manually" button.

Then in "Select task participants" click the address book button to open the "Select users" dialog box. Now select "Workflow lookup for a user", which will trigger the "Lookup for person or group" dialog.

Click OK three times, and the start approval workflow action should now look like this:

Now, as a side-effect the comments for the task has been unbound, so you need to click on properties for the workflow action and bind the comments to the "Parameters: Request" again. This will ensure that the text entered in the request field when starting the approval workflow will not be missing when the approver opens the workflow task to approve or reject the pending content.

Click OK two times, and your customized approval workflow with a dynamic approver is almost ready. There are a couple of workflow parameters that are not needed when automatically assigning the approver, these can be hidden from the workflow initiation form so that the author is not confused when the workflow starting form is shown on check-in. Click "Initiation form parameter" in the ribbon and make sure that the "Approvers" and "Expand groups" parameters are not shown during workflow initiation.

Removing the parameters is also an option, it just feels safer to just hide them rather than deleting them. Save and publish your custom content approval workflow as a globally reusable workflow as shown in step 13-15. You can now follow the Configuring Approval in SharePoint steps to use your workflow on a document library that requires content approval, and it will work for the content types that the custom approval workflow is associated with for the selected list.

However, the "Start this workflow to approve publishing a major version of an item" option will be missing when associating the workflow with the document library, even if it is a bonafide approval workflow and even if the list has "Require content approval for submitted items" turned on. This is caused by the workflow being a content type rather than a list workflow, and only list workflows can be configured to start content approval on check-in. Luckily, it is rather simple to change this by editing the XML config file for the published workflow XOML definition using SPD, as shown in Writing Your Own SharePoint Publishing Approval Workflow. I made these changes to the file:

• changed the Category attribute to "List;Language:1033;#ContentType;Language:1033" 
• changed the AllowStartOnMajorCheckin attribute to "true"
• removed the ContentTypeId attribute completely

After directly editing and saving the workflow config file using SPD, the workflow designer will be somewhat out of sync with the XOML definition, due to the removal of the content type and changing of the association category. Still, the actual workflow logic is still working as expected.

You can now associate and test the custom approval workflow on your document library. These are the typical "Version settings" used for the "Pages" list in publishing sites:

Note that the users that are assigned as the approver must be members of the "Approvers" group in publishing sites, or have the right to edit draft items in the document library, otherwise they cannot open the  workflow task to approve or reject the content, even if the user is the owner of the task.

Migrated Content Database gives Unexpected Error for all Publishing Pages

Today I migrated a SharePoint 2010 publishing site content database to our Azure staging farm. All went smooth after using sp_changedbowner on the restored database before adding the content database to the web-application using Central Admin. However, when I tried to browse the publishing site, I got the "an unexpected error has occurred" message. Browsing /_layouts/settings.aspx worked fine and so did browsing "all site content" and the /pages/ list settings.

Using the correlation ID in combination with the ULS viewer lead me to this infamous portal sitemap provider exception:

DelegateControl: Exception thrown while adding control 'ASP._controltemplates_publishingconsole_ascx': Object reference not set to an instance of an object.

PortalSiteMapProvider was unable to fetch current node, request URL: /Pages/Forsiden.aspx, message: Object reference not set to an instance of an object., stack trace:   
 at Microsoft.SharePoint.SPField.GetTypeOrBaseTypeIfTypeIsInvalid(SPFieldCollection fields, String strType)    
 at Microsoft.SharePoint.SPFieldCollection.GetViewFieldsForContextualListItem()    
 at Microsoft.SharePoint.SPContext.get_Item()    
 at Microsoft.SharePoint.SPContext.get_ListItem()    
 at Microsoft.SharePoint.Publishing.Navigation.PortalSiteMapProvider.get_CurrentNode()

Googling a "PortalSiteMapProvider was unable to fetch current node" exception is no fun. You will typically get it in relation to top navigation and related site map providers. I've chased the cause of that error before, and sometimes had to resort to iisreset each night due to the publishing cache going corrupt over time.

This time, luckily, the exception details indicated a problem with the /pages/ list item definition, which led me to How to fix "System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.SharePoint.SPField.GetTypeOrBaseTypeIfTypeIsInvalid" that helped me solve my problem. By looking closer at the list settings for the pages list, I could see that a list column was flagged as invalid (look for the text "Delete this invalid field").

Trying to browse to Site Settings > Site Columns didn't work, but it gave me the enough information to find out what caused the issue and helped me solve it:

Field type AdvancedCalculated is not installed properly. Go to the list settings page to delete this field.

Deploying the missing feature to the staging farm solved the problem. Now it only remains to fix all those absolute URLs entered by the content authors.