Friday, August 27, 2010

SharePoint 2010 My Tasks Web Part using Search Driven Cross-Site Query with Muenchian Grouping

There always seems to be a requirement for rolling up data from all sites in one or more SharePoint solutions, such as getting a list of my tasks, a list of new documents this week, or creating a searchable news archive for publishing sites; or such as creating a site map or dynamic site directory based on metadata collected in your site provisioning workflow, that are later maintained by site owners.

SharePoint has several web-parts that can do cross-list and cross-subsite queries, such as the Content Query web-part, but all restricted to a single site-collection. In addition, there are the classic Data View web-part and the new XSLT List View web-parts that can be configured using SharePoint Designer. These web-parts can connect to a diverse set of data sources, from internal SharePoint lists to external REST and OData services.

Still, the simplest solution for cross-site/cross-solution rollups is to customize the ootb search web-parts against custom search scopes in the Search Service application. In most cases, no coding will be required, pure configuration of SharePoint will go a long way. This post will show how to configure a search driven "My Tasks" web-part that will show all tasks assigned to the user across all SharePoint sites across all indexed SharePoint solutions. The unstyled cross-site task rollup web-part looks like this, included some debug info:


First you need to configure the results scope behind the search driven web-part in Central Admin. Start by adding a new scope in 'Search Service Application>Scopes' called TaskRollup using the rules as shown here:


If you can't see ContentType when adding a rule, then go to 'Search Service Application>Metadata Properties' and edit the managed property to set Allow this property to be used in scopes.

As the TaskStatus site column is not mapped to any managed property by default, you must map the crawled property ows_Status to one before it can be used. Go to 'Search Service Application>Metadata Properties' and create a managed property called TaskStatus using the mapping as shown here:


Do not go creative with the naming, stay away from spaces and special characters such as ÆØÅ - a SharePoint best practice for any artifact name used as an identifier or an URL fragment. For example, a name like "Contoso Web Ingress" first gets encoded as "Contoso_x0020_Web_x0020_Ingress" when stored, and then once more encoded as "Contoso_x005F_x0020_Web_x005F_x0020_Ingress" in a search result XML.

A full crawl is required after adding or changing crawled or managed properties. Do a full crawl of the content source you used in the TaskRollup scope. Note that there must be some matching content stored in SharePoint for these properties to be added to the property database in the first place. Thus after provisioning new site content types or site columns, you must add some sample content and then do a full recrawl of the applicable content source.

Verifying that the full crawl of the SharePoint sites content source finished without errors completes the Central Admin configuration. Now it's time to configure the ootb Search Core Results web-part to become the customized My Tasks web-part.

Open a team-site and add the Search Core Results web-part to a page. Switch to page edit mode and select 'Edit Web Part' to open the Search Core Results settings panel. Rename the web-part 'Title' to Task Rollup (cross-site) and set the 'Cross Web-Part Query ID' to User query and 'Fixed Keyword Query' to scope: "TaskRollup" as shown here:


The Search Core Results web-part requires a user query, or a configured fixed or appended query, to actually perform a search. No configured or no user query will just show a message asking for query input. The cross-page query ID setting User query is chosen here for reasons explained later.

If you want to further limit what tasks are shown in the My Tasks web-part, just add more query keywords to the 'Append Text to Query' setting as shown here:


The My Tasks web-part will show the two task fields 'Status' and 'Assigned to' in the task list. Any managed crawled property can be added to the search results by configuring the 'Fetched Properties' setting. Add the following XML <Column Name="AssignedTo"/> <Column Name="TaskStatus"/> as shown here:


You need to uncheck the 'Use Location Visualization' setting to enable the controls for customizing the result set and XSL formatting. See A quick guide to CoreResultsWebPart configuration changes in SharePoint 2010 by Corey Roth to learn more about the new search location concept in SharePoint 2010. Read all his Enterprise Search posts for an excellent introduction to the improved SharePoint 2010 search services and web-parts.

After adding 'TaskStatus' and 'AssignedTo' to the fetched properties, you will also need to customize the XSL used to format and show the search results to also include your extra task fields. Click the 'XSL Editor' button in the 'Display Properties' section of the web-part settings panel, and add the fields to the match="Result" xsl:template according to your design. Note that the property names must be entered in lower case in the XSL.

The astute reader will have noticed the nice grouping of the search results. This is done using the Muenchian method as SharePoint 2010 still uses XLST 1.0, thus no simple XSLT 2.0 xsl:for-each-group. The customized "My Tasks" results XSL creates a key called 'tasks-by-status' that selects 'Result' elements and groups them on the 'taskstatus' field as shown here:


Again, note the requirement for lower case names for the fetched properties when used in the XSL. Use the <xmp> trick to see the actual result XML.

The final part of the puzzle is how to turn the cross-site task list into a personal task list. Unfortunately, the [Me] and [Today] filter tokens cannot be used in the enterprise search query syntax, so some coding is required to add such dynamic filter tokens. Export the customized Search Core Results web-part to disk to start packaging into a WSP solution.

Create a new TaskRollupWebPart web-part SPI in your web-parts feature in Visual Studio 2010. Make the new web-part class inherit from CoreResultsWebPart in the Microsoft.Office.Server.Search assembly. Override the methods shown here to add dynamic filtering of the query through the SharedQueryManager for the web-part page:

namespace PuzzlepartTaskRollup.WebParts
{
[ToolboxItemAttribute(false)]
public class TaskRollupWebPart : 
Microsoft.Office.Server.Search.WebControls.CoreResultsWebPart
{
QueryManager _queryManager;
protected override void OnInit(EventArgs e) {
  base.OnInit(e);
  _queryManager = SharedQueryManager.GetInstance(this.Page).QueryManager;
}
 
protected override System.Xml.XPath.XPathNavigator GetXPathNavigator(string viewPath)
{
  SPUser user = SPContext.Current.Web.CurrentUser;
  _queryManager.UserQuery = string.Format("scope:\"TaskRollup\" AssignedTo:\"{0}\""
user.Name);
  return base.GetXPathNavigator(viewPath);
}
 
protected override void CreateChildControls()
{
  base.CreateChildControls();
  //debug info
  //Controls.Add(new Label { Text = string.Format("FixedQuery: {0}<br/>
AppendedQuery: {1}<br/>UserQuery: {2}", 
FixedQuery, AppendedQuery, _queryManager.UserQuery) });
}
}
}

The code in GetXPathNavigator is what adds the current user to the QueryManager.UserQuery to filter tasks based on the assigned user by [me]. There are five query objects available on a search web-part page, where QueryId.Query1 is the default. This is also what is exposed in the web-part settings as the 'User Query' option. Use the GetInstance(Page, QueryId) overload in SharedQueryManager to get at a specific cross-page query object.

Replace the content of the TaskRollupWebPart.webpart file with the exported Search Core Results configuration. This will ensure that all the configuration done to customize the ootb web-part into the My Tasks web-part is applied to the new TaskRollupWebPart. A small change is needed in the metadata type element to load the new TaskRollupWebPart code rather than the CoreResultsWebPart code:

<webParts>
<webPart xmlns="http://schemas.microsoft.com/WebPart/v3">
<metaData>
<type name="PuzzlepartTaskRollup.WebParts.TaskRollupWebPart, 
$SharePoint.Project.AssemblyFullName$" />
<importErrorMessage>$Resources:core,ImportErrorMessage;</importErrorMessage>
</metaData>

Build the feature and deploy the package to your test site from Visual Studio 2010. Add the web-part to a page and verify that you get only your tasks as expected.

I know that this seems like a lot of work, but a search-driven web-part is easily created and tested before lunch. The inevitable styling & layout using XSL and CSS is what will burn hours, as usual.
 
A drawback of search driven web-parts or code is the delay before new/updated content is shown due to the periodical crawling schedule, typically five or ten minutes. On the positive side, the results will be automatically security trimmed for you based on the logged on user - no authentication hassles or stored username password required as with the XSL List View.

Note that most enterprise search classes are still sealed in SharePoint 2010 as in SharePoint 2007, except the CoreResultsWebPart and some new classes, so you're limited to what customizations can be achieved with configuration or the SharedQueryManager. Search driven web-parts works equally well in SharePoint 2007, except that there is no SharedQueryManager, but rather the infamous search results hidden object (SRHO) which is unsupported.

Recommended: SharePoint Search XSL Samples and the Search Community Toolkit at CodePlex.

32 comments:

Anonymous said...

Hi,

This post was helpful. I have a query on creating custom coreresult webpart. I have created a custom coreresult webpart. when I use the XSL editor to display the raw XML data, it displays the result. But its not able to display the result in the format which core result webpart provides. How can I get this? At present it displays 'blank' instead of displaying the result.

Kjell-Sverre Jerijærvi said...

If you're to use the built-it XSL to format your custom results XML, you must make sure that your XML has the same structure as the standard results XML.

Anonymous said...

What is the problem.
Here :
queryManager.UserQuery = string.Format("scope:\"TaskRollup\" AssignedTo:\"{0}\"", user.Name);
You are searching for Name of user. If you have two that have the same name you get collection of tasks for boths.

Kjell-Sverre Jerijærvi said...

That is expected, as the AssignedTo field is stored as a SPFieldUser in the format "id#;username". The crawler use the last part for the AssignedTo managed property.

The id is just a relative reference to the site-collection specific User Information List, and has no usage in search as it is not a "perma-id".

There are several such issues when using search to handle users and groups, as the crawler does a limited job when indexing the lists. Sorry.

Joel Palmer said...

This was fantastic. With so many moving parts I expected to get halfway though, find that something doesn't work because some gotcha was not pointed out. You clearly pointed out the gotchas. I don't like being at the mercy of the crawl schedule but overall, this is powerful stuff. Thanks!

srikanth sapelly said...

Hi,
I followed the same steps you have mentioned in the post but some how i could not able to get the desired results, the error i am getting here is
"Unable to display this Web Part. To troubleshoot the problem, open this Web page in a Microsoft SharePoint Foundation-compatible HTML editor such as Microsoft SharePoint Designer. If the problem persists, contact your Web server administrator".

Is anything i need to configure apart from the steps you have mentioned?

Your help will be appreciated.

Regards,
S

Kjell-Sverre Jerijærvi said...

No extra steps needed. You need to find out if it is the web-part code or the web-part rendering XSL that fails, e.g. by debugging the code using VS2010.

srikanth sapelly said...

Hi,
I have debugged the code, but it is executing perfectly.
One more thing do i need to replace the provided xslt in the place of "" or adding to the existing one.
Please help me.

Regards,
S

Kjell-Sverre Jerijærvi said...

Use the <xmp> trick to see the result XML to verify that you get results. If so, you need to make your XSL work properly.

srikanth sapelly said...
This comment has been removed by the author.
Kjell-Sverre Jerijærvi said...

If the shown XSL fragment won't work for you, then add your own Muenchian grouping as shown in the linked article.

srikanth sapelly said...

Hi,

Thanks for all your support, is it replacing the OOTB body content xslt with the one that you have provided? or adding it to the existing one?

Regards,
S

srikanth sapelly said...
This comment has been removed by the author.
Kjell-Sverre Jerijærvi said...

This blog post like any other is just guidance, with no support given. Do like Avis, Try Harder :)

Vijay Gande said...

Hi, I get the following error, when built the web part as part of sand box solution. appreciate any hlep.

"Error 2 The base class or interface 'Microsoft.SharePoint.WebPartPages.DataFormWebPart' in assembly 'Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c' referenced by type 'Microsoft.Office.Server.Search.WebControls.SearchResultsBaseWebPart' could not be resolved c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\ISAPI\Microsoft.Office.Server.Search.dll"

Kjell-Sverre Jerijærvi said...

Both assemblies in the error message are standard SP2010 assemblies. You can try adding them to your package, but I built this as a farm solution.

Vijay Gande said...

Yes it works well with Form Solution, my requirement is to build as Sandbox solution!.

Wim Hill said...

Hi,

I came accross this post a couple of months ago and it was very useful.
I was wondring if it would be possible to make a cutom user control of this to avoid the web part overhead when you would use this feature on a portal home page?

If you have any hints on how to implement this with VS2010?

Kind Reagrds,
Wim Hill

Kjell-Sverre Jerijærvi said...

Use Reflector on the CoreResultsWebPart to see how it works, then reengineer those parts that are needed for your functionality.

You can also use the keyword query OM to fetch data and then transform the result XML yourself: http://msdn.microsoft.com/en-us/library/ee558911.aspx

Wim Hill said...

Thanks for your feedback!
I'm looking for exact the same functionality as your web part gives. List all tasks for currently logged in user on all site collections.
I'll give it a try and see how far I get...

Anonymous said...

Hi,

this seems not to be working for me. The problem seems to be the "Task"-definition: my Tasks have different Contenttypes, like "Workflow with Infopath Form" or "TestTask" (i have defined a new task type. How can i get the query to work with those different content types for the tasks?

Sincerely
Christian

Kjell-Sverre Jerijærvi said...

You need to modify the task search scope definition to include your custom task content types. You can add multiple content types to the scope using the "include" option for rule behavior.

Kjell-Sverre Jerijærvi said...

How to manage search scopes and rules: http://office.microsoft.com/en-us/sharepoint-server-help/define-scopes-for-searches-HA010241119.aspx

Anonymous said...

Hi!

Thanks for the quick answer. Does that mean that i have to add every single task-content`-type to my search scope? As far as i see every sp-Designer-created task has it's own content type (which equals the name of the task definition). In that case it would be very laborious to maintain the search scope.

btw: great post, anyway!!! Love you blog!

Greetings
Christian

Kjell-Sverre Jerijærvi said...

You might have to; I guess they all inherit the standard task content type, but I have not tried to make a scope rule that includes a parent content type and all derived children - not sure if that's possible.

Unknown said...

Is there any way to use this method accross web applications. I want to display a list of tasks that the user has asigned to him but on his My Site Profile. Do you think this is possible?

Kjell-Sverre Jerijærvi said...

Yes, provided the crawled user id is the same across the web-apps.

Anonymous said...

Hi again Kjell

Please diregard my previous comment, I have it working now. But my XSLT from the webpart before exporting isn't taking after I deploy it. I can see it in the .webpart file but its not showing up on the actual SharePoint site web part. Anything I can do about this?

Thank you so much for this article, you saved me so much time!
Tudor

Kjell-Sverre Jerijærvi said...

Put the xslt in a file under _layoyts and use the XslLink property to Point to the file. Remember to use iisreset when making changes to such linkes files.

Anonymous said...

Thank you Kjell! I have it working now :) but there's one further question I have. So I am trying to staple this web part using another project to the My Sites templates and I am faced with this error in Visual Studio:

cannot convert from 'TaskRollupWebPart.VisualWebPart1.VisualWebPart1' to 'System.Web.UI.WebControls.WebParts.WebPart'

Does this mean that my webpart I created is not part of the UI.WebControls.... library? Any idea how I could add it there?

Thanks again!
Tudor

Kjell-Sverre Jerijærvi said...

A visual web-part is just a user control that gets loaded by a standard ootb web-part. If your web-part doesn't inherit System.Web.UI.WebControls.WebParts.WebPart then it cannot be used as one.

As your visual web-part is actually working when you add it manually, you need to review the code that you use in your stapler to add the visual web-part to your my site.

Anonymous said...

Sir,
I Get the following error in webpart.
DataSourceId is missing or set to an empty string....