Getting Elevated Search Results in SharePoint 2010

I often use the SharePoint 2010 search CoreResultsWebPart in combination with scopes, content types and managed properties defined in the Search Service Application (SSA) for having dynamic search-driven content in pages. Sometimes the users might need to see some excerpt of content that they really do not have access to, and that you don't want to grant them access to either; e.g. to show a summary to anonymous visitors on your public web-site from selected content that is really stored in the extranet web-application in the SharePoint farm.

What is needed then is to execute the search query with elevated privileges using a custom core results web-part. As my colleague Mikael Svenson shows in Doing blended search results in SharePoint–Part 2: The Custom CoreResultsWebPart Way, it is quite easy to get at the search results code and use the SharedQueryManager object that actually runs the query. Create a web-part that inherits the ootb web-part and override the GetXPathNavigator method like this:

namespace Puzzlepart.SharePoint.Presentation

{

    [ToolboxItemAttribute(false)]

    public class JobPostingCoreResultsWebPart : CoreResultsWebPart

    {

        protected override void CreateChildControls()

        {

            base.CreateChildControls();

        }

        protected override XPathNavigator GetXPathNavigator(string viewPath)

        {

            XmlDocument xmlDocument = null;

            QueryManager queryManager = 

              SharedQueryManager.GetInstance(Page, QueryNumber)

                .QueryManager;

            SPSecurity.RunWithElevatedPrivileges(delegate()

            {

                xmlDocument = queryManager.GetResults(queryManager[0]);

            });

            XPathNavigator xPathNavigator = xmlDocument.CreateNavigator();

            return xPathNavigator;

        }

    }

}

Running the query with elevated privileges means that it can return any content that the app-pool identity has access to. Thus, it is important that you grant that account read permissions only on content that you would want just any user to see. Remember that the security trimming is done at query time, not at crawl time, with standard SP2010 server search. It is the credentials passed to the location's SSA proxy that is used for the security trimming. Use WindowsIdentity.GetCurrent() from the System.Security.Principal namespace if you need to get at the app-pool account from your code.

You would want to add a scope and/or some fixed keywords to the query in the code before getting the results, in order to prevent malicious or accidental misuse of the elevated web-part to search for just anything in the crawled content of the associated SSA that the app-pool identity has access to. Another alternative is to run the query under another identity than the app-pool account by using real Windows impersonation in combination with the Secure Store Service (see this post for all the needed code) as this allows for using a specific content query account.

The nice thing about using the built-in query manager this way, rather than running your own KeywordQuery and providing your own result XML local to the custom web-part instance, is that the shared QueryManager's Location object will get its Result XML document populated. This is important for the correct behavior for the other search web-parts on the page using the same QueryNumber / UserQuery, such as the paging and refiners web-parts.

The result XmlDocument will also be in the correct format with lower case column names, correct hit highlighting data, correct date formatting, duplicate trimming, getting <path> to be <url> and <urlEncoded>, have the correct additional managed and crawled properties in the result such as <FileExtension> and <ows_MetadataFacetInfo>, etc, in addition to having the row <id> element and <imageUrl> added to each result. If you override by using a replacement KeywordQuery you must also implement code to apply appended query, fixed query, scope, result properties, sorting and paging yourself to gain full fidelity for your custom query web-part configuration.

If you don't get the expected elevated result set in your farm (I've only tested this on STS claims based web-apps; also see ForceClaimACLs for the SSA by my colleague Ole Kristian Mørch-Storstein), then the sure thing is to create a new QueryManager instance within the RWEP block as shown in How to: Use the QueryManager class to query SharePoint 2010 Enterprise Search by Corey Roth. This will give you correctly formatted XML results, but note that the search web-parts might set the $ShowMessage xsl:param to true, tricking the XSLT rendering into show the "no results" message and advice texts. Just change the XSLT to call either dvt_1.body or dvt_1.empty templates based on the TotalResults count in the XML rather than the parameter. Use the <xmp> trick to validate that there are results in the XML that all the search web-parts consumes, including core results and refinement panel.

The formatting and layout of the search results is as usual controlled by overriding the result XSLT. This includes the data such as any links in the results, as you don't want the users to click on links that just will give them access denied errors.

When using the search box web-part, use the contextual scope option for the scopes dropdown with care. The ContextualScopeUrl (u=) parameter will default to the current web-application, causing an empty result set when using the custom core results web-part against a content source from another SharePoint web-application.

Using SP2010 BCS Resource Files for BDC Model Settings

You've probably seen way too many Business Connectivity Services (BCS) demos using SharePoint Designer (SPD), showing how simple it is to connect to a SQL Server 2008 database and automagically create external content types and operations, with external lists that can be used both to display and update external data.

Have you ever wondered how to manage those data source connection settings across multiple SharePoint 2010 farms? How do you change the SQL Server name and other login information when deploying to your test and staging farm, and then to your production farm without using SPD again? Even good BCS books such as Professional Business Connectivity Services in SharePoint 2010 have very little coverage of BCS external system settings in Central Admin and of how to actually use BDC resource files. There is a nice end-to-end overview in the Migrating Business Connectivity Services External Content Types in SharePoint 2010 article on MSDN, also lacking some of these details.

Data source connection settings are stored as External System properties that can be configured from Central Admin by managing the BDC service application. This can be a bit confusing, as you might run into the "there are no configurable properties" message when trying to manage Settings for an External System. The trick is to remember that these settings are not for the external system per se, but for a specific external system instance aka connection. Chose the External Systems view in the ribbon, click the link of the applicable external system to see the instances, then select the instance and finally click Settings in the ribbon. Change the connection properties and click OK.

In the example data connection I've used the Secure Store Service (SSS) application for the login information because the target database requires SQL Server authentication instead of passthrough integrated Windows authentication.

Instead of manually changing these External System settings whenever deploying a new version across your development, testing, staging and production farms, you can use BDC resource files to apply the settings. This is done by exporting and importing resource files, either using Central Admin, code or Powershell.

Prototype and test your BCS solution on your development farm first, then package your solution into a feature as explained in How to: Deploy a Declarative BDC Model with a Feature on MSDN. Remember to change the model name and external system name, including the entity namespace into durable names, or at least change the entity version, from your SPD prototype when packing your BDC model.

The declarative BDC model is really just some XML stored in a BDCM file. In addition to the BDC model file, SharePoint 2010 also supports using BDC resource files for specific metadata elements that commonly change, such as SQL Server connection configuration. A resource file is really just some XML stored in a BDCR file, that can be merged with the stored model without deleting the existing model and its configuration from the BCS metadata store.

The simplest way to get a BDCR file to start with, is to export the BDC resources using Central Admin. Chose the BDC Models view in the ribbon, then select the applicable model and finally click Export in the ribbon. Set the file type to Resource and select which resources to include in the exported BDCR file, typically properties, and click Export to save the selected set of resources.

Edit the the resource XML to change e.g. the name of the SQL Server (RdbConnection Data Source) and save your changes, using one file per BDC model and target farm. Only the XML for settings that should be updated when applying the resource file need be in the file.

The edited BDC resource file can now be applied to the applicable BDC model in one of your farms, typically when deploying a new version of a model to the staging or production farm. Chose the BDC Models view in the ribbon, then select the applicable model and finally click Import in the ribbon. Click Browse to select the applicable resource file and set the file type to Resource. Then select which resources to import from BDCR file, typically properties, and click Import to load the selected set of resources.

Validate that the correct settings were imported by reviewing the import log warnings, and by reviewing e.g. the settings for your external system instance or the permissions for the BDC model and its external content types.

See How to: Use a Resource File to Specify Localized Names, Properties, and Permissions on MSDN to include an edited BDC resource file in a feature in your Visual Studio 2010 package. Note that BDC models created with SPD cannot be exported from CA as they are not complete. Such declarative BDC models must be exported from SPD.

Secure Store Service: Create Site-Collections avoiding UnauthorizedAccessException

The MSDN forums is full of questions on how to create a new site-collection from code, especially on what permissions are required to avoid the UnauthorizedAccessException "Access is denied" HRESULT: 0x80070005 error. Your code might work fine when running as you on your development server, and of course under the identity of the SharePoint farm account; but not when running under another application pool identity, or as a normal end-user or even as the SHAREPOINT\System account in workflows or event receivers using SPSecurity.RunWithElevatedPrivileges.

The simple answer is that CreateSite requires the same permissions as the SharePoint farm account. You certainly do not want to give all your end-users those permissions, so typically you would do a revert-to-self using RunWithElevatedPrivileges and run the site provisioning code as your SharePoint web-application application pool identity. Alas, adding the app-pool identity to the farm administrator group, giving it "full control" and "run as system" in web-app User Permission Policy, adding it to server local groups such as WSS_WPG or even to SQL Server db_owner and WSS_Content_Application_Pools database roles in the config and content databases, will not provide a configuration that works across all scenarios from web-parts to event receivers, workflows and integration tests.

Always try to assign a managed service account for the SP web-app application pool identity using Central Admin first or run GrantAccessToProcessIdentity in PowerShell. Still, getting access to the content database might not be enough for creating new site-collections, access to the configuration database is also needed for the process account.

The simple solution to this problem is to run all SharePoint web-applications using the farm account. This is definitely not recommended, and the Central Admin Health Analyzer will report the "The server farm account should not be used for other services" error in the security category.

A better alternative is to do a real RevertToSelf using WindowsImpersonationContext rather than the SharePoint RunWithElevatedPrivileges abstraction. This will require that your code knows the login name and password of the farm account. It is not best practice to store these in clear text in your code or config files, so I recommend using the SharePoint 2010 Secure Store Service to provide the farm account credentials to your code. Here's the code to implement impersonation in ASP.NET and the code to get credentials from the default secure store provider to get you started.

Use a Secure Store target application of type "Group" to store the Windows credentials of the farm account. Add only the required app-pool identities as target application "Members" when setting the target application access permissions, and then use RunWithElevatedPrivileges when retrieving the farm account credentials from the Secure Store Service.

So the final solution is this: retrieve the farm account credentials in your code using the app-pool identity, impersonate the farm account using those credentials, and then create a new site-collection from your code.

Note that there are also issues with claims-based authentication (CBA) when adding sites from code. The known workaround is to enable self-service site creation (SSSC) and use SelfServiceCreateSite from code.

Need to provision lists, pages and other stuff to your new site? Check out the SharePoint Site Configurator Feature over at CodePlex. Its a great resource for learning how to configure SharePoint sites.