Showing posts with label Programming. Show all posts
Showing posts with label Programming. Show all posts

Saturday, March 02, 2013

How to Debug SharePoint Solutions in a Multi-Server Farm

Some tips on deploying and debugging code in a multi-server SharePoint 2010 farm:

When debugging on a multi-server SharePoint farm, with Visual Studio on the app-server, then add AAM mapping for the app-server to the web-application to debug, otherwise VS can't attach to the local w3wp process.

For example, to debug the web-app hosted on
  • http://azure-sp2010web:8383/ 
you must add the app-server URL first to AAM and then use it as Site URL when debugging
  • http://azure-sp2010app:8383/
Make sure to browse the site using the added app-server URL to load the code in a local w3wp process.

If the breakpoints have yellow warning triangles, then VS could not load the correct code to the w3wp processes attached to the debugger. Solve by rebuild and deploy to get the latest bits into the [14] hive and GAC. Note that you can’t activate WSPs on deploy in a multi-server farm, set the "Active Deployment Configuration" to “no activation” in Visual Studio project properties. If VS still can’t deploy the WSP, then use powershell to first Add-SPSolution and then Install-SPSolution across the farm.

Validate the WSP deployment status in “Manage farm solutions” in Central Admin first, and make sure that your feature is activated in the site or subsite you try to debug.

Happy debugging :)

Thursday, February 16, 2012

Reusable SPGridView with Multiple Filter and Sort Columns

The venerable SPGridView still has its use in SharePoint 2010 when your data is not stored in a list or accessible as an external content type through BCS. A typical example is when using a KeywordQuery to build a search-driven web-part feeding on results as DataTable as show in How to: Use the SharePoint 2010 Enterprise Search KeywordQuery Class by Corey Roth. Another example is cross-site queries using SPSiteDataQuery.

The SPGridView can use a DataTable as its data source, but several things from sorting arrows to filtering don't work as expected when not using an ObjectDataSource. As Shawn Kirby shows in SPGridView WebPart with Multiple Filter and Sort Columns it is quite easy to implement support for such features.

In this post, I show how to generalize Shawn's web-part code into a SPGridView derived class and a data source class wrapping a DataTable, isolating this functionality from the web-part code itself, for both better reusability and separation of concerns.

First the simple abstract data source class that you must implement to populate your data set:

namespace Puzzlepart.SharePoint.Core
{
    public abstract class SPGridViewDataSource
    {
        public abstract DataTable SelectData(string sortExpression);
 
        protected void Sort(DataTable dataSource, string sortExpression)
        {
            //clean up the sort expression if needed - the sort descending 
            //menu item causes the double in some cases 
            if (sortExpression.ToLowerInvariant().EndsWith("desc desc"))
                sortExpression = sortExpression.Substring(0, sortExpression.Length - 5);
 
            //need to handle the actual sorting of the data
            if (!string.IsNullOrEmpty(sortExpression))
            {
                DataView view = new DataView(dataSource);
                view.Sort = sortExpression;
                DataTable newTable = view.ToTable();
                dataSource.Clear();
                dataSource.Merge(newTable);
            }
        }
    }
}

The SPGridViewDataSource class provides the SelectData method that you must override, and a completed Sort method that allows the SPGridView to sort your DataTable. Note that this class must be stateless as required by any class used as an ObjectDataSource. Its logic cannot be combined with the grid view class, as it will get instantiated new every time the ObjectDataSource calls SelectData.

Then the derived grid view with support for filtering and sorting, including the arrows and filter images:

namespace Puzzlepart.SharePoint.Core
{
    public class SPGridViewMultiSortFilter : SPGridView
    {
        public SPGridViewMultiSortFilter()
        {
            this.FilteredDataSourcePropertyName = "FilterExpression";
            this.FilteredDataSourcePropertyFormat = "{1} = '{0}'";            
        }
 
        private ObjectDataSource _gridDS;
        private char[] _sortingSeparator = { ',' };
        private string[] _filterSeparator = { "AND" };
 
        public ObjectDataSource GridDataSource
        {
            get { return _gridDS; }
            private set
            {
                _gridDS = value;
                this.DataSourceID = _gridDS.ID;
            }
        }
 
        public bool AllowMultiSorting { get; set; }
        public bool AllowMultiFiltering { get; set; }
 
        string FilterExpression
        {
. . .
        }
 
        string SortExpression
        {
. . .
        }
 
        protected override void CreateChildControls()
        {
            base.CreateChildControls();
 
            this.Sorting += new GridViewSortEventHandler(GridView_Sorting);
            this.RowDataBound += new GridViewRowEventHandler(GridView_RowDataBound);
        }
 
        protected void GridView_Sorting(object sender, GridViewSortEventArgs e)
        {
            EnsureChildControls();
            string direction = e.SortDirection.ToString();
            direction = (direction == "Descending") ? " DESC" : "";
 
            SortExpression = e.SortExpression + direction;
            e.SortExpression = SortExpression;
 
            //keep the object dataset filter
            if (!string.IsNullOrEmpty(FilterExpression))
            {
                _gridDS.FilterExpression = FilterExpression;
            }
        }
 
        protected void GridView_RowDataBound(object sender, GridViewRowEventArgs e)
        {
            EnsureChildControls();
            if (sender == null || e.Row.RowType != DataControlRowType.Header)
            {
                return;
            }
 
            BuildFilterView(_gridDS.FilterExpression);
            SPGridView grid = sender as SPGridView;
 
            // Show icon on filtered and sorted columns 
            for (int i = 0; i < grid.Columns.Count; i++)
            {
. . .
            }
        }
 
        void BuildFilterView(string filterExp)
        {
. . .
 
            //update the filter
            if (!string.IsNullOrEmpty(lastExp))
            {
                FilterExpression = lastExp;
            }
 
            //reset object dataset filter
            if (!string.IsNullOrEmpty(FilterExpression))
            {
                _gridDS.FilterExpression = FilterExpression;
            }
        }
 
        public ObjectDataSource SetObjectDataSource(string dataSourceId, SPGridViewDataSource dataSource)
        {
            ObjectDataSource gridDS = new ObjectDataSource();
            gridDS.ID = dataSourceId;
            gridDS.SelectMethod = "SelectData";
            gridDS.TypeName = dataSource.GetType().AssemblyQualifiedName;
            gridDS.EnableViewState = false;
            gridDS.SortParameterName = "SortExpression";
            gridDS.FilterExpression = FilterExpression;
            this.GridDataSource = gridDS;
 
            return gridDS;
        }
    }
}

Only parts of the SPGridViewMultiSortFilter code is shown here, see download link below for the complete code. Note that I have added two properties that controls whether multi-column sorting and multi-column filtering are allowed or not.

This is an excerpt from a web-part that shows search results using the grid:

namespace Puzzlepart.SharePoint.Presentation
{
    [ToolboxItemAttribute(false)]
    public class JobPostingRollupWebPart : WebPart
    {
        protected SPGridViewMultiSortFilter GridView = null;
 
        protected override void CreateChildControls()
        {
            try
            {
                CreateJobPostingGrid();
            }
            catch (Exception ex)
            {
                Label error = new Label();
                error.Text = String.Format("An unexpected error occurred: {0}", ex.Message);
                error.ToolTip = ex.StackTrace;
                Controls.Add(error);
            }
        }
 
        private void CreateJobPostingGrid()
        {
            //add to control tree first is important for view state handling  
            Panel panel = new Panel();
            Controls.Add(panel);

            GridView = new SPGridViewMultiSortFilter();
 
  . . .
 
            GridView.AllowSorting = true;
            GridView.AllowMultiSorting = false;
            GridView.AllowFiltering = true;
            GridView.FilterDataFields = "Title,Author,Write,";
 
  . . .
 
            panel.Controls.Add(GridView) 

            //set PagerTemplate after adding grid to control tree

 
            PopulateGridDataSource();
 
            //must bind in OnPreRender
            //GridView.DataBind();  
        }
 
        protected override void OnPreRender(EventArgs e)
        {
            GridView.DataBind();
        }
 
        private void PopulateGridDataSource()
        {
            var dataSource = new ApprovedJobPostingDataSource();
            var gridDS = GridView.SetObjectDataSource("gridDS", dataSource);
            //add the data source
            Controls.Add(gridDS);
        }
    }
}

Note how the data source is created and assigned to the grid view, but also added to the control set of the web-part itself. This is required for the grid's DataSourceId binding to find the ObjectDataSource at run-time. Also note that data binding cannot be triggered from CreateChildControls as it is too early in the control's life cycle. The DataBind method must be called from OnPreRender to allow for view state and child controls to load before the sorting and filtering postback events

Finally, this is an example of how to implement a search-driven SPGridViewDataSource:

namespace Puzzlepart.SharePoint.Presentation
{
    public class ApprovedJobPostingDataSource : SPGridViewDataSource
    {
        private string _cacheKey = "Puzzlepart_Godkjente_Jobbannonser";
 
        public override DataTable SelectData(string sortExpression)
        {
            DataTable dataTable = (DataTable)HttpRuntime.Cache[_cacheKey];
            if (dataTable == null)
            {
                dataTable = GetJobPostingData();
                HttpRuntime.Cache.Insert(_cacheKey, dataTable, null
DateTime.Now.AddMinutes(1), Cache.NoSlidingExpiration);
            }
 
            this.Sort(dataTable, sortExpression);
 
            return dataTable;
        }
 
        private DataTable GetJobPostingData()
        {
            DataTable results = new DataTable();
            string jobPostingManager = "Puzzlepart Jobbannonse arbeidsleder";
            string jobPostingAssistant = "Puzzlepart Jobbannonse assistent";
            string approvedStatus = "0";
 
            SPSite site = SPContext.Current.Site;
            KeywordQuery query = new KeywordQuery(site);
            query.QueryText = String.Format(
"ContentType:\"{0}\" ContentType:\"{1}\" ModerationStatus:\"{2}\""
jobPostingManager, jobPostingAssistant, approvedStatus);
            query.ResultsProvider = SearchProvider.Default;
            query.ResultTypes = ResultType.RelevantResults;
 
            ResultTableCollection resultTables = query.Execute();
            if (resultTables.Count > 0)
            {
                ResultTable searchResults = resultTables[ResultType.RelevantResults];
                results.Load(searchResults, LoadOption.OverwriteChanges);
            }
 
            return results;
        }
    }
}

That was not too hard, was it? Note that SearchProvider Default work for both FAST (FS4SP) and a standard SharePoint 2010 Search Service Application (SSA).

All the code can be downloaded from here.

Tuesday, January 17, 2012

Simple Feature Files Cleanup using Extension Methods

As every seasoned SharePoint developer knows, deactivating a feature does not remove the files deployed by that feature. The deployed masterpages, web part pages, wiki pages, page layouts, web-part definitions, styling artifacts, etc files will stay in the target libraries - and they will not be overwritten on feature activation. Don't let Visual Studio 2010 trick you into believing otherwise.

You have to delete those deployed files yourself in the FeatureDeactivating event. The classic approach is to delete the files one-by-one, but this is tedious and error-prone. The following is a set of extension methods that allows you to simply do this:

public override void FeatureDeactivating(SPFeatureReceiverProperties properties)
{
     SPSite site = (SPSite) properties.Feature.Parent;
     properties.Definition.DeleteFeatureFiles("MasterPages", site.RootWeb);
     properties.Definition.DeleteFeatureWebPartFiles(site.RootWeb);
}

The code is an adaptation of Corey Roth's LINQ to XML and Deleting Files on Feature Deactivation, using extension methods and supporting cleanup of specific feature modules and all feature web-parts.

namespace Puzzlepart.SharePoint.Core.SPExtentions
{
    public static class SPFeatureDefinitionExtentions
    {
        public class Module
        {
            public string Name { get; set; }
            public string Path { get; set; }
            public List<string> Files { get; set; }
        }
 
 
        public static void DeleteFeatureFiles
(this SPFeatureDefinition spFeatureDefinition, string moduleName, SPWeb web)
        {
            List<Module> modules = GetModuleFiles(spFeatureDefinition, moduleName);
            foreach (Module module in modules)
            {
                DeleteModuleFiles(module, web);
            }
        }
 
 
        public static void DeleteFeatureWebPartFiles
(this SPFeatureDefinition spFeatureDefinition, SPWeb web)
        {
            List<Module> modules = GetAllModuleFiles(spFeatureDefinition);
            foreach (Module module in modules)
            {
                if (string.Compare(module.Path, "_catalogs/wp"
StringComparison.CurrentCultureIgnoreCase) == 0)
                    DeleteModuleFiles(module, web);
            }
        }
 
 
        private static List<Module> GetModuleFiles
(SPFeatureDefinition spFeatureDefinition, string moduleName)
        {
            string elementsPath = string.Format(@"{0}\FEATURES\{1}\{2}\Elements.xml"
SPUtility.GetGenericSetupPath("Template"), 
spFeatureDefinition.DisplayName, moduleName);
            XDocument elementsXml = XDocument.Load(elementsPath);
            XNamespace sharePointNamespace = "http://schemas.microsoft.com/sharepoint/";
 
            // get each module name and the files in it
            var moduleList =
                from module in elementsXml.Root.Elements(sharePointNamespace + "Module")
                select new
                {
                    Name = (module.Attributes("Name").Any()) 
? module.Attribute("Name").Value : null,
                    ModuleUrl = (module.Attributes("Url").Any()) 
? module.Attribute("Url").Value : null,
                    Files = module.Elements(sharePointNamespace + "File")
                };
 
            List<Module> modules = new List<Module>();
            // iterate through each module with files
            foreach (var module in moduleList)
            {
                Module m = new Module()
                               {
                                   Name = module.Name,
                                   Path = module.ModuleUrl
                               };
                List<string> files = new List<string>();
                foreach (var fileElement in module.Files)
                {
                    string filename = (fileElement.Attributes("Name").Any()) 
? fileElement.Attribute("Name").Value : fileElement.Attribute("Url").Value;
                    files.Add(filename);
                }
                m.Files = files;
                modules.Add(m);
            }
            return modules;
        }
 
        private static void DeleteModuleFiles(Module module, SPWeb web)
        {
            foreach (string filename in module.Files)
            {
                if (!string.IsNullOrEmpty(module.Path))
                    web.GetFile(string.Format("{0}/{1}", module.Path, filename)).Delete();
                else
                    web.Files.Delete(filename);
            }
        }
 
        private static List<Module> GetAllModuleFiles
(SPFeatureDefinition spFeatureDefinition)
        {
            var moduleList = new List<Module>();
 
            string modulesPath = string.Format(@"{0}\FEATURES\{1}\", 
SPUtility.GetGenericSetupPath("Template"), 
spFeatureDefinition.DisplayName);
            DirectoryInfo folder = new DirectoryInfo(modulesPath);
            foreach (DirectoryInfo moduleFolder in folder.GetDirectories())
            {
                moduleList.AddRange(GetModuleFiles(spFeatureDefinition, moduleFolder.Name));
            }
 
            return moduleList;
        }
 
    }
}

Note that page layouts cannot simply be deleted if they are in use. Use code to revert the "GhostableInLibrary" files to the uncustomized (ghosted) feature files on disk in the SharePoint root [14].

Wednesday, October 06, 2010

Secure Store Service: Create Site-Collections avoiding UnauthorizedAccessException

The MSDN forums is full of questions on how to create a new site-collection from code, especially on what permissions are required to avoid the UnauthorizedAccessException "Access is denied" HRESULT: 0x80070005 error. Your code might work fine when running as you on your development server, and of course under the identity of the SharePoint farm account; but not when running under another application pool identity, or as a normal end-user or even as the SHAREPOINT\System account in workflows or event receivers using SPSecurity.RunWithElevatedPrivileges.

The simple answer is that CreateSite requires the same permissions as the SharePoint farm account. You certainly do not want to give all your end-users those permissions, so typically you would do a revert-to-self using RunWithElevatedPrivileges and run the site provisioning code as your SharePoint web-application application pool identity. Alas, adding the app-pool identity to the farm administrator group, giving it "full control" and "run as system" in web-app User Permission Policy, adding it to server local groups such as WSS_WPG or even to SQL Server db_owner and WSS_Content_Application_Pools database roles in the config and content databases, will not provide a configuration that works across all scenarios from web-parts to event receivers, workflows and integration tests.

Always try to assign a managed service account for the SP web-app application pool identity using Central Admin first or run GrantAccessToProcessIdentity in PowerShell. Still, getting access to the content database might not be enough for creating new site-collections, access to the configuration database is also needed for the process account.

The simple solution to this problem is to run all SharePoint web-applications using the farm account. This is definitely not recommended, and the Central Admin Health Analyzer will report the "The server farm account should not be used for other services" error in the security category.

A better alternative is to do a real RevertToSelf using WindowsImpersonationContext rather than the SharePoint RunWithElevatedPrivileges abstraction. This will require that your code knows the login name and password of the farm account. It is not best practice to store these in clear text in your code or config files, so I recommend using the SharePoint 2010 Secure Store Service to provide the farm account credentials to your code. Here's the code to implement impersonation in ASP.NET and the code to get credentials from the default secure store provider to get you started.


Use a Secure Store target application of type "Group" to store the Windows credentials of the farm account. Add only the required app-pool identities as target application "Members" when setting the target application access permissions, and then use RunWithElevatedPrivileges when retrieving the farm account credentials from the Secure Store Service.

So the final solution is this: retrieve the farm account credentials in your code using the app-pool identity, impersonate the farm account using those credentials, and then create a new site-collection from your code.

Note that there are also issues with claims-based authentication (CBA) when adding sites from code. The known workaround is to enable self-service site creation (SSSC) and use SelfServiceCreateSite from code.

Need to provision lists, pages and other stuff to your new site? Check out the SharePoint Site Configurator Feature over at CodePlex. Its a great resource for learning how to configure SharePoint sites.

Friday, June 20, 2008

CAB/SCSF: View Parameters

CAB/SCSF use the wellknown Model-View-Presenter design pattern for its views. There is, however, a design flaw in its implementation: you cannot pass parameters to the view constructor. As documenten in the SCSF knowledge base, the proven practice is to pass the parameters using the WorkItem State collection.

This proven practice can be improved using generics to become type-safe, while still adhering to the MVP design rules. Add the following new method to the WorkItemController base class in the Infrastructure.Interface project:

public virtual TView ShowViewInWorkspace<TView, TParams>(string viewId, string workspaceName, TParams viewParams)
where TParams : class
{
string stateKey = StateItemNames.PresenterConstructorParams + typeof(TView).FullName + ":" + typeof(TParams).FullName;
_workItem.State[stateKey] = viewParams;

TView view = ShowViewInWorkspace<TView>(viewId, workspaceName);
return view;
}

Then add the following new method to the Presenter base class in the Infrastructure.Interface project:

protected TParams ViewParameters<TParams>() where TParams : class
{
//NOTE: must use typeof(view instance) to get correct run-time type of generic view type
string stateKey = StateItemNames.PresenterConstructorParams + _view.GetType().FullName + ":" + typeof(TParams).FullName;
return (TParams)_workItem.State[stateKey];
}

Now you can simply pass the parameters from the module controller to the view using ShowViewInWorkspace and then get the parameters in the presenter's OnViewReady event:

_viewParameters = ViewParameters<ViewParameters>();
if (_viewParameters == null)
{
throw new ArgumentException("Required view parameter error", "ViewParameters", null);
}

This little design improvement hides the gory details of using the untyped object State collection, e.g. generating shared keys and type casting.

Friday, May 02, 2008

SCSF Dependency Injection Container, CAB WorkItem

Getting started with CAB/SCSF can be a bit daunting, and this article is a summary of the most important aspects regarding the mysterious CAB workitem. I've found this to be a good intro to CAB, start with part 18: http://richnewman.wordpress.com/intro-to-cab-toc/

The CAB workitem is simply a dependency injection container, even if "wrongly" promoted as an implementation of a use case. A use case contains no UI-details or service implementation references, but of course these artifacts are needed to realize a use case in an application. The workitem is a container of resources needed to implement a use case.

A DI-container is a collection of resources than can be located using a resource locator that is used to resolve resource injection requests at run-time. The resources can be loaded into the container either dynamically or through XML config. The workitem DI-container has several collections of resources, which can be quite confusing. A workitem has these collections:


  • Items: all objects added to any of the child collections including this, except Workspace.SmartParts; independent of type of resource.
  • WorkItems: child WorkItem objects, must inherit CAB.WorkItem.
  • Services: any object added as a service. NOTE: the container can only contain one instance of each service type.
  • Workspaces: encapsulate a particular visual layout of controls and SmartParts, such as within tabbed pages.
  • SmartParts: any object marked with [SmartPart] in the class definition, do not have to be a visual component.
  • Workspace.SmartParts: visual components shown in the workspace, must inherit System.Windows.Forms.Control. NOTE: these items are not added to the .Items or .SmartParts collections on the workitem.
In addition, the workitem DI-containers can be nested, i.e. a workitem can contain other workitems.

When the DI-container tries to resolve a [ServiceDependency] injection or just Get<IService> for a service, it looks in the nested set of containers from the current workitem and up through the parent workitems, including the root workitem to find the service. The service locator does not look through the set of child workitems. Thus, shared services should be placed towards the root or in the root workitem for the service locator to find them.


One aspect of the workitem Services collection that most CAB developers seems to have missed, is that even if you can add only one instance of a service to a workitem, you still can add separate instances of the same service to child workitems. The service locator will find the nearest service in the tree towards the root. E.g. the ActionCatalog service can be added to several ModuleController first-level workitems, in addition to the root workitem. We have used this feature to add several scope-based services such as a SharedModelCacheSevice and a RibbonControllerService at different levels of the workitem structure. This way, the same service can have different scopes in a CAB solution, dependent on where it is injected. The scope controls how workitem resources are located by the resource locator, e.g. where the locator looks for Services, Commands and UIExtensionSites.

Note that the resource locator strategy only applies to these workitem collections: Services, Workspaces, Commands, EventTopics, UIExtensionSites.

When the SmartClient application starts, it will create the root workitem and add a set of CAB services to it:

public ... class SmartClientApplication<TWorkItem, TShell> ...
{
protected override void AddServices()
{
RootWorkItem.Services.AddNew <ProfileCatalogModuleInfoStore, IModuleInfoStore>();
RootWorkItem.Services.AddNew <WorkspaceLocatorService, IWorkspaceLocatorService>();
RootWorkItem.Services.AddNew <XmlStreamDependentModuleEnumerator, IModuleEnumerator>();
RootWorkItem.Services.AddNew <DependentModuleLoaderService, IModuleLoaderService>();
RootWorkItem.Services.AddOnDemand <ActionCatalogService, IActionCatalogService>();
RootWorkItem.Services.AddOnDemand <EntityTranslatorService, IEntityTranslatorService>();
}
}

These standard CAB services are available from all over CAB as they are added to the root workitem that becomes the ultimate parent of all other workitems loaded into the SmartClient application.

In addition to the standard set of CAB services, the custom CAB modules that get loaded will typically add extra CAB services into the root workitem. SCSF will load CAB modules as defined in ProfileCatalog.XML and this will trigger the CAB module initialization mechanism filling the workitem collections and followed by the DI-container resolving.

What adds to the somewhat complicated workitem collections mechanism is that several of these CAB services themselves are collections of other shared objects. E.g. the EntityTranslatorService is a collection of translators that can be added to and located within that service. Note that the DI-container only performs the resource resolving and dependency injection on objects that are added directly to a workitem collection. Thus, objects not added directly to a workitem collection will not get any DI magic performed on them. E.g. if you create a service agent from within a CAB service to call a web-service, that agent will not be subject to any DI resolving. This can be very confusing when [ServiceDependency] objects are not injected as planned, thus be very careful when designing your solution.

NOTE: do not try to access resources that must be resolved by the DI-container, from within class constructors, as the resource may not have been injected yet. Use the [InjectionConstructor] attribute on your constructor if you need to force the resource locator to resolve and inject services that you need.

Using the DI-Container from CAB Modules

CAB Services are usually implemented using CAB foundation modules, but can also be implemented using CAB business modules. The former module type is applicable for shared CAB services and components, such as modules that provide access to web-services; while the latter module type is suitable for use-case specific CAB components. Note that both types of resources are "services" in CAB as they are added to the workitem Services collection. This overloading of the term "service" should not make you confuse CAB service modules with WCF service gateways, even if the CAB service used the WCF service.

Adding CAB services from CAB modules

A foundation module is a bare-bones module (the original CAB module) as it is just for implementing a container for shared components.

A business module is a module (new in SCSF, extending the original CAB module) enhanced with the application controller pattern as it for implementing a container for use-case specific components. The SCSF module adds the
ControlledWorkItem to separate the use-case related code from the workitem container related code (in CAB you had to put the use-case controller code in the workitem class, mixing the DI-container logic with the use-case logic).

So even if you could implement an AddServices method in the ModuleController and call it from its Run method, I would recommended that you still add module services from Module.AddServices method. This keeps the container related code separated from the use-case related code:

public override void AddServices()
{
//NOTE: must add UserAccessClaimSet first, as NavigationService depends on this service
WorkItem.RootWorkItem.Services.AddNew <UserAccessClaimSet, IUserAccessClaimSet>();
WorkItem.RootWorkItem.Services.AddNew <NavigationService, INavigationService>();
}

Prefer adding services like this over the other mechanisms provided by SCSF (
app.config / [Service] attribute).

Resolving and using CAB services


To be able to use the CAB services they must be gotten from the workitem DI-container. There are two ways of doing this, using the service collection Get<IService> method or using dependency injection with the [ServiceDependency] attribute. The DI way is preferred as the class using the service can then get the service reference once and use it multiple places. The Get<IService> way has the advantage that it can be called with a bool EnsureExists parameter that will throw an exception if the service is not (yet) registered in the workitem DI-container.

Use the [ServiceDependency] attribute either to inject a service into a class property or in the class c'tor:


[ServiceDependency]
public IUserAccessClaimSet Service
{set { _service = value; }}

[InjectionConstructor]
public NavigationService([ServiceDependency] IUserAccessClaimSet userAccessClaimSet)
{
_userAccessClaimSet = userAccessClaimSet;
}

Getting the service directly is more explicit and allows for using the EnsureExists parameter:

IUserAccessClaimSet costOfGasService = WorkItem.Services.Get<IUserAccessClaimSet>(true);

Without the EnsureExists parameter, the returned service can be null and your code will then not throw an exception when getting the service, but rather on the first service invokation later on, which can make it harder to diagnose the actual cause of the failure. Using the [ServiceDependency] attribute has the same weakness, and this is a common cause of CAB perplexity.
Still, I would recommend using the [ServiceDependency] attribute.



Using the DI-container from CAB views

All CAB views exist within a CAB business module and must thus use one of the recommended methods for resolving service dependencies, preferring the [ServiceDependency] attribute. CAB views are designed using the 'supervising controller' MVP pattern, and the presenter object is added to the workitem Items collection like this:

_presenter = _workitem.Items.AddNew<TaskViewPresenter>();

Adding it to the Items collection of the workitem DI-container triggers the dependency injection resolver.

Wednesday, April 23, 2008

Generic INotifyPropertyChanged Setter

Those of you who have ever implemented the INotifyPropertyChanged interface to enable databinding in your domain objects will have noticed how the notify code repeats over and over and over again in your property setters. This sure feels like breaking the DRY principle, so it is time to simplify and remove the repeating code using reflection and generics:

protected void SetValueWithChangeNotification<T>(ref T oldValue, T newValue)
{
StackFrame stackFrame = new StackFrame(1);
string stackName = stackFrame.GetMethod().Name;
string propertyName = stackName.Substring(4); //starts with "set_"
//PropertyInfo property = this.GetType().GetProperty(propertyName);

//T oldValue = (T)property.GetValue(this, null);

if (oldValue.Equals(newValue)==false)
{

oldValue = newValue;
RaisePropertyChanged(propertyName);
}

}

Call the method from your property setters like this:

public decimal NominatedValue
{
get { return _nominatedValue; }
set { SetValueWithChangeNotification(ref _nominatedValue, value);}
}

Note that while it is possible to use property.GetValue(this, null) to get the old value, you cannot use property.SetValue(this, newValue, null) to set it as it would cause a infinite recursive call chain.


The private property member is set before raising the property changed event, just to be sure. WinForms UI code runs on a single thread, so no event subscribers should be able to handle the event until the code returns, but this way even multi-threaded code should work fine.

You can always implement the method in a base class if you like, or in a static utility method by passing the object instance as an extra parameter.

Wednesday, April 16, 2008

WCF: Client Domain Objects, Decorator Pattern

The client domain objects (aka business entity - BE) must be related to the data contracts provided by the web-services. Still, they will need to be enhanced with e.g. dirty-tracking, validation, state, property conversions (xs:date), business rules, and other aspects needed to produce a fully working client application. This article describes how to achieve this when sharing domain objects across layers is not an option (i.e. interop with JEE services), and the same applies to AOP.

One option for implementing the BE objects, described in this article, is to apply the Decorator pattern to the WSDL scema-based DTO objects to enhance them into full-fledged domain objects. Read ‘
Design of Client Domain Objects’ for an intro this option and the related Mapper option.

If you do not like this Decorator option due to the weak [XML attribute] link to the data contracts (see end of article), you can always use a derived decorator class instead of a partial class. Make all your BE enhancements in the derived class using a combination of DTO overrides and new domain logic. If you prefer explicit, debuggable code, you're better off using the Mapper option.

Service Proxy

Use the SVCUTIL.EXE /REFERENCE switch when generating the proxy code and messages to reuse your BE objects as DTOs and to avoid getting new, duplicate DTO objects generated (might not compile due to name clashes). Alternatively, just remove the DTO code from the generated proxy. Make sure that the proxy code reference the .NET namespace and interface assembly that contains your BE objects.

Business Entities

You can use the data contract DTO objects generated by SVCTIL.EXE /DCONLY to get started. This will give you data-binding enabled (/EDB) anemic entity objects with all the properties in place, annotated with XML serialization attributes. Just ensure that the output files from SVCUTIL.EXE do not overwrite your code. The DTO objects become your generated BE objects - ready for decoration.

The BE objects must be manually maintained whenever you make changes to the generated BE code, e.g. to add EntLib3 validation attributes on the properties. Keep your additions to the generated BE in a partial class as far as possible, e.g. add new properties and methods in a partial class extending the decorated class. If all your decorator code can be isolated into a partial class, then the generated BE code can be re-generated without losing your extended BE code.

Our programming model for decorator-based client domain objects is based on using two partial classes; one T.schema.cs class file for all code based originally on the data contracts, and one T.cs class file for all added aspects. T is the type name of the BE.

Note that BE constructors will not be called during deserialization. Create a method to be used as a surrogate constructor to be run after deserialization and mark it with [OnDeserialized]. This is a good place for resetting e.g. the dirty flag:

[OnDeserialized]
void DeserializationCtor(StreamingContext context)
{
_isDirty = false;
}

Your BE class must still have a constructor to initialize the object for normal use. The constructor must initialize all properties that must have a defined initial value. In addition, the c'tor must initialize all child objects that are exposed as (de-normalized) public properties, to avoid null reference exceptions when using the BE (make it simple to use the BE). This also applies to child collections; create the collection to make it ready for use.

public NominationType()
{
//NOTE: WILL NOT BE CALLED ON DE-SERIALIZATION
//initialize all normalized valueobjects with members
this.NominatedValueField = new UnitValueType();
this.NominatedValueField.Unit = String.Empty;
}

Also initialize all fields that are required by the XSD schema, failure to initialize these fields will give serialization errors:
[DataMemberAttribute(IsRequired=true, . . .) ]

Don't break the DRY principle by adding the value object initialization code multiple places.

Dirty Tracking

If your BE implements INotifyPropertyChanged to enable data-binding, then the event is a good place to add dirty-tracking: Simple Dirty Tracking in Domain Objects

Converting XML Data Types to/from Strongly Typed Properties

Note that xs:date becomes string in .NET as there is no Date only type, just DateTime. Do not let this kind of weak type design leak into your BE objects. Contain the weak typing inside your BE object by converting to/from strongly typed properties.

private string _demandDateData; //XML xs:date datatype

public DateTime DemandDate
{
get
{
return DateTime.ParseExact(this._demandDateData, WSDateFormat, null);
}
set
{
string xsDate = value.ToString(WSDateFormat);
if (this._demandDateData.Equals(xsDate) != true)
{
this._demandDateData = xsDate;
this.RaisePropertyChanged("DemandDate");
}
}
}

This keeps your BE object within the "Simple vs Easy" principle.

De-normalization of BE Value Objects

The data contracts provided by the web-services will most likely contain some DDD value objects for numeric types annotated with some related attributes (e.g. money with currency) and other similar cases. These value objects will become child objects of the BE object, which is not very data-binding friendly as most controls (grids, etc) needs the data to be non-nested properties of the bound object. In addition, one control can only have one binding source. Thus, unlike master-details type binding that can be resolved using multiple binding sources, nested value objects must be de-normalized and projected as normal properties directly on the BE objects.

Start by making the value object private and give the property a new name (e.g. suffix with Data) while specifying the original name in the serialization attribute to keep the link to the XML. Keep the RaisePropertyChanged parameter equal to the property name.

[DataMemberAttribute(Name = "NominatedValue", IsRequired = true, EmitDefaultValue = false, Order = 2)]
private UnitValueType NominatedValueData
{
get { return this.NominatedValueField; }
set { . . . }
}

Then create the de-normalized public properties (in a partial class) as required by the BE and data-binding needs. The new public properties should be wrappers exposing an element of the original private member value object, using a friendly name and the correct data type. Make sure that the new-old value comparison is correct. Keep the RaisePropertyChanged parameter equal to the new property name.

public Decimal NominatedValue
{
get { return this.NominatedValueField.Value; }
set
{
if (this.NominatedValueField.Value.Equals(value) != true)
{
this.NominatedValueField.Value = value;
this.RaisePropertyChanged("NominatedValue");
}
}
}

public string NominatedValueUnit
{
get { return this.NominatedValueField.Unit; }
set { . . . this.RaisePropertyChanged("NominatedValueUnit"); }
}

As can be seen from the example, the new public properties are named by concatenating the value object name and the exposed property name; except for when the property name is just repeating the last part of the value object (i.e. NominatedValue.Value).

Remember to initialize all the value object children in the BE c'tor; both create the child objects and set their initial values.

Keeping the BE Synchronized with the Data-Contract Schema

It is imperative that the XML serialization attributes in code are correct according to the data contract XSD when using the decorator approach on proxy DTO objects to implement BE objects. There are three typical changes that can break the mapping between the BE code and the payload XML:

  • Changes to the code by renaming properties without updating the XML serialization attributes accordingly
  • Changes to the data contract XSD schema without adding properties with correct XML serialization attributes
  • Changes to the data contract XSD schema without updating the XML serialization attributes accordingly
The first issue can happen when e.g. re-exposing a property by renaming and hiding the original property (such as for de-normalizing a value object). This will require updating the XML serialization attribute to keep the mapping of the code to XSD correct:

[DataMemberAttribute(Name = "NominatedValue", . . .) ]
private UnitValueType NominatedValueData

The next two issues are caused by adding new items to the data contract or changing some existing items. The latter is not recommended and is definitively not a recommended practice for published contracts:

  • It is a proven practice that published contracts are not changed, except for non-breaking changes in minor versions.
  • For new, non-breaking versions of the contract, items must only be added at the end of the existing contract.
When a new minor version of a published data contract must be incorporated into your BE class, then start by adding the properties to hold the new data items, then add the applicable XML serialization attributes with the correct sequence order value:

[DataMemberAttribute(. . . , Order=<order number in XML xs:sequence element for item>) ]

For new major versions of data contracts, you must consider creating a new version of the BE class in a new .NET namespace. This allows you to do controlled versioning of your domain objects in synchronization with the versioning policy of the service contracts.

However, during the development of the web-services, the contracts are bound to get breaking changes as the data contracts evolve (which is OK as they are still not published). For this reason, wait as long as you can to make changes to the generated BE code, stick to working on the partial class used to decorate the BE. This will allow you to re-generate the schema-bound parts of your BE to keep up with the changes in the data contracts.


A side note: deriving your BE objects from the DTO objects is an alternative to partial classes that strictly enforces the separation of decorator code from generated code, and allows the generated code to be auto-generated over and over again - like for the Mapper option. Alas, there are drawbacks to inheriting the DTOs also, as none of the generated proxy artifacts are geared towards this. Look into using an AOP framework to do dynamic subclassing if chosing to derive your BE objects.

For data contracts under development, any combination of adding, changing, removing and reordering of data contract items can be expected to occur. The XML serialization attributes must then be updated accordingly.

Tuesday, April 15, 2008

WCF+SCSF: Client Domain Objects, Mapper Pattern

The client domain objects (aka business entity - BE) must be related to the data contracts provided by the web-services. Still, they will need to be enhanced with e.g. dirty-tracking, validation, state, property conversions (xs:date), business rules, and other aspects needed to produce a fully working client application. This article describes how to achieve this when sharing domain objects across layers is not an option (i.e. interop with JEE services), and the same applies to AOP.

One option for implementing the BE objects, described in this article, is to apply the Mapper pattern to the WSDL scema-based DTO objects to transform them into domain objects. Read ‘Design of Client Domain Objects’ for an intro this option and the related Decorator option.

Service Proxy

The mapper option has no special requirements for the service proxy or its message and data contracts, as there are no shared objects between the proxy and the business entities, i.e. the proxy is totally self-contained. The mapper objects do all transformation to/from the DTO objects and the BE objects.

CAB EntityTranslatorService, CAB EntityMapperTranslator

The EntityTranslatorService class is a service that provides a registry of translators and translation services between two classes. The user must implement the translators and register them with the service.

For more information see: ms-help://MS.VSCC.v80/MS.VSIPCC.v80/ms.practices.scsf.2007may/SCSF/html/03-01-090-How_to_Translate_Between_Business_Entities_and_Service_Entities.htm

The EntityMapperTranslator class is a base helper implementation of an IEntityTranslator that provides placeholders to translate from business entities to service and viceversa.


For more information see: ms-help://MS.VSCC.v80/MS.VSIPCC.v80/ms.practices.scsf.2007may/SCSF/html/03-01-090-How_to_Translate_Between_Business_Entities_and_Service_Entities.htm

Mappers and the Translator Service

A mapper must be derived from the CAB class EntityMapperTranslator<BE, DTO> and implement two methods:

DTO = BusinessToService(BE)
BE = ServiceToBusiness(DTO)

A mapper must traverse the complete object graph to translate all child collections and objects when the mapped object is a DDD "aggregate root".

Always add a mapper for collections of the mapped DTO and BE objects. A list mapper must iterate the collection and use the applicable object translator to map each item in the list.

Each module must register its own mappers with the CAB translator service from the CAB module AddServices:

public override void AddServices()
{
AddEntityTranslators();
_workItem.RootWorkItem.Services.AddNew<MyService, IMyService>();

}

private void AddEntityTranslators()

{
IEntityTranslatorService translator = _workItem.RootWorkItem.Services.Get<IEntityTranslatorService>();

translator.RegisterEntityTranslator(new MyTranslator());
translator.RegisterEntityTranslator(new MyListTranslator());

. . .
}

The mappers are used by the service agents to map DTO objects to BE objects and vice verca. Each CAB service that depends on a service agent must have a [ServiceDependency] on IEntityTranslatorService:

[InjectionConstructor]
public MyService([ServiceDependency]IEntityTranslatorService translator)
{
_translator = translator;
_myServiceAgent = new MyServiceAgent(_translator);
}

MyService is a CAB service and will thus get the EntityTranslatorService injected by the DI-container. It then creates the required service agents using the translator service reference. MyServiceAgent is not a CAB service, thus it is not subject to any DI-container dependency resolving magic.

Service Agents Perform the Mapping

The service agents use the EntityTranslatorService to map the DTO objects returned by the service proxy into BE objects:

AllocationTypeList allocationTypeList;
NominationTypeList nominationTypeList = _proxy.GetNominationAndAllocation (periodType, out allocationTypeList);

nominationList = _translator.Translate <List<Nomination>>(nominationTypeList);
allocationList = _translator.Translate <List<Allocation>>(allocationTypeList);

The reverse mapping must be performed on all BE objects passed as DTO objects into the service proxies.

NEVER LET A DTO OBJECT BE USED OUTSIDE SERVICE AGENTS.

Mapping from DTO to BE

Your BE class must have a constructor to initialize the object for normal use. The constructor must initialize all properties that must have a defined initial value. In addition, the c'tor must initialize all child objects that are exposed as public properties, to avoid null reference exceptions when using the BE (make it simple to use the BE). This also applies to child collections; create the collection to make it ready for use. Don't break the DRY principle by adding the initialization code multiple places.

public Nomination()
{
//initialize all contained valueobjects with members
this.NominatedValueField = new UnitValueType();
this.NominatedValueField.Unit = String.Empty;
. . .
}

The mapper code will create a new instance of the BE class and initialize it based on values from the DTO object. This can involve the whole specter from simple value copying to complex conversion logic.

Mapping from BE to DTO

Your mapper class must initialize the created DTO object before using it. The mapper must initialize all properties that must have a defined initial value. In addition, the mapper must initialize all child objects that are exposed as (de-normalized) public properties, to avoid null reference exceptions when mapping the BE. This also applies to child collections; create the collection to make it ready for use.

//initialize all normalized valueobjects with members
dto.NominatedValueField = new UnitValueType();
dto.NominatedValueField.Unit = String.Empty;

Also initialize all DTO fields that are required by the XSD schema, failure to initialize these fields will give serialization errors:

[DataMemberAttribute(IsRequired=true, . . .)]

The mapper code will create a new instance of the DTO class and initialize it based on values from the BE object. This can involve the whole specter from simple value copying to complex conversion logic.

Converting XML Data Types to/from Strongly Typed Properties

Note that xs:date becomes string in .NET as there is no Date only type, just DateTime. DO NOT LET THIS KIND OF WEAK TYPE DESIGN LEAK INTO YOUR BE OBJECTS. Contain the weak typing in your DTO object by converting to/from strongly typed properties in the mapper.

dto.DemandDate = value.DemandDate.ToString(WSDateFormat);

entity.DemandDate = DateTime.ParseExact(value.DemandDate, WSDateFormat, null);

This keeps your BE object within the "Simple vs Easy" principle.

Dirty Tracking

If your BE implements INotifyPropertyChanged to enable data-binding, then the event is a good place to add dirty-tracking: Simple Dirty Tracking in Domain Objects.

Monday, April 14, 2008

Simple Dirty Tracking in Domain Objects

This article shows how to add dirty-tracking to domain objects (aka business entity - BE) by changing their code and adding a public bool IsDirty property. This design is simple and intrusive, and is best suited when you have to implement your BE objects yourself anyway. It requires no usage of BE interfaces, deriving and overriding, AOP, nor any DI-container.

If you cannot change the BE code, then consider implementing non-intrusive logic to compare a unit-of-work BE with its original values; either by reading the original BE from the repository and perform a recursive Equals(originalEntity) check of the BE's object graph, or by using an extended identity map that has original copies of the unit-of-work BE objects and performing the same Equals(originalEntity) check. The latter approach is better for client-side dirty-tracking; while the former is better suited for server-side implementations (get - compare - save).

Never implement dirty-tracking in client applications by using click events and the like, to set the dirty flag from view or presenter code. Implementing dirty-tracking in client code breaks both the SRP and DRY principles, first by making dirty-tracking a responsibility of the view/presenter instead of the BE object, and then by repeating dirty-tracking code across all places where the BE object is used.

If your BE objects implements INotifyPropertyChanged to enable data-binding, then the event is a good place to add dirty-tracking. Note that this dirty-tracking mechanism depends on all changes to the BE state being done through the property setters. Luckily, the _isDirty flag can also be set directly from anywhere within the BE code if necessary.

protected void RaisePropertyChanged(string propertyName)
{
//NOTE: must be here to run even if no handlers are registered
this._isDirty = true;

PropertyChangedEventHandler propertyChanged = this.PropertyChanged;

if (propertyChanged != null)
{
propertyChanged(this, new PropertyChangedEventArgs(propertyName));
}
}

If your BE is a DDD "aggregate root" and thus contains a complex object graph, then your BE should implement an IsDirty method, in addition to the private _isDirty member, that iterates through all child objects and checks their IsDirty status. Each object within the aggregate root must implement its own dirty tracking mechanism; either as just an IsDirty property for simple objects, or both an _isDirty flag and an IsDirty method for iterating its children also for complex objects.

public bool IsDirty()
{
if (this._isDirty) return true;

bool isDirty = this._isDirty;
foreach (Allocation allocation in _allocationList)

{
if (allocation.IsDirty) isDirty = true;
break;

}
return isDirty;

}

You could add event handlers to listen for the INotifyPropertyChanged event from contained child objects, but this is not recommended as you would need to add and remove a dynamic number of event handlers based on how many objects there are inside the aggregate root throughout its complex object graph - also when objects are added or removed from child collections.

Thursday, April 10, 2008

Simple vs Easy Revisited

New client, new project, new developers - same old story on code monkey approach to designing and implementing objects.

I am again gently enforcing these two simple rules:

The objects must be simple to use, not (just) easy to make. E.g. do not expose a date as string even if XML xs:date types becomes strings in .NET and strings are easy to code for you - rather make it simple to use the object by converting the string to and from a real .NET DateTime value.

There is no such thing as "throw-away code" if it enters the source control system. You will not have time, due to time preasure, to come back at a later time to do things right in throw-away code that you write due to time preasure. Therefore all code that you check-in must be production quality code.

In addition, I try to enforce this principle: There should be no green code (comments) in the code. All comments should be considered a missed opportunity to use 'Refactor - Extract Method' and use of the comment as the basis for the new method name.

Friday, June 01, 2007

Enum Flags, Specification Pattern & Explicit Semantics

We use the specification pattern for all the dynamic query operations in our WCF service, and this includes using Nullable<T> on all criteria elements and several "multiple choice" criteria based on [Flags] enumerations. To ensure that our contracts convey explicit semantics, we have used long meaningful names in the enumerations instead of the more cryptic codes that are wellknown in the internal system and that are the domain terms used by the biz-people. These internal codes is not, however, very meaningful to the partners that are consuming the WCF service.

As the multi-choice lists sometimes contain 20+ items, I feared that interpreting the [Flags] enum
and building the repository query would become huge switch statements with lots of ugly bit-wise "and" logic to deduce which of the 20+ items were specified in the criteria. In addition, I needed a way to add the translated code items to a SQL @parameter list as part of a SQL in clause to actually filter records on the criteria.

As I never embark on implementing something ugly, I decided to look for a simpler way of interpreting the [Flags] enum. Knowing that an enum internally is represented by the integers and not the human-friendly names, I decided to add another internal enum that exactly mirrors the contract enum, just using the internal domain codes, and then cast to the internal enum. But the DataContractSerializer has a simpler approach using the [EnumMember] attribute:

[DataContract]
[Flags]

public enum DisciplineFlags
{
None = 0,
. . .
[EnumMember(Value = "
Instrumentation ")]
I = 0x000100,
[EnumMember(Value = "
MarineOperations")]
J = 0x000200,
[EnumMember(Value = "
Materials")]
M = 0x000400,
[EnumMember(Value = "
Navigation")]
N = 0x000800,
//
[EnumMember(Value = "
Process")]
P = 0x001000,
[EnumMember(Value = "
QualityManagement")]
Q = 0x002000,
[EnumMember(Value = "
Piping")]
S = 0x004000,
[EnumMember(Value = "
Telecommunications")]
T = 0x008000,
. . .
}

Note that you must apply the [DataContract] attribute to the enum for the [EnumMember] to take effect. If you just use the plain enum as a [DataMember] property, then the internal names will be published in the contract.

The "None" item is there as the default value for the DataContractSerializer when the incoming XML specification contains no flags filter element (xsi:nil="true"). The nullable flags filter must be asserted like this before usage:

if (filter.DisciplineFlags.HasValue
&& filter.DisciplineFlags != DisciplineFlags.None)

{
. . .
}


So now I had the real one-letter domain codes, but how to avoid bit-wise interpretation ? Also, I needed a list of the codes making up the flags combination. The answer is really simple:

string disciplineList = filter.DisciplineFlags.ToString();
string[] inValues = disciplineList.Split(new char[] {','}, StringSplitOptions.RemoveEmptyEntries);

The last problem is that you cannot add the list of values for an in-clause as a @parameter. Just concatenating the CSV-string into the SQL is not an option due to SQL-injection attacks. So a for-loop is needed to add SQL parameters and related values:

for (int i = 0; i < inValues.Length; i++ )
{
string param = "@in" + i;
if (i > 0) sql += ", ";
sql += param;
filterCommand.Parameters.AddWithValue(param, inValues[i].Trim());
}

I would rather have avoided the for-loop, but at least the specification interpreter became much simpler than I expected it to be. No pesky switch statements and bit-wise logic.

See the end of this earlier post for further details about implementing specification filters using TableAdapters.

Wednesday, February 21, 2007

Effective use of TableAdapters and DataSets

Some of our customers are not ready for using an O/R Mapper as the data access layer, so I have to do the boring job of implementing it manually the old-fashioned way (sigh...). I then use VS2005 TableAdapters for implementing the domain object persistence using only the wizards and thus mostly generated code for the DAL.

TableAdapters are code-generated extension to typed datasets; and even if I am no fan of datasets, as long as the stay in the black-box, I can live with them. In addition to the auto-generated TableAdapters, I must manually create adapters that map my domain objects to/from the typed datasets to complete the repository component. This is where the effort goes, I have never written less SQL statements since using TableAdapters - and as a DBA, I actually like writing and tuning SQL statements.

I have been using TableAdapters since fall 2005, and this post is about effective use of TableAdapters and their DataSets. To illustrate my recommendations, I will use the DataSet shown in this figure (click to enlarge):


The central entity table in the dataset is 'DocumentCard', which is an aggregate root that contains a file-collection table (identifying relationship) and some extra information tables that are needed in the 'DocumentCard' domain object. The document cards are structured into a category treeview, using the 'DocumentCategory' entity. The TableAdapters options are configured to make the DataSets updatable as applicable for each table:


The Dataset objects are connected to each other using Relation objects, and together with the generated CRUD SQL statements, this makes for reading and easily modifying data the typed dataset way. Applying a filter when reading data using a combination of a single query and the specification pattern is not as easy, but I shown how this can be done towards the end.

I recommend that the default query of the table adapter is used only as a 'schema definition' query for the dataset, i.e. the query used for defining the table and for generating the CRUD statements that make the typed dataset updatable so that TableAdapter.Update( DataSet) automagically modifies the database to reflect the changes to the dataset. I give the default query a name like 'FillByNoFilter' (perhaps 'DefineSchemaQuery' would be better), and then add a separate 'Fill' query for use by the code. This way the table schema and the C_UD statements stay constant when the inevitable changes affect the logic of the data access layer.

Note that changes to the default query are immediately propagated by the table adapter mechanism to all other queries in the TableAdapter to ensure that they all adhere to the same schema. E.g. if you remove a field from the schema, all queries are automatically updated to have the same SELECT as the default query. This has a rather nasty side-effect on a specific type of scalar type queries, e.g. the 'LookupCardIdFromKey' method from the above example; it will just replace the select statement returning the scalar value with the default select statement returning a record set. This only happens to scalar queries that do not use a SQL function, and I recommend applying the MAX() function to any such query the return just a field instead of a computed value.

I recommend using "info" as the suffix for read-only datasets that are aggregated into a domain object. In the above example, status info about the physical files is aggregated from the FileHandler file management system; e.g. file upload progress. The info-tables have no generated C_UD SQL statements as they are read-only. I do not recommend aggregating this kind of info-table into the root-table of the entity as "assimilated" fields, as this quickly leads to the TableAdapter not being able to generate the C_UD statements for the dataset. You could of course add the info fields manually to the dataset and set their read-only property to true, but as you add more queries to the table adapter, you will have a hard time making the returned data fit the dataset schema - and not the least, ensuring that the fields are filled with data.

The relations of the data set are useful for more than just defining the structure of the tables making up the data set. The way that a typed dataset automatically populates the relations when filling the related tables and then exposes a relation as a collection, makes it possible to keep the number of database queries to a minimum when filling a complex object graph such as the three level deep annotated structure shown in the above example.

A typical implementation of filling a treeview with structured master-details data for a specific customer, would be fetching the nodes and then for each node fetch the related data. I call this read approach recursive-chunked. If there are N nodes, there would be N+1 database reads; one for the node table, and recursively one read for each node to read the related table in chunks. In my example, there are five tables, and with a large N, the number of database reads would explode using such a fill implementation.

I recommend that each table in a relation structure has a query that takes the aggregate root 'read specification' as parameters and returns non-chunked datasets, leaving it to the typed dataset to fill the relations and automatically provide the structure. This way, if there are M tables, there will be only M database reads to fill the complete structure. If there is e.g. two tables the recursive-chunked will use N+1 reads, while the non-chunked will always use 1+1 reads. Using the relation mechanism of typed datasets of course incurs some overhead, but as the use of TableAdapters implies use of typed datasets, why not utilize them to the maximum?

In my example the non-chunked implementation would always require 5 database reads. The recursive-chunked approach would hardly ever use fewer reads, e.g. with just a single category with two related document cards would result in 1+1*(1+2*(1+1+1)) = 8 reads. If there was four categories each with six cards each, there would be 1+4*(1+6*(1+1+1)) = 77 reads. So even if the chunked reads each contain less data, the overhead more of database roundtrips will soon dwarf the five non-chunked roundtrips.

The last tip about effective use of table adapters is about how to support the specification pattern while keeping the number of TableAdapter queries to a minimum. Even if the queries are created using wizards and generated code, each extra item is still another item to maintain. My way of doing this is based on my previous recommendation to use the default query as a schema definition and SQL statement provider only, making it the stable information source for the table adapter.

I recommend using the default query as the basis for building a dynamic parameterized SQL statement that filters the record set according to the specification, and dynamically injecting it into the TableAdapter to apply the selection. The custom ORM adapter is the specification interpreter and needs to build a DbCommand object containing a parameterized SELECT statement adhering to the schema of the TableAdapter. Thus, the interpreter needs access to the TableAdapter's private command collection to get the schema definition query. In addition, the adapter needs access to the private DataAdapter to be able to inject and execute the dynamic query to fill the DataSet.

As the needed TableAdapter members are private, you need to utilize the partial class mechanism to create extra internal properties that provides access to the private members. Double-click the design canvas of the dataset class to generate a partial class extending the dataset, then add the properties to the table adapter:

using System.Data.SqlClient;

namespace KjellSJ.DataAccess.DocumentCardDataSetTableAdapters
{
public partial class DocumentCardTableAdapter
{
internal SqlDataAdapter DataAdapter
{
get { return this._adapter; }
}

internal string SelectFromStatement()
{
return this.CommandCollection[0].CommandText;
}
}
}

namespace KjellSJ.DataAccess
{
partial class DocumentCardDataSet
{
//empty class generated by double-clicking the dataset design canvas
}
}


Adding specification filters that requires the use of related database tables in the query complicates the logic of the selection interpreter, especially when the filter is Nullable<T>. A nullable filter should only be applied if it is not null, i.e. the .HasValue property is true. Basically you have two options, joining the related table to the table of the defining query, or using sub-selects. As the interpreter should only add the related table to the query when at least one refering filter is defined in the specification, using a sub-select is usually the simplest thing to do.

This is how you can implement a hard-coded interpreter and apply the parameterized selection to the TableAdapter:

namespace KjellSJ.DataAccess
{
public class DocumentCardAdapter
{
public ProjectDocuments ListProjectDocuments( EApprovalSession session)
{
return DoListProjectDocuments(session, null, null);
}

public ProjectDocuments ListProjectDocuments( EApprovalSession session, DocumentCardFilter cardFilter)
{
DocumentCardTableAdapter ta = new DocumentCardTableAdapter();
SqlCommand filterCommand = new SqlCommand();
filterCommand.CommandType = CommandType.Text;

string sql = ta.SelectFromStatement();
sql += " WHERE 1=1";

filterCommand.Parameters.AddWithValue( "@ClientKey", customerKey);
filterCommand.Parameters.AddWithValue( "@ProjectKey", projectKey);

if (cardFilter.IsMapped.HasValue)
{
sql += (cardFilter.IsMapped.Value) ? " AND EXISTS " : " AND NOT EXISTS";
sql +=
" (SELECT * FROM DocumentRequirementMapping . . .)";
}

if(String.IsNullOrEmpty( cardFilter.FreeText)==false)
{
sql += " AND (Code like @FreeText";
sql += " OR Title like @FreeText";
sql += " OR Description like @FreeText";
sql += ")";

filterCommand.Parameters.AddWithValue( "@FreeText", wildcard);
}
. . .

filterCommand.CommandText = sql;

return DoListProjectDocuments(session, filterCommand, cardFilter.ExcludeEmptyCategoryNodes);
}

private ProjectDocuments DoListProjectDocuments( EApprovalSession session, SqlCommand filterCommand, bool? excludeEmptyNodes)
{

DocumentCardTableAdapter taData = new DocumentCardTableAdapter();
. . .
if(filterCommand==null)
{
taData.Fill(ds.DocumentCard, customerKey, projectKey);
}
else
{
filterCommand.Connection = taData.Connection;
taData.DataAdapter.SelectCommand = filterCommand;
taData.DataAdapter.Fill(ds.DocumentCard);
}
. . .
}

Having SQL fragments as text in the code should be avoided, but this is the simplest way to make a table adapter support dynamic criteria. Alternatively, I could have used a stored procedure taking the specifications as parameters and implementing all this logic in the database itself; having a 'FillBySpecification' query using the sproc. Alas, I don't recommend using sprocs for queries, and I hardly ever use sprocs for CRUD operations either. Why ? Well, read the discussions summarized in this post by Frans Bouma and decide for your self.

Following these guidelines should help you get the most out of table adapters while writing almost no SQL statements, and still have a maintainable data access layer. My strongest advice is to follow the 'schema definition' query recommendation, as having a stable foundation for the TableAdapter is a must for ensuring that the dataset CRUD operations stays correct as the adapters evolve.