InfoPath text box handling of TAB and other whitespace

Today I got a surprise with regards to how InfoPath handled whitespace in strings. Our data contained text that was nicely aligned using TAB characters and sometimes mono-spaced font with multiple space characters. While testing my form, I noticed that all the formatting was gone when displayed in InfoPath even if the whitespace was preserved in the XML data. All TABs and all sequences of spaces got converted into a single space.

In addition, I could not enter TABs in the text box using the keyboard. The same thing happend with \t characters added to strings in the form code (JScript).

I inspected the properties of the text box, and the only formatting option available is "Paragraph breaks". As my options were limited, I chose that option and re-loaded the XML data. I was mildly delighted when all the alignment stuff in the string data were displayed correctly, including the TAB characters. I can now also enter TABs from the keyboard using CTRL-TAB.

The InfoPath team has blogged about line breaks (CR LF \r \n), and they outline how to make these formatting characters available in rules. Their solution should also be applicable to TAB.

Submitting xsi:nil="true" values from InfoPath to ASP.NET 1.x web services

In my ongoing quest to build an InfoPath solution based on a WSCF web service implemented with ASP.NET 1.1, I have finally been able to:

• make non-string data types such as xs:date and xs:decimal truly optional in InfoPath
• submit data to the web service by modifing the XML data in OnSubmitRequest to circumvent the missing support for nillable in .NET 1.x ASP.NET web services

The first issue is caused by the fact that ASP.NET 1.x web service by design do not support nillable elements in the web service parameters. Check the <wsdl:types> element of any ASMX WSDL, and you will not find any nillable="true" elements even if your data XSD contains such elements. The reason for this is the lack of support for NULL in .NET 1.x value types. ASP.NET 2.0 web services will support nillable="true".

After modifying the underlying XSD schemas of your InfoPath as outlined in my previous post to manually re-introduce the nillable="true" attributes, the fields can be left empty in your form. But you will not be able to sumbit the form to the web service. The SOAP request will fail like this:

The SOAP response indicates that an error occurred:
Server was unable to read request. --> There is an error in XML document (47, 104). --> Input string was not in a correct format.

This is caused by this kind of element in the submitted XML:

<s1:Amount xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"></s1:Amount>

The web service is not able to deserialize this element into a value type due to the lack of support for nillable. Thus, the XML data must be modified before submitting the data to the web service.

InfoPath allows you to use the OnSubmitRequest event to write your own script for submitting the forms data. This event is only available by using Tools-Submitting Forms and selecting "Custom submit using form code" in the 'Submit to' dropdown. Check the 'Edit Form Code' checkbox and click OK to add an event handler for OnSubmitRequest.

This example shows how to look for all instances of a specific element that is NULL, remove the nil attribute and set a dummy value, and finally submitting the form using code:

function XDocument::OnSubmitRequest(eventObj)
{
try
{
var submitDataSource = "Main submit";

//debugger
//get a collection of nodes with xsi:nil="true"
var nodeList = XDocument.DOM.selectNodes("/dfs:myFields/dfs:dataFields//s1:Amount[@xsi:nil='true']");

//iterate the list and
//set values acceptable to ASP.NET 1.x web service
for(var i=0; i < nodeList.length; i++)
{
//get node (element)
var xmlNode = nodeList(i);

//remove nil attribute and
//put zero into the element
//(uses code from InfoPath common.js)
// The xsi:nil needs to be removed before we set the value.
xmlNode.removeAttribute("xsi:nil");
// Setting the value will mark the document as dirty.
xmlNode.text = -1; //biz logic will interpret this as NULL
}

//call the Submit method of the
//primary data source to send the data
XDocument.DataAdapters(submitDataSource).Submit();

eventObj.ReturnStatus = true;
}
catch(ex)
{
XDocument.UI.Alert("Failed while sending the request.\r\n" + ex.number + " - " + ex.description);
eventObj.ReturnStatus = false;
}
}

You will now be able to submit the form's XML data to the web service. The data will of course contain dummy values for all elements that are really NULL (blank), and your 'save' business logic must handle this accordingly. In addition, you will need to add the same type of logic to the form's request operation and your 'read' business logic. Otherwise the users will see the dummy values when viewing existing data from your web service

PS! remember to set the script language of the form before adding any code, as it is not trivial to change the language afterwards. This KB article explains how to change the language, note that you will need to delete all existing scripts.

MSCRM ColumnSet XSD: fields, sorting & filtering

Many of the MSCRM .Retrieve*() methods take an optional ColumnSet string parameter that is normally set to an empty string (""). Most of the samples in the SDK never uses or explains what this parameter is used for, and you might easily think that it is used only for specifying which columns (fields) that should be returned in the result set.

It was not until I needed to sort the result set, that I started to look for how to make the MSCRM services perform sorting, to avoid sorting the result set with a DataView connected to our typed DataSet. By chance, I browsed to the schema of the ColumnSet parameter, and found out that it allows you to specify several clauses for the retrieve operation: select fields, specify sorting, apply filters, etc.

This is an example of how to retrieve a light-weight, sorted and filtered list of sub-accounts:

//fields, sorting, filter
string colset = "<columnset>";
colset += "<column>accountid</column>";
colset += "<column>name</column>";
colset += "<column>emailaddress1</column>";
colset += "<column>ownerid</column>";
//sort on name
colset += "<ascend>name</ascend>";
//only active accounts
colset += "<filter column='statecode' operator='eq' value='" + Microsoft.Crm.Platform.Types.ACCOUNT_STATE.AS_ACTIVE + "' />";
colset += "</colset>";

//retrieve sub accounts
_crmAccount.RetrieveSubAccounts(_crmUserAuth, accountId, colset)

Note that you must specify one or more <column> elements to get data back, as the MSCRM services will return no fields if you do not specify any.

The services do, however, behave very nicely when specifying a column that is a relation to e.g. the biz user that owns an account (ownerid), as not only the owner GUID is returned, but it is also annotated with XML attributes containing the full name of the owner, etc. These extra attributes saves you from doing extra lookups in MSCRM to present human-readable data to the user.

Refer to the SDK for more information about the ColumnSet XML Schema.

Optional numeric fields in InfoPath

In an earlier post I wrote about optional and non-required elements in XSD schemas, especially date and decimal (double) elements, and how InfoPath deals with such elements. I was annoyed that I could not get them to be optional when included in the InfoPath main data source (schema scope).

Today I took a closer look at this problem, and instead of starting with the service contract (WSCF) and creating an InfoPath form based on the generated web service, I designed a new form from scratch by building the data source (XSD schema) using InfoPath. And, lo and behold, the decimal fields added manually to the data source are by default optional! The 'Cannot be blank' checkbox is enabled and unchecked!

I extracted the form files and inspected the XSD schema, and the element had both minOccurs="0" and nillable="true". Just as I had defined the decimal elements in the XSD schemas used in my service contract. So, why the different behavior ?

To find out why, I opened one of the InfoPath forms made by connecting to a WSCF web service and then extracted the form files. When I inspected the schemas, the nillable="true" was not in the XSD schema induced by InfoPath. By adding the attribute manually to the schema and then opening the form definition file (.XSF) with InfoPath, the problem was solved and my decimal fields are now truly optional in InfoPath. Note that this fix will be gone as soon as you re-induce the data source schemas.

Is this a bug with regards to nillable in the generated WSDL, web service or just the way it is ? Or, more likely, a weakness in InfoPath ? To be continued...

Using XPath preceding-sibling in InfoPath rules

Several of our InfoPath forms have a repeating table that contains details about the entity being edited, e.g. an order with zero, one or more order lines for the products that makes up the order.

In this example I will outline how to add a rule to the repeating table that assigns incremental values to an element in a row based on the value of the preceding row's same element. I will also outline how to add a rule to the form's 'Open behavior' to set the seed value for this element in the first row of the table. This will ensure that the rules of the table will work as expected.

The XML used in this example has this structure:
<PaymentPlan>
<PaymentPlanRow>
<Year/><Product/><Amount/>
</PaymentPlanRow>
<PaymentPlanRow>
<Year/><Product/><Amount/>
</PaymentPlanRow>
</PaymentPlan>

To add a rule to the 'new table row' event, start by opening the properties dialog for the repeating table and click 'Rules'. Do not add the rule to the table cell, ensure that you add it to the table. Add a rule called 'SetRowYear' with a condition that the 'Year' field is blank. Then add an action of type 'Set a field's value' to calculate the next value for the 'Year' element. Enter this formula to look up the value of the preceding row's 'Year' element, convert it to a number and add one:

number(preceding-sibling::PaymentPlanRow[1]/Year) + 1

The XPath preceding-sibling axis consists of all nodes that have the same parent as the current node, from the current node and up to the start. The following-sibling axis consists of the nodes from the current node and down to the end of the parent's child nodes set. The important thing to notice is the index into the node set: [1], as when XPath is used to get a value from a set of nodes, it will always take the first node in document order when an index is not applied. Thus, if you do not specify the node set index, the value returned will always be that of the first row in the table, not the preceding row. Note that the index is relative from the current node; i.e. preceding-sibling[1] is the previous row, while preceding-sibling[2] is the row before that again.

There are several ways to set the value for the first row's 'Year' field to ensure that the above formula has a seed value. I have used a rule when the form is opened. Start by opening the 'Form Options' dialog and go to the 'Open and Save' folder, then click on 'Rules' in the 'Open Behavior' section. Add a rule called 'SetInitialYear' with a condition that the 'Year' field is blank. The condition will ensure that the value will not be set when opening an existing form. Then add an action of type 'Set a field's value' to calculate the seed value for the 'Year' element. Enter this formula to use the current year:

substring(today(); 1; 4)

These examples shows how you can use rules to perform node set operations, calculations and setting of values, without having to use form programming and without having to manipulate the XML DOM with JScript.

A few annoyances with InfoPath

InfoPath is a good application for controlled viewing and form entry of data into XML documents and web services, but there are a few things that could have been much better or logical.

What has annoyed me most this week in InfoPath are the handling of optional and non-required date and numerical elements in XSD (e.g. xs:date and xs:decimal), and the handling of default values based on formulas.

First the optional (minOccurs="0") and non-required (nillable="true") "object" types in XSD: they will still be mandatory in InfoPath when using the WSCF (web services contract first) approach. What you gain is the option to leave these elements out of the InfoPath form's scope. If you include these fields in the scope, you must yourself handle the fields in an OnSubmit event handler, setting a dummy date into the fields. Likewise, you must clear the dummy dates when loading existing data into InfoPath.

Then the default value mechanism of InfoPath: we have some fields that are calculated based on other fields, but the user can override the calculated value by typing a number of their own choice into the field. This seems to function OK with InfoPath, and when you submit the form to the web service, the user's number is there alright. However, when the form loads the stored data later on, InfoPath actually reapplies the default value formula, even when the field already has content. Not exactly the way I expected it to work. The solution is to use InfoPath rules. But even rules has a small quirck; they only trigger on changing the content of a field, thus it is hard to apply "default value when user leaves field empty" logic in InfoPath.

Debug the InfoPath submit SOAP message

I am currently building a prototype for using web services contract first (WSCF from Thinktecture) to build web services to support InfoPath as the client. Setting up InfoPath (SP1) to query and submit to the web service is straightforward, and getting data from the web service worked immediately. I could not, however, get submitting the form to the web service to work, and I suspected my setup of which part of the XML to post to be incorrect.

During my attempts to get submit working, I needed to spy on the SOAP messages generated by InfoPath to see why the posted XML could not be deserialized according to my message schema. As I am a long time fan of Altova XmlSpy, I decided to try out their XmlSpy SOAP Debugger. The XmlSpy integration with VS.NET makes working with XSD schemas quite easy.

[UPDATE] XmlSpy is still great, but you might find that Fiddler is sufficient for inspecting the HTTP request/response traffic to between InfoPath and your web-service.

Follow the steps outlined in the above link to configure and start the SOAP debugger, noting that the debugger is a proxy process that sets up a new HTTP port (:8080) that your SOAP traffic must be sent to. The debugger proxy then does its magic and finally forwards the traffic to the original port (:80).

So to be able to intercept and spy on the SOAP message submitted by InfoPath, you must open the form and change the data connection used by submit. In the first step of the 'Data Connection' wizard, you must change the location in 'web service details (submit data)' to use the SOAP debugger proxy at port :8080. Then complete the remaining steps of the wizard, selecting the correct data to use as the 'submit operation' parameters. Close the wizard, save the form and then test it using 'Preview Form'. When you now submit the form, XmlSpy will break on the selected operation and you can review the SOAP request made by InfoPath.

My mistake was that I had chosen to include 'XML subtree...' instead of 'Text and child elementes only', thus InfoPath posted my main parameter element twice, nested within itself (<message><dto><dto>...). I would most likely have found the correct configuration anyway, but using XmlSpy certainly saved me a lot of time.