My last post showed how to get explicit semantics in a WCF contract description using [Flags] enum. As readers of my blog may know, our main service consumer is a Flex 2.x client. As it turns out, Flex has an issue with correctly interpreting such enumerations.
A [Flags] enum looks like this in the WSDL:
<s:simpleType name="DisciplineFlags">
<s:list>
<s:simpleType>
<s:restriction base="s:string">
<s:enumeration value="None" />
<s:enumeration value="Administration" />
. . .
The correct input format for an element that uses the above type is XML whitespace separated values, such as "None Administration". Flex, on the other hand, interprets the list schema construct too litterally, and tries to pass in an array. This causes an exception in the WCF service, as the message cannot be deserialized due to the incorrect XML.
As the Flex guys had no other solution to this problem, I just added a simple "FlexDisciplineFlags" string property to the contract. Flex has to pass a CSV list of the filter enum values, and the service will "cast" the input into the official filter condition:
private void HackFlexIssues(DocumentCardFilter filter)
{
if (String.IsNullOrEmpty(filter.FlexDisciplineFlags)==false)
{
//input string must be a CSV-list of enum items
filter.DisciplineFlags = (DisciplineFlags)Enum.Parse(typeof(DisciplineFlags), filter.FlexDisciplineFlags, false);
}
}
Note that I'm kind of making a workaround for a technical limitation, maybe I'll get sued by Adobe :)
Showing posts with label ASMX. Show all posts
Showing posts with label ASMX. Show all posts
Monday, June 04, 2007
Monday, February 26, 2007
Two years of blogging
Today is my two year anniversary as a blogger. I have managed a rate of one post per week on average, which is far better than I expected when I started. The topics have changed over time as my projects have moved from MSCRM+Exchange+Outlook+ActiveDirectory via InfoPath and .NET 2/VS2005 stuff to service-oriented solutions using WCF and other NETFX3 technologies.
The most popular article on my blog has been 'Outlook 2003 .NET add-ins with VSTO 2005' for quite a while, with Sue Mosher's OutlookCode.com as the main referrer. My posts about .NET 2 WinForms databinding have also been very popular, especially those about object binding source problems and how to solve them. Google is top-dog referrer to my blog, which is nice as it shows that what I write about are useful for the community.
Of my recent posts, the WCF 'Providing exception shielding details using Generics' article has entered the top five list; along with the 'How to expose a WCF service also as an ASMX web-service' post. I suspect that the growing popularity of Flex for RIAs cause some of the traffic for the latter posting.
That's is all for now, I'll be back soon with more about WCF and stuff from EntLib3 that you just got to love.
The most popular article on my blog has been 'Outlook 2003 .NET add-ins with VSTO 2005' for quite a while, with Sue Mosher's OutlookCode.com as the main referrer. My posts about .NET 2 WinForms databinding have also been very popular, especially those about object binding source problems and how to solve them. Google is top-dog referrer to my blog, which is nice as it shows that what I write about are useful for the community.
Of my recent posts, the WCF 'Providing exception shielding details using Generics' article has entered the top five list; along with the 'How to expose a WCF service also as an ASMX web-service' post. I suspect that the growing popularity of Flex for RIAs cause some of the traffic for the latter posting.
That's is all for now, I'll be back soon with more about WCF and stuff from EntLib3 that you just got to love.
Monday, January 15, 2007
SOA Darwinism - Natural selection of agile services
The LEGO block analogy for SOA is wellknown and the message is that a service should be like a LEGO block: well-defined interfaces, reusability, easy to assemble into new compositions, orchestrations and mashups; but also a governance problem with all the different services floating around; and add that even if it is easy to compose new structures from Lego blocks, it might not be that simple to change Lego systems.
Another analogy that is useful when discussing service-orientation is Darwinism (Charles Darwin) - more specifically that specialization of a species to a specific habitat makes the species less adaptable and more vulnerable to changes in their environment. If you think of a service as a 'species' and of the 'habitat' as the service context, you'll see that a service that has a very high coupling to its context is not very well suited to being reused in another context. The service is just not agile. The service is just not ready for SOA bliss.
Just as if you relocated e.g. a moose to Africa, it would die as it is specialized to the climate and the type of food (birch/pine) of its habitat; a service that depends on e.g. the employee directory cannot easily be reused on your company's public web-site.
I think that the agility of a service is a good indication of whether your services are truly SOA rather than plain JBOWS. Providing good SOA services is more difficult than you think, making just a web-service is far to easy. This is the hardest message to get across to JBOWS developers claiming to provide SOA services. Afterall, a web-service that is specialized to fit perfectly to e.g. the current intranet application will become extinct when the business environment changes. And nothing changes faster than business. A moose in Africa stands a better chance.
Evolve, as natural selection will see to that repeating the failure of DCOM/CORBA for distributed systems using web-services has no future in SOA.
Btw, Dion Hinchcliffe has a good article about SOA, WOA and SaaS in 2007, referring to the web as a Darwinistic 'petri dish' software experiment that will show us what works.
Another analogy that is useful when discussing service-orientation is Darwinism (Charles Darwin) - more specifically that specialization of a species to a specific habitat makes the species less adaptable and more vulnerable to changes in their environment. If you think of a service as a 'species' and of the 'habitat' as the service context, you'll see that a service that has a very high coupling to its context is not very well suited to being reused in another context. The service is just not agile. The service is just not ready for SOA bliss.
Just as if you relocated e.g. a moose to Africa, it would die as it is specialized to the climate and the type of food (birch/pine) of its habitat; a service that depends on e.g. the employee directory cannot easily be reused on your company's public web-site.
I think that the agility of a service is a good indication of whether your services are truly SOA rather than plain JBOWS. Providing good SOA services is more difficult than you think, making just a web-service is far to easy. This is the hardest message to get across to JBOWS developers claiming to provide SOA services. Afterall, a web-service that is specialized to fit perfectly to e.g. the current intranet application will become extinct when the business environment changes. And nothing changes faster than business. A moose in Africa stands a better chance.
Evolve, as natural selection will see to that repeating the failure of DCOM/CORBA for distributed systems using web-services has no future in SOA.
Btw, Dion Hinchcliffe has a good article about SOA, WOA and SaaS in 2007, referring to the web as a Darwinistic 'petri dish' software experiment that will show us what works.
Friday, December 22, 2006
How to: Expose WCF service also as ASMX web-service
In my last post, I wrote about my endeavors to consume a WCF service from an ASMX client to support non-WCF clients such as .NET 2 clients. The WCF service is exposed using a BasicHttpBinding endpoint and our main consumer is a Flex 2 web-application.
As the support for WS-I Basic Profile 1.x compliant web-services is quite limited in Flex, I had to make ASMX-style clients works. What made it worse, is that there is a bug in Flex regarding the support for xsd:import: Flex will repeatedly request the XSD files on .LoadWSDL(), causing bad performance for the web-service and the client. Use Fiddler to watch all the HTTP GETs issued by Flex. The solution is to ensure that the generated WSDL is a single file, without any xsd:import, and this is just what a classic .NET 2 ASMX web-service provides.
WCF has good support for migrating ASMX web-services to WCF, and this feature can also be used to actually expose a native WCF service also as an ASMX web-service. Start by adding the classic [WebService] and [WebMethod] attributes to your service interface:
Note that there is no need for adding [XmlSerializerFormat] to the [ServiceContract] attribute, just leave the WCF service as-is. Note also that you must specify the web-service binding Name parameter to avoid getting duplicate wsdl:port elements in the generated WSDL file. You should apply the [WebService] attribute using the same Name and Namespace parameters as for the [WebServiceBinding], to the class implementing the service. If not, you might get a wsdl:include and and xsd:import in the generated WSDL file.
Then add a new .ASMX file to the IIS host used for your WCF service, using the service implementation assembly to provide the web-services. I added a file called "ProjectDocumentWebService.asmx" with this content:
That's it. You now have a classic ASMX web-service at your disposal. WCF supports running WCF services and ASMX web-services side-by-side in the same IIS web-application: Hosting WCF Side-by-Side with ASP.NET.
Most likely you must also apply [XmlElement], [XmlArray] and [XmlIgnore] at relevant places in your message and data contracts to ensure correct serialization (XmlSerializer) and ensure consistent naming for both WCF and ASMX clients/proxies. Remember that the XmlSerializer works only with public members and that it does not support read-only properties. Do not mark message and data contract classes as [Serializable] as this changes the wire-format of the XML, which might cause problems in Flex 2.Note that the ASMX-style WSDL will not respect the [DataMember (IsRequired=true)] setting or any other WCF settings because it uses the XmlSerializer and not the DataContractSerializer. You will get minOccurs=0 in the generated WSDL for string fields, as string is a reference type. This is because the XmlSerializer by default makes reference types optional, while value types by default are mandatory. Read more about XmlSerializer 'MinOccurs Attribute Binding Support' at MSDN.
Note the difference between optional elements (minOccurs) and non-required data (nillable) in XSD schemas. This is especially useful in query specification objects, make all the elements optional and apply only the specified elements as criteria - even "is null" criteria.
The unsolved WSDL problem described in my last post is no longer an issue for ASMX-clients, as the WSDL generated by the classic ASMX web-service of course natively supports .NET 2 proxies generated by WSDL.EXE. The issue with the Name parameter for the [CollectionDataContract] attribute is still a problem, it must be removed.
Thanks to Mattavi for the "Migrating .NET Remoting to WCF (and even ASMX!)" article that got me started.
As the support for WS-I Basic Profile 1.x compliant web-services is quite limited in Flex, I had to make ASMX-style clients works. What made it worse, is that there is a bug in Flex regarding the support for xsd:import: Flex will repeatedly request the XSD files on .LoadWSDL(), causing bad performance for the web-service and the client. Use Fiddler to watch all the HTTP GETs issued by Flex. The solution is to ensure that the generated WSDL is a single file, without any xsd:import, and this is just what a classic .NET 2 ASMX web-service provides.
WCF has good support for migrating ASMX web-services to WCF, and this feature can also be used to actually expose a native WCF service also as an ASMX web-service. Start by adding the classic [WebService] and [WebMethod] attributes to your service interface:
[ServiceContractAttribute(Namespace = "http://kjellsj.blogspot.com/2006/11",
Name = "IProjectDocumentServices")]
[ServiceKnownTypeAttribute(
typeof(DNVS.DNVX.eApproval.FaultContracts.DefaultFaultContract))]
[WebService(Name = "ProjectDocumentServices")]
[WebServiceBinding(Name = "ProjectDocumentServices",
ConformsTo = WsiProfiles.BasicProfile1_1, EmitConformanceClaims = true)]
public interface IProjectDocumentServices
{
[OperationContractAttribute(Action = "GetDocumentCard")]
[FaultContractAttribute(
typeof(DNVS.DNVX.eApproval.FaultContracts.DefaultFaultContract))]
[WebMethod]
DocumentCardResponse GetDocumentCard(KeyCriteriaMessage request);
. . .
Note that there is no need for adding [XmlSerializerFormat] to the [ServiceContract] attribute, just leave the WCF service as-is. Note also that you must specify the web-service binding Name parameter to avoid getting duplicate wsdl:port elements in the generated WSDL file. You should apply the [WebService] attribute using the same Name and Namespace parameters as for the [WebServiceBinding], to the class implementing the service. If not, you might get a wsdl:include and and xsd:import in the generated WSDL file.
Then add a new .ASMX file to the IIS host used for your WCF service, using the service implementation assembly to provide the web-services. I added a file called "ProjectDocumentWebService.asmx" with this content:
<%@ WebService Language="C#" Class="DNVS.DNVX.eApproval.ServiceImplementation
.ProjectDocumentServices, DNVS.DNVX.eApproval.ServiceImplementation" %>
That's it. You now have a classic ASMX web-service at your disposal. WCF supports running WCF services and ASMX web-services side-by-side in the same IIS web-application: Hosting WCF Side-by-Side with ASP.NET.
Most likely you must also apply [XmlElement], [XmlArray] and [XmlIgnore] at relevant places in your message and data contracts to ensure correct serialization (XmlSerializer) and ensure consistent naming for both WCF and ASMX clients/proxies. Remember that the XmlSerializer works only with public members and that it does not support read-only properties. Do not mark message and data contract classes as [Serializable] as this changes the wire-format of the XML, which might cause problems in Flex 2.Note that the ASMX-style WSDL will not respect the [DataMember (IsRequired=true)] setting or any other WCF settings because it uses the XmlSerializer and not the DataContractSerializer. You will get minOccurs=0 in the generated WSDL for string fields, as string is a reference type. This is because the XmlSerializer by default makes reference types optional, while value types by default are mandatory. Read more about XmlSerializer 'MinOccurs Attribute Binding Support' at MSDN.
Note the difference between optional elements (minOccurs) and non-required data (nillable) in XSD schemas. This is especially useful in query specification objects, make all the elements optional and apply only the specified elements as criteria - even "is null" criteria.
The unsolved WSDL problem described in my last post is no longer an issue for ASMX-clients, as the WSDL generated by the classic ASMX web-service of course natively supports .NET 2 proxies generated by WSDL.EXE. The issue with the Name parameter for the [CollectionDataContract] attribute is still a problem, it must be removed.
Thanks to Mattavi for the "Migrating .NET Remoting to WCF (and even ASMX!)" article that got me started.
Thursday, December 21, 2006
Consume WCF BasicHttpBinding Services from ASMX Clients
The interoperability of WCF services has been touted quite a bit: just expose an endpoint using BasicHttpBinding and you will have a WS-I Basic Profile adherent web-service. This is of course backed by some simple examples of math services, all RPC-style and none of them messaging-style services.
We have for some time been consuming our BasicHttpBinding WCF services from a Flex 2 client and a WinForms smart client using a WCF generated client proxy (svcutil.exe). Just to be certain that our partners would have no problem setting up an application-to-application integration; I decided to test our services ASMX-style using "Add web reference" in Visual Studio (wsdl.exe). This to ensure that the services are consumable from non-WCF systems, i.e. systems without the .NET 3 run-time installed.
Well, surprise, surprise; it wasn't as straightforward as shown in the examples. There are several details that you need to be aware of, and some bugs in the service/message/data contract serializing mechanism.
I started by creating a new WinForms application and just used "Add web reference" to reference my "projectdocumentservices.svc?WSDL" file and WSDL.EXE generated an ASMX-style proxy for me. So far this looks good. I then added code to call my HelloWorld method on the proxy, which gave me this run-time error:
System.InvalidOperationException: Method ProjectDocumentServices.GetDocumentRequirement can not be reflected. ---> System.InvalidOperationException: The XML element 'KeyCriteriaMessage' from namespace 'http://DNVS.DNVX.eApproval.ServiceContracts/2006/11' references a method and a type. Change the method's message name using WebMethodAttribute or change the type's root element using the XmlRootAttribute.
Note that the Flex 2 client has no such problems with the BasicHttpBinding web-service, thus it must be related to how the generated proxy interprets the WSDL.
My service uses the specified message in several operations:
[System.ServiceModel.OperationContractAttribute(Action = "GetDocumentCard")]DocumentCardResponse GetDocumentCard(KeyCriteriaMessage request);
[System.ServiceModel.OperationContractAttribute(Action = "GetDocumentCategory")]DocumentCategoryResponse GetDocumentCategory(KeyCriteriaMessage request);
[System.ServiceModel.OperationContractAttribute(Action = "GetDocumentRequirement")]DocumentRequirementResponse GetDocumentRequirement(KeyCriteriaMessage request);
I turned to MSDN for more info and found the article 'ASMX Client with a WCF Service' (note the RPC-style math services) which lead me to 'Using the XmlSerializer'. So, accordingly, I added the [XmlSerializerFormat] attribute to the [ServiceContract] attribute in my service interface. Then I compiled and inspected the service, and got a series of different errors when trying to view the generated WSDL:
[SEHException (0x80004005): External component has thrown an exception.]
[CustomAttributeFormatException: Binary format of the specified custom attribute was invalid.]
[InvalidOperationException: There was an error reflecting type 'DNVS.DNVX.eApproval.DataContracts.DocumentCard'.]
Applying the well-known XML serialization attributes [Serializable] and [XmlIgnore] at relevant places in the data and message contracts helped me to isolate the problem to the use of collections and the WCF [CollectionDataContract] attribute.
To make a long story short, there is a bug in the WCF framework that affects .asmx but not .svc when using the [CollectionDataContract(Name="...")] attribute parameter:
"This happens only with Asmx pages but not for Svc due to a known Whidbey issue. Removing the "Name" parameter setting would work. If it is possible, you could also have your client instead use svc only."
Removing the Name parameter from all my [CollectionDataContract] attributes made all the WSDL reflection errors disappear, and I was again able to view the generated WSDL file. This time for a service that used the [XmlSerializerFormat] attribute.
Full of hope, I updated the web reference in my test client and ran it. Calling the test method lead me straight back to my original problem: the message contract used in multiple operations. Note that removing all but one of the operations makes the problem go away.
According to 'Message WSDL Considerations' WCF should have done this when generating the WSDL:
"When using the same message contract in multiple operations, multiple message types are generated in the WSDL document. The names are made unique by adding the numbers "2", "3", and so on for subsequent uses. When importing back the WSDL, multiple message contract types are created and are identical except for their names."
Inspecting the WSDL showed me that multiple XSD schemas are not generated. Certainly another WCF issue/bug. The problem can be solved by deriving the message class into new classes with unique names and using the derived classes in the operations. But when you have a message that is used ubiquitously, e.g. DefaultOperationResponse, this is not a viable solution.
As of now, our BasicHttpBinding WCF service cannot be consumed by ASMX clients. So much for interoperability...
[UPDATE] Read my new post about how to expose a WCF service also as an ASMX web-service.
We have for some time been consuming our BasicHttpBinding WCF services from a Flex 2 client and a WinForms smart client using a WCF generated client proxy (svcutil.exe). Just to be certain that our partners would have no problem setting up an application-to-application integration; I decided to test our services ASMX-style using "Add web reference" in Visual Studio (wsdl.exe). This to ensure that the services are consumable from non-WCF systems, i.e. systems without the .NET 3 run-time installed.
Well, surprise, surprise; it wasn't as straightforward as shown in the examples. There are several details that you need to be aware of, and some bugs in the service/message/data contract serializing mechanism.
I started by creating a new WinForms application and just used "Add web reference" to reference my "projectdocumentservices.svc?WSDL" file and WSDL.EXE generated an ASMX-style proxy for me. So far this looks good. I then added code to call my HelloWorld method on the proxy, which gave me this run-time error:
System.InvalidOperationException: Method ProjectDocumentServices.GetDocumentRequirement can not be reflected. ---> System.InvalidOperationException: The XML element 'KeyCriteriaMessage' from namespace 'http://DNVS.DNVX.eApproval.ServiceContracts/2006/11' references a method and a type. Change the method's message name using WebMethodAttribute or change the type's root element using the XmlRootAttribute.
Note that the Flex 2 client has no such problems with the BasicHttpBinding web-service, thus it must be related to how the generated proxy interprets the WSDL.
My service uses the specified message in several operations:
[System.ServiceModel.OperationContractAttribute(Action = "GetDocumentCard")]DocumentCardResponse GetDocumentCard(KeyCriteriaMessage request);
[System.ServiceModel.OperationContractAttribute(Action = "GetDocumentCategory")]DocumentCategoryResponse GetDocumentCategory(KeyCriteriaMessage request);
[System.ServiceModel.OperationContractAttribute(Action = "GetDocumentRequirement")]DocumentRequirementResponse GetDocumentRequirement(KeyCriteriaMessage request);
I turned to MSDN for more info and found the article 'ASMX Client with a WCF Service' (note the RPC-style math services) which lead me to 'Using the XmlSerializer'. So, accordingly, I added the [XmlSerializerFormat] attribute to the [ServiceContract] attribute in my service interface. Then I compiled and inspected the service, and got a series of different errors when trying to view the generated WSDL:
[SEHException (0x80004005): External component has thrown an exception.]
[CustomAttributeFormatException: Binary format of the specified custom attribute was invalid.]
[InvalidOperationException: There was an error reflecting type 'DNVS.DNVX.eApproval.DataContracts.DocumentCard'.]
Applying the well-known XML serialization attributes [Serializable] and [XmlIgnore] at relevant places in the data and message contracts helped me to isolate the problem to the use of collections and the WCF [CollectionDataContract] attribute.
To make a long story short, there is a bug in the WCF framework that affects .asmx but not .svc when using the [CollectionDataContract(Name="...")] attribute parameter:
"This happens only with Asmx pages but not for Svc due to a known Whidbey issue. Removing the "Name" parameter setting would work. If it is possible, you could also have your client instead use svc only."
Removing the Name parameter from all my [CollectionDataContract] attributes made all the WSDL reflection errors disappear, and I was again able to view the generated WSDL file. This time for a service that used the [XmlSerializerFormat] attribute.
Full of hope, I updated the web reference in my test client and ran it. Calling the test method lead me straight back to my original problem: the message contract used in multiple operations. Note that removing all but one of the operations makes the problem go away.
According to 'Message WSDL Considerations' WCF should have done this when generating the WSDL:
"When using the same message contract in multiple operations, multiple message types are generated in the WSDL document. The names are made unique by adding the numbers "2", "3", and so on for subsequent uses. When importing back the WSDL, multiple message contract types are created and are identical except for their names."
Inspecting the WSDL showed me that multiple XSD schemas are not generated. Certainly another WCF issue/bug. The problem can be solved by deriving the message class into new classes with unique names and using the derived classes in the operations. But when you have a message that is used ubiquitously, e.g. DefaultOperationResponse, this is not a viable solution.
As of now, our BasicHttpBinding WCF service cannot be consumed by ASMX clients. So much for interoperability...
[UPDATE] Read my new post about how to expose a WCF service also as an ASMX web-service.
Wednesday, February 08, 2006
Conditional XML serialization
In my post about implementing entity object row state using 'is dirty' and 'cloning' I described how to exclude properties during binary serialization using [NonSerialized]. The same can be done with XML serialization using [XmlIgnore]. In addition, properties can be temporarily excluded when serializing to XML. The mechanism behind conditionally excluding properties from XML serialization is not well known, thus this post to shed some light on it.
The bool <property name>Specified mechanism is the .NET serialization framework answer to XSD minOccurs="0" applied by code generators such as XSD.EXE, XsdObjectGen, and others. Most developers actually implicitly use this mechanism when they design and use typed datasets with optional fields (columns).
In short, whenever .NET serializes the properties of a class to XML, it always checks to see if there exists a boolean metod whose name is the name of the property suffixed with Specified. If such a method exists, and it returns false, the property is not included in the serialized XML; if it returns true, the property and value is serialized. The oppsite logic is applied on de-serialization of optional XML elements.
Thus, you can implement and use Specified methods at will to get conditional XML serialization in .NET. The serializer does not discriminate between methods implemented by code generators and methods implemented by you. Partial classes in .NET 2.0 also makes adding such methods to code generated classes much simpler.
I got to know the details of how XSD.EXE handles XSD minOccurs="0" and xsi:nil="true" very well when doing "web-service contract first" in combination with InfoPath last year. The support for doing WSCF in .NET 2.0 has improved with the support for e.g. interface based .ASMX web-services, but Visual Studio 2005 still has a way to go to support WSCF to its maximum extent. I expect that this will be finally solved when WCF (Indigo) is released.
The bool <property name>Specified mechanism is the .NET serialization framework answer to XSD minOccurs="0" applied by code generators such as XSD.EXE, XsdObjectGen, and others. Most developers actually implicitly use this mechanism when they design and use typed datasets with optional fields (columns).
In short, whenever .NET serializes the properties of a class to XML, it always checks to see if there exists a boolean metod whose name is the name of the property suffixed with Specified. If such a method exists, and it returns false, the property is not included in the serialized XML; if it returns true, the property and value is serialized. The oppsite logic is applied on de-serialization of optional XML elements.
Thus, you can implement and use Specified methods at will to get conditional XML serialization in .NET. The serializer does not discriminate between methods implemented by code generators and methods implemented by you. Partial classes in .NET 2.0 also makes adding such methods to code generated classes much simpler.
I got to know the details of how XSD.EXE handles XSD minOccurs="0" and xsi:nil="true" very well when doing "web-service contract first" in combination with InfoPath last year. The support for doing WSCF in .NET 2.0 has improved with the support for e.g. interface based .ASMX web-services, but Visual Studio 2005 still has a way to go to support WSCF to its maximum extent. I expect that this will be finally solved when WCF (Indigo) is released.
Wednesday, August 31, 2005
Web-services: interoperability & encoding
I am currently implementing MSCRM at a customer that is also implementing an enterprise portal using SiteVision delivered by Objectware's Java department. The need for providing the SiteVision portal with information about MSCRM accounts made me implement a few .NET XML web-services to be used by the Java guys. The web-service provides basic read, create and update services for accounts and contacts. The 'data transfer object' was simply implemented as a string that would contain XML according to a predefined XSD schema.
The enterprise application integration server used is BIE (Business Integration Engine; Java open source), chosen by the Java lead developer on this project. Their use of my web-services went along quite smootly as was to be expected as .NET web-services complies with WS-I BP1.0 (intro article here). I had even used the WS-I compliance tools to check my web-services.
What caused a major halt on the integration implementation was when we started testing with data containing Norwegian characters (ÆØÅ). To be more precise, this was tested and found to be OK by using MSIE as the test-client, and by using XmlSpy to retrieve data about e.g. an account, modifying the data, and finally updating the account. I strongly recommend the XmlSpy SOAP Debugger tool both for testing and debugging web-serives.
The problem was that although BIE supports web-servies using UTF-8, it does not differentiate between UTF-8 as a text encoding in the web-service and UTF-8 encoded XML and the W3C character encoding rules that dictates that a unicode character must be encoded as a #xHHHH numeric character reference (i.e. 16-bit unicode code points/units; UTF-16 NCR). For more details see 'Can XML use non-Latin characters?'.
The MSCRM 1.2 object model supports and returns XML using the #xHHHH character encoding. All strings in .NET are unicode.
The character encoding went through these phases (examples for Æ and æ):
A nice unicode character encoding test tool is available here.
The Java lead developer had run into these encoding problems in BIE and modified the portal code to do some character replacing (Æ to Æ, etc) on all text to/from BIE on their side, and I asked them to change their integration implementation to comply with the WS-I and W3C standards. Unfortunately, this lead developer is more interested in hailing the glory of the application architecture & design and the open source movement than complying with standards "invented by Microsoft" :-)
Thus, I had to write a SoapExtension to modify the incoming SOAP message before it was deserialized, to change the "wrong" UTF-8 encoding into correct XML W3C UTF-8 encoding. I used this GotDotNet sample as the basis for my code and this is how I modify the incoming and outgoing SOAP messages:
public override void ProcessMessage(SoapMessage message)
{
switch (message.Stage)
{
//INCOMING
case SoapMessageStage.BeforeDeserialize:
this.ChangeIncomingEncoding();
break;
case SoapMessageStage.AfterDeserialize:
_isPostDeserialize = true;
break;
//OUTGOING
case SoapMessageStage.BeforeSerialize:
break;
case SoapMessageStage.AfterSerialize:
this.ChangeOutgoingEncoding();
break;
}
}
public override Stream ChainStream( Stream stream )
{
//http://hyperthink.net/blog/CommentView,guid,eafeef67-c240-44cc-8550-974f5d378a8f.aspx
if(!_isPostDeserialize)
{
//INCOMING
_inputStream = stream;
_outputStream = new MemoryStream();
return _outputStream;
}
else
{
//OUTGOING
_outputStream = stream;
_inputStream = new MemoryStream();
return _inputStream;
}
}
public void ChangeIncomingEncoding()
{
//at BeforeDeserialize
if(_inputStream.CanSeek)
_inputStream.Position = 0L;
TextReader reader = new StreamReader(_inputStream);
TextWriter writer = new StreamWriter(_outputStream);
string line;
while((line = reader.ReadLine()) != null)
{
writer.WriteLine( Utilities.FixBieEncoding (line) );
}
writer.Flush();
//reset the new stream to ensure that AfterDeserialize is called
if(_outputStream.CanSeek)
_outputStream.Position = 0L;
}
public void ChangeOutgoingEncoding()
{
//at AfterSerialize
if(_inputStream.CanSeek)
_inputStream.Position = 0L;
Regex regex = new Regex("utf-8", RegexOptions.IgnoreCase);
TextReader reader = new StreamReader(_inputStream);
//HACK: TextWriter writer = new StreamWriter(_outputStream);
TextWriter writer = new StreamWriter(_outputStream, System.Text.Encoding.GetEncoding(_encoding));
string line;
while((line = reader.ReadLine()) != null)
{
// change the encoding only is needed
if(_encoding != null && !_encoding.Equals("utf-8"))
line = regex.Replace(line, _encoding);
writer.WriteLine(line);
}
writer.Flush();
}
The central method here is the ChangeIncomingEncoding method that converts all the "wrong" UTF-8 encodings from BIE into correct XML W3C NCRs. Note the resetting of the position to zero on the output stream after modifying the message; this is important as forgetting to do so will cause the AfterDeserialize step of the SoapMessageStage in ProcessMessage not to be called.
After deploying the modified web-service and testing it with XmlSpy, it was time to test it with the BIE workflow dashboard. The BIE developer had set up some test cases for me, and they all worked as expected. No more funny farm in the MSCRM database.
What an illustrious victory for Java-.NET web-service interoperability!
Note that you cannot test/debug your SoapExtension code by using MSIE as the test-client as HTTP POST/GET will not trigger the SoapExtension. I used XmlSpy to test and debug my code; just set some breakpoints, start your web-service in debug mode and leave it running, then trigger the SoapExtention by making a SOAP request using XmlSpy.
The enterprise application integration server used is BIE (Business Integration Engine; Java open source), chosen by the Java lead developer on this project. Their use of my web-services went along quite smootly as was to be expected as .NET web-services complies with WS-I BP1.0 (intro article here). I had even used the WS-I compliance tools to check my web-services.
What caused a major halt on the integration implementation was when we started testing with data containing Norwegian characters (ÆØÅ). To be more precise, this was tested and found to be OK by using MSIE as the test-client, and by using XmlSpy to retrieve data about e.g. an account, modifying the data, and finally updating the account. I strongly recommend the XmlSpy SOAP Debugger tool both for testing and debugging web-serives.
The problem was that although BIE supports web-servies using UTF-8, it does not differentiate between UTF-8 as a text encoding in the web-service and UTF-8 encoded XML and the W3C character encoding rules that dictates that a unicode character must be encoded as a #xHHHH numeric character reference (i.e. 16-bit unicode code points/units; UTF-16 NCR). For more details see 'Can XML use non-Latin characters?'.
The MSCRM 1.2 object model supports and returns XML using the #xHHHH character encoding. All strings in .NET are unicode.
The character encoding went through these phases (examples for Æ and æ):
- from our MSCRM web-service: Æ encoded as Æ
- back from BIE on create/update: Æ "encoded" as Æ (not even valid UTF-8 code units)
- data in MSCRM after operation: Æ
- from our MSCRM web-service: æ encoded as æ
- back from BIE on create/update: æ encoded as æ (which are valid UTF-8 code units)
- data in MSCRM after operation: æ
A nice unicode character encoding test tool is available here.
The Java lead developer had run into these encoding problems in BIE and modified the portal code to do some character replacing (Æ to Æ, etc) on all text to/from BIE on their side, and I asked them to change their integration implementation to comply with the WS-I and W3C standards. Unfortunately, this lead developer is more interested in hailing the glory of the application architecture & design and the open source movement than complying with standards "invented by Microsoft" :-)
Thus, I had to write a SoapExtension to modify the incoming SOAP message before it was deserialized, to change the "wrong" UTF-8 encoding into correct XML W3C UTF-8 encoding. I used this GotDotNet sample as the basis for my code and this is how I modify the incoming and outgoing SOAP messages:
public override void ProcessMessage(SoapMessage message)
{
switch (message.Stage)
{
//INCOMING
case SoapMessageStage.BeforeDeserialize:
this.ChangeIncomingEncoding();
break;
case SoapMessageStage.AfterDeserialize:
_isPostDeserialize = true;
break;
//OUTGOING
case SoapMessageStage.BeforeSerialize:
break;
case SoapMessageStage.AfterSerialize:
this.ChangeOutgoingEncoding();
break;
}
}
public override Stream ChainStream( Stream stream )
{
//http://hyperthink.net/blog/CommentView,guid,eafeef67-c240-44cc-8550-974f5d378a8f.aspx
if(!_isPostDeserialize)
{
//INCOMING
_inputStream = stream;
_outputStream = new MemoryStream();
return _outputStream;
}
else
{
//OUTGOING
_outputStream = stream;
_inputStream = new MemoryStream();
return _inputStream;
}
}
public void ChangeIncomingEncoding()
{
//at BeforeDeserialize
if(_inputStream.CanSeek)
_inputStream.Position = 0L;
TextReader reader = new StreamReader(_inputStream);
TextWriter writer = new StreamWriter(_outputStream);
string line;
while((line = reader.ReadLine()) != null)
{
writer.WriteLine( Utilities.FixBieEncoding (line) );
}
writer.Flush();
//reset the new stream to ensure that AfterDeserialize is called
if(_outputStream.CanSeek)
_outputStream.Position = 0L;
}
public void ChangeOutgoingEncoding()
{
//at AfterSerialize
if(_inputStream.CanSeek)
_inputStream.Position = 0L;
Regex regex = new Regex("utf-8", RegexOptions.IgnoreCase);
TextReader reader = new StreamReader(_inputStream);
//HACK: TextWriter writer = new StreamWriter(_outputStream);
TextWriter writer = new StreamWriter(_outputStream, System.Text.Encoding.GetEncoding(_encoding));
string line;
while((line = reader.ReadLine()) != null)
{
// change the encoding only is needed
if(_encoding != null && !_encoding.Equals("utf-8"))
line = regex.Replace(line, _encoding);
writer.WriteLine(line);
}
writer.Flush();
}
The central method here is the ChangeIncomingEncoding method that converts all the "wrong" UTF-8 encodings from BIE into correct XML W3C NCRs. Note the resetting of the position to zero on the output stream after modifying the message; this is important as forgetting to do so will cause the AfterDeserialize step of the SoapMessageStage in ProcessMessage not to be called.
After deploying the modified web-service and testing it with XmlSpy, it was time to test it with the BIE workflow dashboard. The BIE developer had set up some test cases for me, and they all worked as expected. No more funny farm in the MSCRM database.
What an illustrious victory for Java-.NET web-service interoperability!
Note that you cannot test/debug your SoapExtension code by using MSIE as the test-client as HTTP POST/GET will not trigger the SoapExtension. I used XmlSpy to test and debug my code; just set some breakpoints, start your web-service in debug mode and leave it running, then trigger the SoapExtention by making a SOAP request using XmlSpy.
Monday, May 02, 2005
Do not use XSD default values with InfoPath
I have had some strange behavior in InfoPath with numeric fields that became locked even if they were not configured to be read-only. These fields were never locked when using the form to enter new data, but sometimes when modifying existing data.
After a bit of investigation, I found out that when an element in the XSD has e.g. default="0" and the user does not change it to something else, then this value will not be included in the XML when submitting to the web service. When the user later on opens the saved data, the element will not be in the XML returned by the web service. This will cause InfoPath to lock the field in the form, as InfoPath cannot bind to an element that is not there. This is very annoying, and setting default values in the InfoPath form will not make this go away. A field that contains a value different than the XSD default value, will be in the XML and thus fully editable.
In short, do not use the XSD default attribute in schemas intended for usage with InfoPath. It is sad that a client application imposes its weaknesses on the contract schema design, but the best is the enemy of the good, so I changed the schema to suite InfoPath.
Note that I have only tested this in combination with a WSCF web service.
After a bit of investigation, I found out that when an element in the XSD has e.g. default="0" and the user does not change it to something else, then this value will not be included in the XML when submitting to the web service. When the user later on opens the saved data, the element will not be in the XML returned by the web service. This will cause InfoPath to lock the field in the form, as InfoPath cannot bind to an element that is not there. This is very annoying, and setting default values in the InfoPath form will not make this go away. A field that contains a value different than the XSD default value, will be in the XML and thus fully editable.
In short, do not use the XSD default attribute in schemas intended for usage with InfoPath. It is sad that a client application imposes its weaknesses on the contract schema design, but the best is the enemy of the good, so I changed the schema to suite InfoPath.
Note that I have only tested this in combination with a WSCF web service.
Thursday, April 28, 2005
Submitting xsi:nil="true" values from InfoPath to ASP.NET 1.x web services
In my ongoing quest to build an InfoPath solution based on a WSCF web service implemented with ASP.NET 1.1, I have finally been able to:
- make non-string data types such as xs:date and xs:decimal truly optional in InfoPath
- submit data to the web service by modifing the XML data in OnSubmitRequest to circumvent the missing support for nillable in .NET 1.x ASP.NET web services
The first issue is caused by the fact that ASP.NET 1.x web service by design do not support nillable elements in the web service parameters. Check the <wsdl:types> element of any ASMX WSDL, and you will not find any nillable="true" elements even if your data XSD contains such elements. The reason for this is the lack of support for NULL in .NET 1.x value types. ASP.NET 2.0 web services will support nillable="true".
After modifying the underlying XSD schemas of your InfoPath as outlined in my previous post to manually re-introduce the nillable="true" attributes, the fields can be left empty in your form. But you will not be able to sumbit the form to the web service. The SOAP request will fail like this:
The SOAP response indicates that an error occurred:
Server was unable to read request. --> There is an error in XML document (47, 104). --> Input string was not in a correct format.
This is caused by this kind of element in the submitted XML:
<s1:Amount xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"></s1:Amount>
The web service is not able to deserialize this element into a value type due to the lack of support for nillable. Thus, the XML data must be modified before submitting the data to the web service.
InfoPath allows you to use the OnSubmitRequest event to write your own script for submitting the forms data. This event is only available by using Tools-Submitting Forms and selecting "Custom submit using form code" in the 'Submit to' dropdown. Check the 'Edit Form Code' checkbox and click OK to add an event handler for OnSubmitRequest.
This example shows how to look for all instances of a specific element that is NULL, remove the nil attribute and set a dummy value, and finally submitting the form using code:
function XDocument::OnSubmitRequest(eventObj)
{
try
{
var submitDataSource = "Main submit";
//debugger
//get a collection of
var nodeList = XDocument.DOM.selectNodes("/dfs:myFields/dfs:dataFields//s1:Amount[@xsi:nil='true']");
//iterate the list and
//
for(var i=0; i < nodeList.length; i++)
//put zero into the element
//
// The xsi:nil needs to be removed before we set the value.
xmlNode.removeAttribute("xsi:nil");
// Setting the value will mark the document as dirty.
xmlNode.text = -1; //biz logic will interpret this as NULL
}
//call the Submit method of the
//
XDocument.DataAdapters(submitDataSource).Submit();
eventObj.ReturnStatus = true;
}
catch(ex)
{
XDocument.UI.Alert("Failed while sending the request.\r\n" + ex.number + " - " + ex.description);
eventObj.ReturnStatus = false;
}
}
Thursday, April 21, 2005
Optional numeric fields in InfoPath
In an earlier post I wrote about optional and non-required elements in XSD schemas, especially date and decimal (double) elements, and how InfoPath deals with such elements. I was annoyed that I could not get them to be optional when included in the InfoPath main data source (schema scope).
Today I took a closer look at this problem, and instead of starting with the service contract (WSCF) and creating an InfoPath form based on the generated web service, I designed a new form from scratch by building the data source (XSD schema) using InfoPath. And, lo and behold, the decimal fields added manually to the data source are by default optional! The 'Cannot be blank' checkbox is enabled and unchecked!
I extracted the form files and inspected the XSD schema, and the element had both minOccurs="0" and nillable="true". Just as I had defined the decimal elements in the XSD schemas used in my service contract. So, why the different behavior ?
To find out why, I opened one of the InfoPath forms made by connecting to a WSCF web service and then extracted the form files. When I inspected the schemas, the nillable="true" was not in the XSD schema induced by InfoPath. By adding the attribute manually to the schema and then opening the form definition file (.XSF) with InfoPath, the problem was solved and my decimal fields are now truly optional in InfoPath. Note that this fix will be gone as soon as you re-induce the data source schemas.
Is this a bug with regards to nillable in the generated WSDL, web service or just the way it is ? Or, more likely, a weakness in InfoPath ? To be continued...
Today I took a closer look at this problem, and instead of starting with the service contract (WSCF) and creating an InfoPath form based on the generated web service, I designed a new form from scratch by building the data source (XSD schema) using InfoPath. And, lo and behold, the decimal fields added manually to the data source are by default optional! The 'Cannot be blank' checkbox is enabled and unchecked!
I extracted the form files and inspected the XSD schema, and the element had both minOccurs="0" and nillable="true". Just as I had defined the decimal elements in the XSD schemas used in my service contract. So, why the different behavior ?
To find out why, I opened one of the InfoPath forms made by connecting to a WSCF web service and then extracted the form files. When I inspected the schemas, the nillable="true" was not in the XSD schema induced by InfoPath. By adding the attribute manually to the schema and then opening the form definition file (.XSF) with InfoPath, the problem was solved and my decimal fields are now truly optional in InfoPath. Note that this fix will be gone as soon as you re-induce the data source schemas.
Is this a bug with regards to nillable in the generated WSDL, web service or just the way it is ? Or, more likely, a weakness in InfoPath ? To be continued...
Wednesday, April 13, 2005
A few annoyances with InfoPath
InfoPath is a good application for controlled viewing and form entry of data into XML documents and web services, but there are a few things that could have been much better or logical.
What has annoyed me most this week in InfoPath are the handling of optional and non-required date and numerical elements in XSD (e.g. xs:date and xs:decimal), and the handling of default values based on formulas.
First the optional (minOccurs="0") and non-required (nillable="true") "object" types in XSD: they will still be mandatory in InfoPath when using the WSCF (web services contract first) approach. What you gain is the option to leave these elements out of the InfoPath form's scope. If you include these fields in the scope, you must yourself handle the fields in an OnSubmit event handler, setting a dummy date into the fields. Likewise, you must clear the dummy dates when loading existing data into InfoPath.
Then the default value mechanism of InfoPath: we have some fields that are calculated based on other fields, but the user can override the calculated value by typing a number of their own choice into the field. This seems to function OK with InfoPath, and when you submit the form to the web service, the user's number is there alright. However, when the form loads the stored data later on, InfoPath actually reapplies the default value formula, even when the field already has content. Not exactly the way I expected it to work. The solution is to use InfoPath rules. But even rules has a small quirck; they only trigger on changing the content of a field, thus it is hard to apply "default value when user leaves field empty" logic in InfoPath.
What has annoyed me most this week in InfoPath are the handling of optional and non-required date and numerical elements in XSD (e.g. xs:date and xs:decimal), and the handling of default values based on formulas.
First the optional (minOccurs="0") and non-required (nillable="true") "object" types in XSD: they will still be mandatory in InfoPath when using the WSCF (web services contract first) approach. What you gain is the option to leave these elements out of the InfoPath form's scope. If you include these fields in the scope, you must yourself handle the fields in an OnSubmit event handler, setting a dummy date into the fields. Likewise, you must clear the dummy dates when loading existing data into InfoPath.
Then the default value mechanism of InfoPath: we have some fields that are calculated based on other fields, but the user can override the calculated value by typing a number of their own choice into the field. This seems to function OK with InfoPath, and when you submit the form to the web service, the user's number is there alright. However, when the form loads the stored data later on, InfoPath actually reapplies the default value formula, even when the field already has content. Not exactly the way I expected it to work. The solution is to use InfoPath rules. But even rules has a small quirck; they only trigger on changing the content of a field, thus it is hard to apply "default value when user leaves field empty" logic in InfoPath.
Tuesday, April 05, 2005
Debug the InfoPath submit SOAP message
I am currently building a prototype for using web services contract first (WSCF from Thinktecture) to build web services to support InfoPath as the client. Setting up InfoPath (SP1) to query and submit to the web service is straightforward, and getting data from the web service worked immediately. I could not, however, get submitting the form to the web service to work, and I suspected my setup of which part of the XML to post to be incorrect.
During my attempts to get submit working, I needed to spy on the SOAP messages generated by InfoPath to see why the posted XML could not be deserialized according to my message schema. As I am a long time fan of Altova XmlSpy, I decided to try out their XmlSpy SOAP Debugger. The XmlSpy integration with VS.NET makes working with XSD schemas quite easy.
[UPDATE] XmlSpy is still great, but you might find that Fiddler is sufficient for inspecting the HTTP request/response traffic to between InfoPath and your web-service.
Follow the steps outlined in the above link to configure and start the SOAP debugger, noting that the debugger is a proxy process that sets up a new HTTP port (:8080) that your SOAP traffic must be sent to. The debugger proxy then does its magic and finally forwards the traffic to the original port (:80).
So to be able to intercept and spy on the SOAP message submitted by InfoPath, you must open the form and change the data connection used by submit. In the first step of the 'Data Connection' wizard, you must change the location in 'web service details (submit data)' to use the SOAP debugger proxy at port :8080. Then complete the remaining steps of the wizard, selecting the correct data to use as the 'submit operation' parameters. Close the wizard, save the form and then test it using 'Preview Form'. When you now submit the form, XmlSpy will break on the selected operation and you can review the SOAP request made by InfoPath.
My mistake was that I had chosen to include 'XML subtree...' instead of 'Text and child elementes only', thus InfoPath posted my main parameter element twice, nested within itself (<message><dto><dto>...). I would most likely have found the correct configuration anyway, but using XmlSpy certainly saved me a lot of time.
During my attempts to get submit working, I needed to spy on the SOAP messages generated by InfoPath to see why the posted XML could not be deserialized according to my message schema. As I am a long time fan of Altova XmlSpy, I decided to try out their XmlSpy SOAP Debugger. The XmlSpy integration with VS.NET makes working with XSD schemas quite easy.
[UPDATE] XmlSpy is still great, but you might find that Fiddler is sufficient for inspecting the HTTP request/response traffic to between InfoPath and your web-service.
Follow the steps outlined in the above link to configure and start the SOAP debugger, noting that the debugger is a proxy process that sets up a new HTTP port (:8080) that your SOAP traffic must be sent to. The debugger proxy then does its magic and finally forwards the traffic to the original port (:80).
So to be able to intercept and spy on the SOAP message submitted by InfoPath, you must open the form and change the data connection used by submit. In the first step of the 'Data Connection' wizard, you must change the location in 'web service details (submit data)' to use the SOAP debugger proxy at port :8080. Then complete the remaining steps of the wizard, selecting the correct data to use as the 'submit operation' parameters. Close the wizard, save the form and then test it using 'Preview Form'. When you now submit the form, XmlSpy will break on the selected operation and you can review the SOAP request made by InfoPath.
My mistake was that I had chosen to include 'XML subtree...' instead of 'Text and child elementes only', thus InfoPath posted my main parameter element twice, nested within itself (<message><dto><dto>...). I would most likely have found the correct configuration anyway, but using XmlSpy certainly saved me a lot of time.
Subscribe to:
Comments (Atom)