Character encoding in SPS2003 and regional settings

Most of our SharePoint installations, both SPS and WSS, uses the Norwegian version or at least require that areas, surveys, listings and fields added to SPS can handle Norwegian characters (ÆØÅ). The character encoding is configured in WEB.CONFIG using the globalization element, and this defaults to fileEncoding="utf-8". What ever you do, do not change this into e.g. fileEncoding="iso-8859-1" to get support for the encoding you use in your web parts.

The problem you can get into is that SPS requires unicode text (utf-8 encoding) and when a listing field or a web part contains non-unicode text (e.g. iso-8859-1 encoded), this will cause the unicode parsing to fail because the Norwegian characters signals an escape sequence to the parser, which it really is not. The parser detects an invalid utf-8 escape sequence and outputs a ? instead of the expected and following characters. This is very annoying in listing field names as it causes the list rendering to fail and then the list cannot be modified to fix this problem.

We have experienced problems with the encoding when deploying web parts that are really ASP.NET user controls. To make this kind of web parts render correctly, and allow the standard SPS stuff to work correctly with ÆØÅ, you must ensure that the globalization element has this configuration:

fileEncoding="utf-8" requestEncoding="utf-8" responseEncoding="utf-8"

Then you must ensure that your ASP.NET user control's .ASCX file uses unicode as the encoding by using "File - Advanced Save Options" in VS.NET. Select "Unicode (UTF-8 with signature) - Codepage 65001" as the encoding, then save the file. Switch to HTML view if the menu item is not visible in design mode. The same procedure must be applied to all .ASCX and .ASPX files that you use as part of a SPS solution. Read more about Unicode in ASP.NET in this blog.

In addition, you must always set the correct regional settings for your SPS portal site (once for all areas) and for all your WSS team sites (on every single site).

Note that the WEB.CONFIG file is replaced when restoring a SharePoint portal site, and this will cause all your changes to the configuration to be lost. Always make a backup copy of WEB.CONFIG before restoring portal sites.