Use Azure VMs with On Premises Gateway as SP2010 Branch Office Farm

Here are some notes from my ongoing experience of setting up a SharePoint 2010 "branch office" farm in Azure using the current preview of persistent Azure virtual machines in a Azure virtual network connected to the on premises Active Directory using an Azure gateway to a Juniper VPN device.

Useful TechEd Europe 2012 web-casts that show how things work in general:
VM+VNET in Azure: http://channel9.msdn.com/Events/TechEd/Europe/2012/AZR208
AD in Azure: http://channel9.msdn.com/Events/TechEd/Europe/2012/SIA205
SP in Azure: http://channel9.msdn.com/Events/TechEd/Europe/2012/OSP334

Step by step instructions that I followed:
How to Create a Virtual Network for Cross-Premises Connectivity using the Azure gateway.
How to Install a Replica Active Directory Domain Controller in Windows Azure Virtual Networks.

Creating a virtual network (vnet) is easy and simple using the Azure preview management portal. I recommed creating the local network first, as the vnet wizard otherwise might fail - without giving any useful exception message. We had an issue caused by naming the local network "6sixsix" which didn't work due to the name starting with a digit. Also note that the VPN gateway only supports one LAN subnet in the current preview.

Plan your subnets upfront and make sure that they don't overlap with the on premises subnets. Register both the existing on premises DNS server and the planned vnet DNS server when configuring the vnet. A tip here is that the first VM created in a subnet will get .4 as the last part of the IP address, so if your ADDNSSubnet is 10.3.4.0/24, then the vnet DNS will get 10.3.4.4 as its IP address. Note that you can't change the DNS configuration after adding the first VM to the network, this includes creating the Azure gateway which adds devices to the gateway subnet.

After creating the Azure virtual network, we created and started the Azure gateway for connecting to the on premises LAN using a Site-to-Site VPN tunnel using a secure IPSec connection. Creating the gateway takes some time as some devices or VMs are provisioned in the gateway subnet you specified. We then sent the public IP address of the gateway, plus the shared key and the configuration script for the Juniper VPN device to our network admin. The connection wouldn't come up, and to make a long story short, the VPN configuration needs the 'peerid' to be set to an IP address of a device in the gateway subnet. Our gateway subnet was 10.3.1.0/24 and after trying 10.3.1.4 first (see above tip), the network admin tried 10.3.1.5 and that worked. I'll come back to this below when telling you about our incident when our trial Azure account was deactivated by Microsoft.

With the Azure virtual network up and running and connected to the on premises LAN, I created the AD DNS virtual machine using the preview portal "create from gallery" option. As SP2010 is not supported on WS2012 yet, I decided to use the WS2008R2 server image in this Azure server farm. Note that you should use size "large" for hosting AD as you need to attach two data disks for storing the AD database, log files and system state backup.

I did not use powershell for creating this first VM, instead I manually changed the DNS setting on the network adapter (both IPv4 and IPv6) and then manually joined the to-be AD DNS VM to the existing domain. Note that while you're at it, also set the advanced DNS option "Use this connection's DNS suffix in DNS registration" for both network adapters. Otherwise you will get the "Changing the Primary Domain DNS name of this computer to "" failed" error when trying to join the domain.

Following the how-to for setting up a replica AD in Azure work fine, we only had some minor issues due to the existing AD being on WS2003. For instance, we found no DEFAULTIPSITELINK when creating a new site in AD, so we had to create a new site link first, then create the site and finally modify the site link so that it linked the Azure "CloudSite" and the LAN site. Then the dcpromo wizard step for AD site detection didn't manage to resolve against our WS2003 domain controller, just click "ok" on the error message and manually select the "CloudSite" in the "Select a site" page.

I really wanted to set up a read-only domain controller (RODC) to save some outgoing (egress) traffic and thus save some money, as this branch farm don't need a full fidelity domain controller. However, it is not possible to create a RODC when the existing DC is on WS2003, because RODC is a WS2008 feature. So for "Additional Domain Controller Options" we went with DNS and "Global Catalog" (GC). GC isn't required, but if not installed then all authentication traffic on login need to go all the way to the on premises DC. So to save some traffic (and money), and get faster authN in the branch farm, we added the GC - even if the extra data will drive up Azure storage cost.

The next servers in the farm were added using powershell to ensure that 1) the VM is domain joined on boot, and 2) that the DNS settings for the VM is automatically configured.

Here are some tips for using New-AzureDNS, New-AzureVMConfig and New-AzureVM:

• You can use the Azure vnet DNS server or the on premises DNS server with New-AzureDNS. I used the former.
• The New-AzureVMConfig Name parameter is used when naming and registering the server in AD and DNS. Make sure that the full computer name is unique across the domain.
• The New-AzureVM ServiceName parameter is used for the cloud DNS name prefix in the .cloudapp.net domain. It is also used to provision a "Cloud Service" in the Azure preview management portal. Even if multiple VMs can be added to the same service name (shared workload), I used uniqe names for the farm VMs (standalone virtual machine), connected using the vnet for load balancing.
• To get the built-in Azure image names, use this powershell to go through the images in the gallery until you find the one you're looking for:
         (Get-AzureVMImage)[1].ImageName

After adding the SQL Server 2012, web server and application server VMs using powershell, I logged in using RDP and verified that each server was up and running, domain joined and registered in the DNS. Note that the SQL Server image is not by default configured with separate data disks for data and log files. This means that the SQL Server 2012 master database etc is stored on the OS disk in this preview. You need to add data disks and then change the SQL Server file location configuration your self. Adding two data disks will require that the SQL Server VM is of size "large".

The next step was to intall SharePoint 2010 on the farm the next day. Thats when the trial account was deactivated because all the computing hours was spent. Even if you then reactivate the account, all your VM instances are deleted, keeping only the VHD disks. As Microsoft support says, it is easy to recreate the VMs, but they also tell you that the AD virtual machine needs a static IP which you can only get in Azure if you never delete the VM. Remember to recreate the VMs in the correct order so that they get the same IP addresses as before.

What is worse is that they also delete the virtual network and the gateway. Even if it is also easy to recreate these, your gateway will get a new public IP address and a new shared key, so you need to call your network provider again to make them reconfigure the VPN device.

I strongly recommend not using a spending capped trial account for hosting your Azure branch office farm. Microsoft deleted the VMs and the network to stop incurring costs, which was fine with non-persistent Azure VM Roles (PaaS) anyway, but not as nice for a IaaS service with a persistent server farm.

I recommend exporting your VMs using Export-AzureVM so that you don't have to recreate the VMs manually if something should happen. The exported XML will contain all the VM settings, including the attached data disks.

How to deatch Azure VMs to move or save cost: http://michaelwasham.com/2012/06/18/importing-and-exporting-virtual-machine-settings/

When we recreated the Azure virtual network and the gateway, the VPN connection would not come back up again. The issue was that this time the gateway devices had got different IP addresses, so now the "peerid" had to be configured as 10.3.4.4 to make things work.

Now the gateway is back up again, and next week I'll restore the VMs and continue with installing SP2010 on the Azure "branch office" farm. More notes to come if I run into other issues.

- - - continued - - -

Installing the SharePoint 2010 bits went smooth, but running the config wizard did not. First you need to allow incoming TCP traffic on port 1433 on the Azure SQL Server. Then the creation of the SharePoint_Config database failed with:

    Could not find stored procedure 'sp_dboption'.

...even if I had downloaded and installed SP2010 with SP1 bits. So I downloaded and installed SharePoint Server 2010 SP1 and June CU from 2011 due to the issue caused by using SQL Server 2012 and that fixed the problem. Got "Configuration Successfull" without any further issues.

Finally, I tested and verified it all by creating a SP2010 web-application with a team site, creating a self-signed certificate with IIS7 and adding an Azure port mapping for SSL (virtual machine endpoint, TCP 443 to 443), allowing me to login to the team site using my domain account over HTTPS from anywhere.

A note on the VM firewall config is that ping is by default blocked, thus you can't ping other machines in the vnet unless you configure the firewall to allow it. Also note that you can't ping addresses outside of the virtual network and the connected LAN anyway; even if you can browse to www.puzzlepart.com, you can't ping us.

Problem creating a FAST Content SSA in SharePoint 2010

While installing Fast Search Server for SharePoint 2010 (FS4SP) on a dev farm today, I got a problem with the provisioning of a new FAST Content SSA (Search Service Application), it would hang forever at "0:01 Configuring the Search Service..." waiting for the TopologyConfigFinish.aspx page to complete.

The problem turned out to be that the SharePoint 2010 Administration service wasn't started after the mandatory server reboot after installing FS4SP. The FAST "nctrl status" cmdlet does not check this. Make sure that both the SP2010 Administration and Timer services are running:

If you still can't create new or delete search service application instances, or make topology changes at all, then you might need to delete the old SSA the hard way. See Deleteing the search service application and How to delete orphan configuration objects from SharePoint farm. Heed this warning: "Please be VERY careful when executing the deleteconfigurationobject command, if this command is not used in the correct way (if you end up deleting the wrong object) there is NO way to revert back the changes and it has the potential to render your Configuration Database useless, hence you may require to restore / rebuild your SharePoint farm".

Remember to configure SSL enabled communication again when recreating the FAST Content SSA, otherwise your next crawl will be stuck on starting while retrying every 60 seconds to connect to the document engine. Also remember to restart the FAST Search for SharePoint and the SharePoint Server Search 14 services before starting a new full crawl.

Scalable SharePoint 2010 Farm & Services Architecture

This post describes a proposed solution design for providing an intranet collaboration farm capable to scale out to support a publishing portal and extranet collaboration with partners and suppliers. The design is based on a set of diverese sites types identified in an Information Architecture analysis, and the general planning recommendations for these types. In addition, several non-functional requirements is accounted for in the design, such as farm security, robustness and availability.

The SharePoint 2010 farm is designed with scale out support for different future solution areas. The initial farm architecture will be designed to support collaboration for internal users only, with an option to also add an intranet publishing solution. The long term goal of the farm is to eventually support collaboration with external users also, such as partners an suppliers.

All SharePoint 2010 solutions are built from a set of elements ranging from the farm hosting the solutions down to sites that contains the actual functionality and information of the solutions.

SharePoint solutions are deployed into SharePoint web-applications in a SharePoint farm. Web-applications are management containers for site collections. SharePoint uses site collections to structure the sites and subsites that implements the actual functionality of SharePoint solutions.

Site collections bridge logical architecture and information architecture (IA). The design goals for site collections in the model are to create logical divisions of content and functionality, in addition to satisfy requirements for URL design. To satisfy the requirements for URL design, each web application includes a single root-level site collection, such as ://portal/. In addition, a set of managed paths are used to incorporate a second tier of site collections, such as ://portal/HR/ and ://portal/sites/***.

All web-applications in a farm share a group of common SharePoint 2010 service applications that provide shared services such as indexing & search to the farm. I use the term “shared services provider” (SSP) in this post for such a group of service applications. A farm can contain multiple groups of shared services, such as an intranet SSP and an extranet SSP.

The overall SharePoint farm architecture for internal collaboration can be summarized like this (click to enlarge):

The initial SharePoint 2010 farm will be configured to support two solutions: providing team-sites for internal collaboration in groups or projects, and for providing My Sites for employees. These two web-applications will share a group of shared service applications (“default group”) for search, user profiles and enterprise taxonomy.

The User Profile service is required to provide My Sites. In addition, this service application is what provides all social features such as tagging, rating and social bookmarking.

Note that a search service application is needed for more than just crawling and queries, the social tagging functionality in SP2010 also requires search. The reason is that several web-parts such as Note Board, Tag Cloud and Tagged Items in some modes depends on the search service to do security trimming to provide their content. This includes the tag profile page, which use such web-parts. You can also use the search service and the content search, people search and social search capabilities when customizing the user experience of your solutions. A typical example is creating a tag cloud web-part for managed metadata and enterprise keywords columns for lists and libraries, not just for social bookmarking (it is not really social tagging that is implemented in SP2010) with the standard "Tags & Notes" ribbon.

If you use FAST Search for SharePoint 2010 (FS4SP) then the FAST Content SSA will provide the content sources that are crawled for search results, except for people search. The FAST Query SSA has a different set of content sources, including the one feeding the people scope. Only one Search Service Application (SSA) should be associated with your web-application, either FAST or standard SharePoint 2010 Server search. Note that FAST doesn't index the social tags, which affects those social tagging web-parts that depends on search (see above).

The intranet will use Active Directory as the identity provider for authentication in classic mode. The web-application authentication mode can later on be switched from classic to claims-based as needed. Note that it is not recommended to switch the mode from claims-based to classic.

The farm is also designed to scale-out to support future SharePoint 2010 solutions such as a publishing intranet (://puzzlepart). Note that other service applications not shown here might be required to support the functional solution design and future solutions.

The different web-applications in the farm runs in separate application pools to achieve process isolation. This way an error or crash in one application will be isolated and not affect the other web-applications in the farm.

Anywhere access for mobile employees to the intranet sites is provided using Forefront Unified Access Gateway (UAG). This supports a wide range of locations such as at home or on the road, and a wide range of devices from laptops to smart phones.

The overall SharePoint farm architecture target for collaboration with external partners and suppliers can be summarized like this (click to enlarge):

The extended 2011+ version of the farm is scaled-out to support two new solutions; one is the ://puzzlepart web-application for intranet publishing, the other is an extranet web-application for collaboration with external users such as partners and suppliers. The former connects to the intranet shared services provider (“default group”) and will as such reuse and share search, user profiles, and the enterprise taxonomy (metadata and content types) with the existing solutions for team-sites and my sites.

The extranet collaboration solution must not be connected to the intranet SSP for information security reasons. Using a separate group of shared services for all service applications that may contain or expose confidential information is strongly recommended – an extranet SSP (“custom group”). A typical service application of this kind is search. Using separate service applications prevents accidental information exposure due to e.g. misconfiguration of a service application.

The User Profile service application is included in the extranet SSP even if the extranet will not provide My Sites at all. This service is what enables user to do social tagging and bookmarking, and must thus be part of the extranet SSP to provide social features. It is not recommended to connect the extranet to the intranet User Profile service for the same reasons as for the Search service.

Still, some service applications are typically shared across both the intranet and the extranet web-applications. This includes the enterprise taxonomy – you need to provide for consistent classification and tagging of content across all solutions to drive findability. The managed metadata service, including content type hubs, is specifically built for the purpose of being both shared and syndicated across multiple solutions.

Another security aspect is access control; the extranet must use a separate claims-based authentication provider, that can be federated with external identity providers. The intranet and extranet web-applications must trust different providers for authorization. Using separate providers prevents accidental information exposure due to e.g. misconfiguration of site access control and group memberships. Note that multi-mode authentication is preferred over mixed mode for extranet collaboration.

A final aspect of the extended farm design is that the extranet web-application and shared services are run in separate application pools from the intranet. This ensures process isolation to prevent errors and crashes in the extranet processes to affect the intranet solutions. It also isolates malicious use of the extranet process resources from affecting the intranet processes, such as denial of service attacks.

Remote access for external users to the extranet collaboration web-application is provided using Forefront Unified Access Gateway (UAG) combined with ADFS for federated identity and access management (IAM). Using UAG saves you from setting up and managing a separate extranet farm in a DMZ perimeter zone; instead you give a controlled set of external users secure anywhere access to a controlled set of internal applications hosted on the intranet farm.