Azure Virtual Machines do not come with a Sysadmin

Don’t be fooled (as I was) into thinking that the pre-made Windows Azure Virtual Machine images do not require any maintenance out of the box.

I’ve spent most of the day attempting to migrate a .NET web application running in a Web Role (PAAS) over to a Persistent Virtual Machine (IAAS).

The same code that works locally (IISExpress, Windows 7), on our dev server (IIS 7.5, Server 2008 R2) and in the current web role would not run on the new virtual machine.

The cause it seemed was that I was using the Azure Storage Client to set up some containers in Azure Blob Storage during Application_Start - well at least this was all I could gather from the stack trace. The exception:

Response is not available in this context

I say this was the cause, but I didn’t have the same issue in any other environment. At this point you know it is going to be a long day.

Sascha Dittman was kind enough to spend some time troubleshooting with me but it looked like the only thing to do was to defer executing this code until we had a http context.

That annoyed me. The code worked in our other environments without any issue. Also, why did I need a HttpResponse to use the Storage Client? I could use the Storage Client in a console application after all.

Sascha pointed me to a post by Ingvar Jensen who had run into a similar issue. He hinted that maybe Azure used the Response object for logging.

I started to dig through the Storage Client code (now opensource) looking for any kind of dependency on HttpContext. Unfortunately this yielded no results.

Tearing my hair out and googling frantically I came across a bug reported to Microsoft by Joshua Flanagan where he was unable to use HttpUtility.HtmlEncode during Application_Start. The error: “Response is not available in this context”.

Unfortunately there was no reference to any kind of hotfix. The only update from Microsoft was that “this is fixed in ASP.NET 4.5”. Not so useful for my .NET 4 application!

But now we had something to go on. I made a call to HttpUtility.HtmlEncode in Application_Start and pushed the code up to my Windows Azure Virtual Machine. Sure enough it through the same exception.

Now I did a search within the Storage Client source for all HttpUtility references. Inside the CloudBlobContainer (what I was using to create a Blob Container) I found the following method:

private void ParseQueryAndVerify(Uri completeUri, CloudBlobClient existingClient, bool? usePathStyleUris)
{
    // ...

    var queryParameters = HttpUtility.ParseQueryString(completeUri.Query);
    SharedAccessSignatureHelper.ParseQuery(queryParameters, out sasCreds);

    // ...
}

I wonder?? I updated my Application_Start method to the following and pushed up to the VM:

protected void Application_Start()
{
    HttpUtility.ParseQueryString("foo=1&bar=2");
	
	// ...       
}

Sure enough - “Response is not available in this context.”.

This was clearly not an issue with the Storage Client. This was an issue with .NET 4.0 itself. Since I didn’t have the problem on our dev server, there had to be a patch.

Back on the Windows Azure Virtual Machine I trawled through the release notes of most of the “Important” updates. Nothing!

In desperation I move onto the optional updates and came across the following:

Update for the .NET Framework 4 - KB2468871.

In the release notes, issue number 12:

In the .NET Framework 4, the Application_Start and PreAppStart methods do not have access to the HttpUtility.HtmlEncode method and to the related APIs.

I installed the hotfix and my application worked. What a day!

So the moral of the story is don’t assume your Windows Azure Virtual Machines come with a SysAdmin. They don’t. It is your reponsibility to maintain them like you would any other server. In this instance, even Automatic Updates would not have helped me.

Hopefully tomorrow will be a little more productive.