August 11, 2011

Windows Azure - My experience so far

The other day I launched fabrik, a multi-tenant portfolio and blogging service running on the Windows Azure platform. Prior to this I had absolutely no experience of “cloud” development.

When I start working with a new technology I like to gain a good level of understanding before I write code. Don’t get me wrong, using the technology is one of the best ways to learn, but I think that with Azure, you need to learn the fundamentals first.

For me it was a case of attending a few web seminars, reading some books and watching a few video tutorials.

The books I read:

The first book is more of a case study than a reference book so I would recommend reading the “Step by Step” book first, to get a good all-round understanding of Azure.

Once I felt I had a good understanding of what it was all about, I started working through the samples available on the Azure web site. Then I got bored and just started working on a “real-world” application.

For me, this is where the problems started. You see, there are no shortage of articles that explain the pitfalls of migrating on-premise applications to the cloud, in particular:

  • Session State
  • Storage
  • Configuration

In an on-premise application the above resources are normally local to the application. Session state is often stored in the memory of the running application, storage will be your local file system and configuration in a local configuration file (e.g. web.config). This presents a problem when migrating to a cloud environment where you may have several instances running of your application.

So you will see examples like this.

void SaveFile(byte[] fileContent, string fileName)
{
    File.WriteAllBytes(fileName, fileContent);
}

Here we are touching the local file system directly. This is no good when we “migrate to the cloud” so we’ll need to change our code to use Blob storage:

void SaveFile(byte[] fileContent, string fileName)
{
    var account = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
    var client = account.CreateCloudBlobClient();

    var container = client.GetContainerReference("public");
    container.CreateIfNotExist();

    var blob = container.GetBlobReference(fileName);
    blob.UploadByteArray(fileContent);
}

Great, so now we can save files to Azure Blob Storage. But wait, there’s a problem here, and a fundamental problem in how Microsoft are teaching Azure - all we’ve done is swap out one tightly coupled implementation (using the local file system) with another (blob storage).

The fact is that Microsoft should not be telling us how to migrate applications to Azure, they should be helping developers to write applications that can run in both on-premise and cloud environments.

The frustrating thing is that if you look through some of the more advanced learning resources (say the Tailspin scenario), we do get to see how we can abstract use of cloud storage to create loosely coupled applications that are easier to test. However, Microsoft are not pushing this, which is only going to end badly for developers new to Azure.

These abstractions are important, not just for supporting multiple environments, but also for development and testing. Within about 10 minutes of working with Azure for the first time I fell out with the Azure Emulator. It’s incredibly slow to use, especially when debugging, and a PITA when testing.

But how we can develop our “cloud” applications without the emulator. Simple, don’t couple your application to Azure resources. Write some abstractions with Azure implementations that can be tested separately, then use the local (on-premise) implementations for development.

I did exactly that. I created abstractions around working with the file system, message queues (using MSMQ locally) and configuration. My entire application (two web apps and one “worker”) was developed without any of the Windows Azure development tools.

This approach really helped when I came to deployment. Even before I had actually tested my application in Azure compute (via the emulator) I was able to run in a kind-of hybrid mode. In less than 5 minutes (really a case of switching over to the Azure implementations of my storage/queue/configuration abstractions) my application was running locally but using Azure Storage and SQL Azure.

Finally, I was able to test the application running in the emulator, making sure that everything worked with multiple instances of my web and worker roles. Then I deployed to the cloud.

My file system abstractions were particularly interesting. Essentially there are two web sites, the admin site and a public site. When running in the cloud, both sites use the same storage abstraction (using blob storage). The admin site is responsible for saving images and other files to blob storage and the public site is predominantly displaying files stored in blob storage.

When it came to working this locally it was a bit more tricky. I actually ended up with an implementation for the admin site that saved files to a directory in the root of the site, and an implementation for the public site that just appended the public address of this directory (e.g. http://localhost:65656/public) to the file name. Since we only store the file name in our database and not the full path, it makes it really easy to switch providers if we need to. For example we would be able to move all our files over to Amazon S3, simply by writing a new implementation of our storage abstraction.

I’m going to be releasing these abstractions on NuGet in the next few weeks. I can’t take all the credit as much of the code has come from various samples I’ve found online including the Orchard Project.

This really just reiterates the point - lots of people are doing the same thing. So why don’t Microsoft give us a standard set of abstractions that we can use? It’s about time Microsoft followed the trend of some other frameworks and started being a bit more opinionated with it’s products.

© 2020 Ben Foster