The Daily Parker

Politics, Weather, Photography, and the Dog

Continuing migrations to Azure

I've finished two complete migrations from my living room the Inner Drive Technology Worldwide Data Center to Microsoft Windows Azure web sites. Astute readers may remember that in one case I moved to the Web site offering and then moved it to a full-fledged Web role. Well, today, I moved it back. Even though I'm still on the free trial, it turned out that the Web role would cost $15 per month, which, for a site that gets one or two visitors per day, simply wasn't worth it.

Moving the second site, a silly thing from 2004 created to share photos and commentary about a 10-year series of Presidents Day parties a friend of mine hosted back in the day, went a lot more smoothly:

0. I archived the project after its last deployment to the Web, which was in November 2006, when the Web server went online. (The age of this particular server is one of the reasons I'm moving everything to Azure, in fact.) I use SourceGear Vault for source control. So all the source code still lived in the Obsolete repository.

1. The old source code was C# 1.1 code from 2004, so the old project and solution files were in Visual Studio 2003 format. Forget upgrading; I just created a new solution.

2. Some of the code files in the Obsolete area were shared with other, obsolete projects. I branched them.

3. In source control, I moved files from the obsolete folder to new folder, except for obsolete configuration and code files. Vault doesn't automatically fetch the files when you do this, to prevent disasters. That was helpful.

4. Here I ran into a minor Visual Studio annoyance with its project naming schemes. I want the Web project to have a different, standard name for Web projects, so I had to rename it by hand in the file system and in the .sln file. No big deal.

5. Now I bound the new solution to source control. The procedure has to do with my source control setup, and may not be typical:

  • Got latest files from Vault in their new locations.
  • Added the files to the solution and project.
  • Closed the solution
  • In Vault, manually added the .sln and .csproj files
  • Reopened solution and changed the source control bindings

6. Added references to current version of Inner Drive Extensible Architecture:

  • Wound up replacing global.asax.cs with the version from Boxer's Shorts. The party site's was too old to bother updating.
  • Added references to InnerDrive.Azure, the Azure mail provider, etc.

7. Reviewed the code throughout the project to ensure nothing in them should prevent a quick compile:

  • Converted the project files to Web application. This adds designer files to all Web forms with code-behind files, so Visual Studio can compile the site down to a deployable assembly.
  • Went through each page to make sure the designers and the pages match
  • Built the project (without running it): OK.

At this point I'd spent about 50 minutes on it. So far, so good.

8. Updated configuration files (web.config, email configuration, etc.) to match the new versions of Inner Drive code files.

9. All Inner Drive websites get page metadata from a data tables, so I had to add the site's metadata to the SQL Database running in my Azure account. This is scripted, and the scripts, moreover, are idempotent to allow for easy automation.

10. It turned out that half the photos and two of the pages on the site weren't in source control after all. Short pause to fetch them from the current Web server and add them to the repository.

11. Compile and run in Debug mode: Yay! Evertyhing works! Time elapsed: 90 minutes (including getting my salad out of the fridge and answering a colleague's questions about something unrelated.)

12. In the Windows Azure portal, created a new Web site, and dowloaded its publish profile to the Visual Studio solution.

13. Right-clicked the Web project and clicked "Publish..."

14. While the publication went on (there were 3.6 Mb of photos to copy up to Azure), updated the production Pages tables with the site's page metadata.

15. Publication completed and...crashed immediately. Crap.

16. Uploaded a new web.config file with <customErrors mode="Off">

"The page cannot be displayed because an internal server error has occurred." That's not helpful. And it led to some thrashing before I realized the PEBCAK:

  • Removed the local-only sections of web.config
  • Stripped down the MessageConfig.xml to do absolutely nothing
  • Check the /LogFiles folder on the virtual machine about 20 times only to see it's empty
  • Check the Azure portal's "Configure" tab and turned on diagnostics
  • Oops. There it is: "Configuration file is not well-formed XML"
  • Uploaded a new web.config file with <customErrors mode="Off" /> (notice the terminating slash).

17. "Could not load file or assembly 'Microsoft.WindowsAzure.ServiceRuntime, Version=, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified." No problem, I've seen this before. Fixed it.

18. Reloaded the page: success! Except all of my messaging was turned off from the debugging at step 16, so I just had to add them back.

19. Success!

And it only took 2½ hours, including lunch and talking to colleagues about something else. Next time, I hope to do it in a lot less time.

Unfortunately, I can't share the URL, because the content is old and unflattering. Those of us who attended the parties may want to remember them, or figure out what happened in cases where we can't actually remember. But it's not for general consumption.

Hot hot hot

The temperature at O'Hare just hit 35°C, and it's still rising. This is the 7th day in a row of above-32°C temperatures, and it looks like we're in for another 4 more days of it—including 38°C tomorrow and 39°C Thursday.

But that isn't the main weather story of the day; the drought is:

The latest USDA Illinois Weather and Crops report was released this afternoon. The topsoil conditions were rated at 52 percent “very short” and 37 percent “short” and only 11 percent adequate. Soil moisture conditions were best in northern Illinois, and deteriorated southward. Hardest hit was southeastern Illinois with 100 percent of the topsoil and 100 percent of subsoil rated as “very short”.

Corn and soybeans—Illinois produces 18% of the country's corn and 16% of the county's soybean— make up 73% of the state's gross agricultural product, amounting to $7.2bn per year. This summer is bad, but possibly not as bad as 1988. Yet.

It is finished.

I have successfully ported my first (existing) application to the Microsoft Windows Azure platform, and have shut down the running instance on my local Web server. I hope the second one takes less than a week.

It's a funny little site called Boxer's Shorts. Dr. Bob Boxer is a local allergist who likes puns. He worked with a local illustrator, Darnell Towns, and self-published the five paperback pun compilations advertised on the site.

Local web designer Lauren Johnson (née Liss) did the look and feel, and I provided the platform. I think we completed the site in two weeks or so. I've hosted it since it went live in September 2006—just a few days after I got Parker, in fact.

And now it's in the cloud, the first Inner Drive site to be ported. From what I learned doing it, I hope to get two more of my older sites deployed to Azure this week.

Why I haven't finished deploying to Azure

I've spent much of the past week trying to get a single, small website up into the cloud on the Windows Azure platform. Much of this effort revolved around the Azure Website product, mainly because it's free. Well, I got the application up as an Azure website...and there's a big problem with it that means I'll have to redeploy it as a Web role after all.

First, let me just outline how much fun I've had today, starting from this morning when I first tried to publish the application to the cloud:

  • For the first hour or so, I dealt with a missing patch that prevented publishing entirely.
  • My next task, after I got the bits up to Azure, was to track down why the application failed on the method RoleEnvironment.IsAvailable, which you need if you want to deploy something to an Azure Web role. First I got this:
    Could not load file or assembly 'Microsoft.WindowsAzure.ServiceRuntime, Version=, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
  • The solution was simply to go to the Solution Explorer and mark the ServiceRuntime assembly as "copy local" instead of "do not copy." That fixed it. Until the next time I tried to run, when I got this:
    Could not load file or assembly 'msshrtmi, Version=, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
  • Fortunately, developer Jake Ginnivan had exactly this problem yesterday. (Wow, who doesn't love Google?) Of course, that required me to compile the whole site as x86, not x64:

At this point, the entire site seemed to work. Except now I wasn't getting any exception messages. Nor was I able to get a "contact us" email. Some digging showed that my code—yes, my code—was swallowing exceptions, and throwing the wrong exceptions.

This is because the Inner Drive Extensible Architecture message system loads configuration if you don't provide it, getting the configuration file spec from the web.config file. Only, in every single instance where the thing is in production, I control the file system, so I've never had a problem knowing where the configuration file actually lives. (I also solved the problem of how to get configuration from Azure storage, as a pain I described earlier today.) The configuration setting looks like this:

   <add key="messageConfigFile" value="c:\SomeActualPath\MessageConfig.xml"/>

You can also use a relative path if you know the root. On Azure web sites, it turns out the root is D:\Windows. Seriously. On my little slice of a shared server, the actual path to the application is something like C:\DWASFiles\Sites\Site-Name\VirtualDirectory0\site\wwwroot\. The "virtual" part scares me; this path is not guaranteed. This is what we call in programming "brittle."

So I wound up using a relative path from the application root and adding this code to the application start method:

var configurationFile = ConfigurationManager.AppSettings["messageConfigFile"];
var fileSpec = Server.MapPath(configurationFile);

I deployed it again and yay! After adding that bit of code, and spending a total of 6.9 non-billable hours on this, I finally got the application to run as a free Azure web site with its address. So all I have to do now is add a CNAME record pointing the site's real URL to this one and...

You're kidding me.

Free Azure websites running on shared instances only allow a single hostname per site:

This means that I can't use to point to, which means the whole effort is kind of wasted, except to use the thing as a staging site. Moving to a reserved instance solves the problem, but costs 8c per hour of compute time. But for only 2c and hour, I can use a traditional Web role in an extra-small instance. I'm pretty sure this particular site won't run continuously, but still, why pay any more than required?

Almost everything I did today also applies to publishing the application to a Web role, but now I have to pay money for it. Dang.

Why Parker is disappointed in me right now

Yes, I just said I was taking Parker out for a walk, but I cut it short after five minutes. Here's why:

Just as we got back home the gust front hit. Trees are now moving in ways that trees probably shouldn't. This should be a lot of fun to watch.

...but Parker is sulking. Tant pis, mon bête noir.

Update, 1:25 pm: Huh. The storm just missed us, though reports have come in of 145 km/h gusts in Elmhust and Lombard, which "looks like a war zone" according to the Tribune.

Who may I strangle, please?

In the past week, I've been "on the bench" at work, so I've take the time to get deeply familiar with Microsoft Windows Azure. My company, 10th Magnitude, is a 100% cloud-computing shop, and a Microsoft partner. I've been developing for Azure Web applications for a year, but I haven't had to deal with migrating existing sites, pricing, or configuration on my own; this is why we're a team, right?

So, anyway, I've taken what I've learned at work, and:

  • Selected a simple website to migrate; in this case, Boxer's Shorts, a project I completed in 2006 that has five rows of data and five content pages;
  • Worked for about 16 hours this week creating Azure implementations of the Inner Drive Extensible Architecture features that won't work on Azure—but only the two features (messaging and logging) that Boxer's Shorts needs;
  • Spent 6 hours yesterday and an hour today preparing the application for Azure; and
  • Set up an Azure Web site this morning into which I was about to publish the ready-to-roll Web site.

Remember the messaging and logging services I spent lots of time migrating? Well, it looks like I made the right choice in the long run. The services use Azure tables and blobs for logging, holding the site configuration data, and in this site's case, for holding the list of books. Azure storage is really, really cheap, less than 10c per gigabyte for locally-redundant storage or 12.5c per gigabyte for geo-redundant storage. This is de rigeur for a traditional Azure Web role deployment for everything that can run without relational data. (For relational data you need SQL Server.)

I thought—mistakenly, it turns out—that if it worked in a Web role it would work in a Web site. No, not so much. In the short run, I just discovered this:

Ah. No blob storage for Azure web sites. So now I have to strip out all of the stuff that uses blob storage on this web site, and modify the book list to use SQL Server instead of Azure tables. Two more hours.

So why not just publish the site to a Web role instead? Price. With Azure, you get 10 free Web sites with your subscription; but each Web role costs at least 2c per hour for the smallest possible footprint. There are 720 hours in a month, so even though you only pay for the time the application is actually doing something, you have to plan for about $14 per month. For a site that gets 100 page-views per week, has five content pages, and five pieces of data, that's really a lot of money. And it's infinitely more than free.

All right then. I hope that Azure websites get access to Azure storage soon. For now, I'm just going to rip the logging out of the site and fix the rest of it. But first I'm going to walk the dog.