Politics, Weather, Photography, and the Dog
Sunday 10 February 2013

Last night I made the mistake of testing a deployment to Azure right before going to bed. Everything had worked beautifully in development, I'd fixed all the bugs, and I had a virgin Windows Azure affinity group complete with a pre-populated test database ready for the Weather Now worker role's first trip up to the Big Time.

The first complete and total failure of the worker role I should have predicted. Just as I do in the brick-and-mortar development world, I create low-privilege SQL accounts for applications to use. So immediately I had a bunch of SQL exceptions that I resolved with a few GRANT EXEC commands. No big deal.

Once I restarted the worker role, it connected to the database, loaded its settings, downloaded a file from NOAA and...crashed:

Inner Drive Weather threw System.Data.Services.Client.DataServiceRequestException

One of the request inputs is out of range.

at System.Data.Services.Client.DataServiceContext.SaveResult.d__1e.MoveNext()

Oh no. The dreaded Azure Storage exception that tells you absolutely nothing.

Flash forward fifteen minutes (now past midnight; and for context, I'm writing this on the 9am flight to Los Angeles), with Fiddler running on a local instance connecting to production Azure storage, and I found the XML block on which real Azure Storage barfed but the Azure storage emulator passed without a second thought. The offending table entity is metadata that the NOAA downloader worker task stores to let the weather parsing worker task know it has work to do:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
   <entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" 
  <title />
    <name />
  <id />
  <content type="application/xml">
      <d:FileTime m:type="Edm.DateTime">2013-02-09T05:35:00Z</d:FileTime>
      <d:IsParsed m:type="Edm.Boolean">false</d:IsParsed>
      <d:ParseTime m:type="Edm.DateTime">0001-01-01T00:00:00</d:ParseTime>
      <d:RetrieveTime m:type="Edm.DateTime">2013-02-10T05:55:29.1084794Z</d:RetrieveTime>
      <d:Size m:type="Edm.Int32">68202</d:Size>
      <d:Timestamp m:type="Edm.DateTime">0001-01-01T00:00:00</d:Timestamp>

Notice that the ParseTime and Timestamp values are equal to System.DateTimeOffset.MinValue, which, it turns out, is not a legal Azure table value. Wow, would it have helped me if the emulator horked on those values during development.

The fix was simply to make sure that neither System.DateTimeOffset.MinValue nor System.DateTime.MinValue ever got into an outbound table entity, which took me about five minutes to implement. Also, it turned out that even though my table entity inherited from TableServiceEntity, I still had to set the Timestamp property when using real Azure storage. (The emulator sets it for you.)

By this point it was 12:30 and I needed to get some sleep, however. So my plan to run an overnight test will have to wait until this evening at my hotel. Then I'll find the other bits of code that work fine against the emulator but, for reasons that pass understanding, the emulator gets completely wrong.

Sunday 10 February 2013 10:27:27 CST (UTC-06:00)  |  | Software | Cloud | Windows Azure#
On this page....
When the Azure emulator is more forgiving than real life
The Daily Parker +3617d 08h 01m
Whiskey Fest 21d 01h 42m
My next birthday 330d 15h 06m
Parker's 10th birthday 250d 05h 12m
Aviation (380) Baseball (114) Best Bars (10) Biking (46) Chicago (1039) Cubs (199) Duke (134) Geography (386) Higher Ground (5) Jokes (284) Kitchen Sink (743) London (86) Parker (207) Daily (204) Photography (171) Politics (312) US (1185) World (297) Raleigh (21) Readings (8) Religion (69) San Francisco (96) Software (235) Blogs (93) Business (263) Cloud (96) Cool links (157) Security (113) Travel (302) Weather (782) Astronomy (108) Windows Azure (65) Work (112) Writing (15)
<October 2015>
Full archive
David Braverman and Parker
David Braverman is the Chief Technology Officer of Holden International in Chicago, and the creator of Weather Now. Parker is the most adorable dog on the planet, 80% of the time.
All content Copyright ©2015 David Braverman.
Creative Commons License
The Daily Parker by David Braverman is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License, excluding photographs, which may not be republished unless otherwise noted.
Admin Login
Sign In
Blog Stats
Total Posts: 4990
This Year: 376
This Month: 12
This Week: 8
Comments: 0