Politics, Weather, Photography, and the Dog
Sunday 10 February 2013

Last night I made the mistake of testing a deployment to Azure right before going to bed. Everything had worked beautifully in development, I'd fixed all the bugs, and I had a virgin Windows Azure affinity group complete with a pre-populated test database ready for the Weather Now worker role's first trip up to the Big Time.

The first complete and total failure of the worker role I should have predicted. Just as I do in the brick-and-mortar development world, I create low-privilege SQL accounts for applications to use. So immediately I had a bunch of SQL exceptions that I resolved with a few GRANT EXEC commands. No big deal.

Once I restarted the worker role, it connected to the database, loaded its settings, downloaded a file from NOAA and...crashed:

Inner Drive Weather threw System.Data.Services.Client.DataServiceRequestException
...
OutOfRangeInput

One of the request inputs is out of range.
RequestId:572bcfee-9e0b-4a02-9163-1c6163798d60
Time:2013-02-10T06:05:41.5664525Z

at System.Data.Services.Client.DataServiceContext.SaveResult.d__1e.MoveNext()

Oh no. The dreaded Azure Storage exception that tells you absolutely nothing.

Flash forward fifteen minutes (now past midnight; and for context, I'm writing this on the 9am flight to Los Angeles), with Fiddler running on a local instance connecting to production Azure storage, and I found the XML block on which real Azure Storage barfed but the Azure storage emulator passed without a second thought. The offending table entity is metadata that the NOAA downloader worker task stores to let the weather parsing worker task know it has work to do:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
   <entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" 
   xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" 
   xmlns="http://www.w3.org/2005/Atom">
  <title />
  <author>
    <name />
  </author>
  <updated>2013-02-10T05:55:49.3316301Z</updated>
  <id />
  <content type="application/xml">
    <m:properties>
      <d:BlobName>20130209-0535-sn.0034.txt</d:BlobName>
      <d:FileName>sn.0034.txt</d:FileName>
      <d:FileTime m:type="Edm.DateTime">2013-02-09T05:35:00Z</d:FileTime>
      <d:IsParsed m:type="Edm.Boolean">false</d:IsParsed>
      <d:ParseTime m:type="Edm.DateTime">0001-01-01T00:00:00</d:ParseTime>
      <d:PartitionKey>201302</d:PartitionKey>
      <d:RetrieveTime m:type="Edm.DateTime">2013-02-10T05:55:29.1084794Z</d:RetrieveTime>
      <d:RowKey>20130209-0535-41d536ff-2e70-4564-84bd-7559a0a71d4d</d:RowKey>
      <d:Size m:type="Edm.Int32">68202</d:Size>
      <d:Timestamp m:type="Edm.DateTime">0001-01-01T00:00:00</d:Timestamp>
    </m:properties>
  </content>
</entry>

Notice that the ParseTime and Timestamp values are equal to System.DateTimeOffset.MinValue, which, it turns out, is not a legal Azure table value. Wow, would it have helped me if the emulator horked on those values during development.

The fix was simply to make sure that neither System.DateTimeOffset.MinValue nor System.DateTime.MinValue ever got into an outbound table entity, which took me about five minutes to implement. Also, it turned out that even though my table entity inherited from TableServiceEntity, I still had to set the Timestamp property when using real Azure storage. (The emulator sets it for you.)

By this point it was 12:30 and I needed to get some sleep, however. So my plan to run an overnight test will have to wait until this evening at my hotel. Then I'll find the other bits of code that work fine against the emulator but, for reasons that pass understanding, the emulator gets completely wrong.

Sunday 10 February 2013 10:27:27 CST (UTC-06:00)  |  | Software | Cloud | Windows Azure#
Search
On this page....
When the Azure emulator is more forgiving than real life
Countdowns
The Daily Parker +3424d 10h 14m
Italy 59d 12h 45m
Parker's 9th birthday 77d 03h 00m
My next birthday 158d 07h 05m
Categories
Aviation (358) Baseball (110) Best Bars (10) Biking (44) Chicago (967) Cubs (198) Duke (133) Geography (352) Higher Ground (5) Jokes (284) Kitchen Sink (689) London (66) Parker (198) Daily (204) Photography (148) Politics (304) US (1126) World (274) Raleigh (21) Readings (8) Religion (67) San Francisco (93) Software (210) Blogs (82) Business (242) Cloud (90) Cool links (147) Security (103) Travel (255) Weather (744) Astronomy (93) Windows Azure (62) Work (95) Writing (14)
Links
Archive
<March 2015>
SunMonTueWedThuFriSat
22232425262728
1234567
891011121314
15161718192021
22232425262728
2930311234
Full archive
Blogroll
About
David Braverman and Parker
David Braverman is the Chief Technology Officer of Holden International in Chicago, and the creator of Weather Now. Parker is the most adorable dog on the planet, 80% of the time.
Legal
All content Copyright ©2015 David Braverman.
Creative Commons License
The Daily Parker by David Braverman is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License, excluding photographs, which may not be republished unless otherwise noted.
Admin Login
Sign In
Blog Stats
Total Posts: 4737
This Year: 123
This Month: 40
This Week: 2
Comments: 0