The Daily Parker

Politics, Weather, Photography, and the Dog

The tragedy of Agile

Uncle Bob riffs on Martin Fowler's speech at Agile Australia this week. He is saddened:

It was programmers who started the Agile movement as a way to say: “Hey look! Teams matter. Code should be clean. We want to collaborate with the customer. And we want to deliver early and often.”

The Agile movement was started by programmers, and software professionals, who held the ideals of Craftsmanship dear. But then the project managers rushed in and said: “Wow! Agile is a cool new variation on how to manage projects.”

There’s an old song, by Alan Sherman, called J. C. Cohen. It’s about a subway conductor who did such a great job at pushing people into the train cars, that he pushed the engineer out. This is what happened to the Agile movement. They pushed so many project managers in, they pushed the programmers out.

The programmers continued to pursue Agile as it was originally conceived. Read the opening line of the Agile Manifesto: “We are uncovering better ways of developing software by doing it and helping others do it.” It is Software Crafts-men and -women who are continuing that work. It’s not the project managers in the Agile movement. They’re off pursuing something else?

He has hit on the sadness all us old craftsmen feel when we encounter Management.

You're not wrong, voice-command assistant UIs are horrible

The Nielsen-Norman Group has released recent research on user interactions with intelligent assistants like Alexa and Google Home. The results are not great:

Usability testing finds that both voice-only and screen-based intelligent assistants work well only for very limited, simple queries that have fairly simple, short answers. Users have difficulty with anything else.

Our user research found that current intelligent assistants fail on all 6 questions (5 technologies plus integration), resulting in an overall usability level that’s close to useless for even slightly complex interactions. For simple interactions, the devices do meet the bare minimum usability requirements. Even though it goes against the basic premise of human-centered design, users have to train themselves to understand when an intelligent assistant will be useful and when it’s better to avoid using it.

Our ideology has always been that computers should adapt to humans, not the other way around. The promise of AI is exactly one of high adaptability, but we didn’t see that that when observing actual use. In contrast, observing users struggle with the AI interfaces felt like a return to the dark ages of the 1970s: the need to memorize cryptic commands, oppressive modes, confusing content, inflexible interactions — basically an unpleasant user experience.

Are we being unreasonable? Isn’t it true that AI-based user interfaces have made huge progress in recent years? Yes, current AI products are better than many of the AI research systems of past decades. But the requirements for everyday use by average people are dramatically higher than the requirements for a graduate student demo. The demos we saw at academic conferences 20 years ago were impressive and held great promise for AI-based interactions. The current products are better, and yet don’t fulfill the promise.

We're not up to HAL or Her yet, in other words, but we're making progress.

The whole article is worth a read.

Cool new feature in Weather Now

I've finally gotten around to extending the historical weather feature in Weather Now. Now, you can get any archival report that the system has, back to 2013. (I have many more archival reports from before then but they're not online.)

For example, here's the last time I arrived in London, or the time I took an amazing photo in Hermosa Beach, Calif.

I don't know why it took me so long to code this feature. It only took about 4 hours, including testing. And it also led me to fix a bug that has been in the feature since 2008.

Morning links

I didn't have a chance to read these yesterday:

Now I'm off to work. The heat wave of the last few days has finally broken!

Lunchtime reading

Not all of this is as depressing as yesterday's batch:

I'm sure there will be more later.

List of 2018 A-to-Z topics

Blogging A to ZHere's the complete list of topics in the Daily Parker's 2018 Blogging A-to-Z challenge on the theme "Programming in C#":

Generally I posted all of them at noon UTC (7am Chicago time) on the proper day, except for the ones with stars. (April was a busy month.)

I hope you've enjoyed this series. I've already got topic ideas for next year. And next month the blog will hit two huge milestones, so stay tuned.

Z is for Zero

Blogging A to ZToday is the last day of the 2018 Blogging A-to-Z challenge. Today's topic: Nothing. Zero. Nada. Zilch. Null.

The concept of "zero" only made it into Western mathematics just a few centuries ago, and still has yet to make it into many developers' brains. The problem arises in particular when dealing with arrays, and unexpected nulls.

In C#, arrays are zero-based. An array's first element appears at position 0:

var things = new[] { 1, 2, 3, 4, 5 };
Console.WriteLine(things[1]);

// -> 2

This causes no end of headaches for new developers who expect that, because the array above has a length of 5, its last element is #5. But doing this:

Console.WriteLine(things[5]);

...throws an IndexOutOfRange exception.

You get a similar problem when you try to read a string, because if you recall, strings are basically just arrays of characters:

var word = "12345";
Console.WriteLine(word.Substring(4));

// 5

Console.WriteLine(word.Substring(5));

// IndexOutOfRange exception

The funny thing is, both the array things and the string word have a length of 5.

The other bugaboo is null. Null means nothing. It is the absence of anything. It equals nothing, not even itself (though this, alas, is not always true).

Reference types can be null, and value types cannot. That's because value types always have to have a value, while reference types can simply be a reference to nothing. That said, the Nullable<T> structure gives value types a way into the nulliverse that even comes with its own cool syntax:

int? q = null;
int r = 0;
Console.WriteLine(q ?? 0 + r);
// 0

(What I love about this "struct?" syntax is you can almost hear it in a Scooby Doo voice, can't you?)

Line 1 defines a nullable System.Int32 as null. Line 2 defines a bog-standard Int32 equal to zero. If you try to add them, you get a NullReference exception. So line 3 shows the coalescing operator that basically contracts both of these statements into a succinct little fragment:

// Long form:
int result;
if (q.HasValue)
{
	result = q.Value + r;
}
else
{
	result = 0 + r;
}

// Shorter form:
int result = (q.HasValue ? q.Value : 0) + r;

// Shortest form:
int result = q ?? 0 + r;

And so the Daily Parker concludes the 2018 Blogging A-to-Z challenge with an entire post about nothing. I hope you've enjoyed the posts this month. Later this morning, I'll post the complete list of topics as a permanent page. Let me know what you think in the comments. It's been a fun challenge.

Y is for Y2K (and other date/time problems)

Blogging A to ZI should have posted day 25 of the Blogging A-to-Z challenge. yesterday, but life happened, as it has a lot this month. I'm looking forward to June when I might not have the over-scheduling I've experienced since mid-March. We'll see.

So it's appropriate that today's topic involves one of the things most programmers get wrong: dates and times. And we can start 20 years ago when the world was young...

A serious problem loomed in the software world in the late 1990s: programmers, starting as far back as the 1950s, had used 2-digit fields to represent the year portion of dates. As I mentioned Friday, it's important to remember that memory, communications, and storage cost a lot more than programmer time until the last 15 years or so. A 2-digit year field makes a lot of sense in 1960, or even 1980, because it saves lots of money, and why on earth would people still use this software 20 or 30 years from now?

You can see (or remember) what happened: the year 2000. If today is 991231 and tomorrow is 000101, what does that do to your date math?

It turns out, not a lot, because programmers generally planned for it way more effectively than non-technical folks realized. On the night of 31 December 1999, I was in a data center at a brokerage in New York, not doing anything. Because we had fixed all the potential problems already.

But as I said, dates and times are hard. Start with times: 24 hours, 60 minutes, 60 seconds...that's not fun. And then there's the calendar: 12 months, 52 weeks, 365 (or 366) days...also not fun.

It becomes pretty obvious even to novice programmers who think about the problem that days are the best unit to represent time in most human-scale cases. (Scientists, however, prefer seconds.) I mentioned on day 8 that I used Julian day numbers very, very early in my programming life. Microsoft (and the .NET platform) also uses the day as the base unit for all of its date classes, and relegates the display of date information to a different set of classes.

I'm going to skip the DateTime structure because it's basically useless. It will give you no end of debugging problems with its asinine DateTime.Kind member. This past week I had to fix exactly this kind of thing at work.

Instead, use the DateTimeOffset structure. It represents an unambiguous point in time, with a double value for the date and a TimeSpan value for the offset from UTC. As Microsoft explains:

The DateTimeOffset structure includes a DateTime value, together with an Offset property that defines the difference between the current DateTimeOffset instance's date and time and Coordinated Universal Time (UTC). Because it exactly defines a date and time relative to UTC, the DateTimeOffset structure does not include a Kind member, as the DateTime structure does. It represents dates and times with values whose UTC ranges from 12:00:00 midnight, January 1, 0001 Anno Domini (Common Era), to 11:59:59 P.M., December 31, 9999 A.D. (C.E.).

The time component of a DateTimeOffset value is measured in 100-nanosecond units called ticks, and a particular date is the number of ticks since 12:00 midnight, January 1, 0001 A.D. (C.E.) in the GregorianCalendar calendar. A DateTimeOffset value is always expressed in the context of an explicit or default calendar. Ticks that are attributable to leap seconds are not included in the total number of ticks.

Yes. This is the way to do it. Except...well, you know what? Let's skip how the calendar has changed over time. (Short answer: the year 1 was not the year 1.)

In any event, DateTimeOffset gives you methods to calculate time and dates accurately across a 20,000-year range.

Which is to say nothing of time zones...

X is for XML vs. JSON

Blogging A to ZWelcome to the antepenultimate day (i.e., the 24th) of the Blogging A-to-Z challenge.

Today we'll look at how communicating between foreign systems has evolved over time, leaving us with two principal formats for information interchange: eXtensible Markup Language (XML) and JavaScript Object Notation (JSON).

Back in the day, even before I started writing software, computer systems talked to each other using specific protocols. Memory, tape (!) and other storage, and communications had significant costs per byte of data. Systems needed to squeeze out every bit in order to achieve acceptable performance and storage costs. (Just check out my computer history, and wrap your head around the 2400 bit-per-second modem that I used with my 4-megabyte 386 box, which I upgraded to 8 MB for $350 in 2018 dollars.)

So, if you wanted to talk to another system, you and the other programmers would work out a protocol that specified what each byte meant at each position. Then you'd send cryptic codes over the wire and hope the other machine understood you. Then you'd spend weeks debugging minor problems.

Fast forward to 1996, when storage and communications costs finally dropped below labor costs, and the W3C created XML. Now, instead of doing something like this:

METAR KORD 261951Z VRB06KT 10SM OVC250 18/M02 A2988

You could do something like this:

<?xml version="1.0" encoding="utf-8"?>
<weatherReport>
	<station name="Chicago O'Hare Field">KORD</station>
	<observationTime timeZone="America/Chicago" utc="2018-04-26T19:51+0000">2018-04-26 14:51</observationTime>
	<winds>
		<direction degrees="">Variable</direction>
		<speed units="Knots">6</speed>
	</winds>
	<visibility units="miles">10</visibility>
	<clouds>
		<layer units="feet" ceiling="true" condition="overcast">25000</layer>
	</clouds>
	<temperature units="Celsius">18</temperature>
	<dewpoint units="Celsius">-2</dewpoint>
	<altimeter units="inches Hg">29.88</altimeter>
</weatherReport>

The XML only takes up a few bytes (612 uncompressed, about 300 compressed), but humans can read it, and so can computers. You can even create and share an XML Schema Definition (XSD) describing what the XML document should contain. That way, both the sending and receiving systems can agree on the format, and change it as needed without a lot of reprogramming.

To display XML, you can use eXtensible Style Language (XSL), which applies CSS styles to your XML. (My Weather Now project uses this approach.)

Only a few weeks later, Douglas Crockford defined an even simpler standard: JSON. It removes the heavy structure from XML and presents data as a set of key-value pairs. Now our weather report can look like this:

{
  "weatherReport": {
    "station": {
      "name": "Chicago O'Hare Field",
      "icao code": "KORD"
    },
    "observationTime": {
      "timeZone": "America/Chicago",
      "utc": "2018-04-26T19:51+0000",
      "local": "2018-04-26 14:51 -05:00"
    },
    "winds": {
      "direction": { "text": "Variable" },
      "speed": {
        "units": "Knots",
        "value": "6"
      }
    },
    "visibility": {
      "units": "miles",
      "value": "10"
    },
    "clouds": {
      "layer": {
        "units": "feet",
        "ceiling": "true",
        "condition": "overcast",
        "value": "25000"
      }
    },
    "temperature": {
      "units": "Celsius",
      "value": "18"
    },
    "dewpoint": {
      "units": "Celsius",
      "value": "-2"
    },
    "altimeter": {
      "units": "inches Hg",
      "value": "29.88"
    }
  }
}

JSON is easier to read, and JavaScript (and JavaScript libraries like JQuery) can parse it natively. You can add or remove key-value pairs as needed, often without the receiving system complaining. There's even a JSON Schema project that promises to give you the security of XSD.

Which format should you use? It depends on how structured you need the data to be, and how easily you need to read it as a human.

More reading: