The Daily Parker

Politics, Weather, Photography, and the Dog

Backlog

I just started Sprint 52 in my day job, after working right up to the last possible minute yesterday to (unsuccessfully) finish one more story before ending Sprint 51. Then I went to a 3-hour movie that you absolutely must see.

Consequently a few things have backed up over at Inner Drive Technology World Headquarters.

Before I get into that, take a look at this:

That 17.1°C reading at IDTWHQ comes in a shade lower than the official reading at O'Hare of 17.8°, which ties the record high maximum set in 1971. The forecast says it'll hang out here for a few hours before gale-force winds drive the temperature down to more seasonal levels overnight. I've even opened a few windows.

So what else is new?

So what really is new?

But Sprint 52 at my office, that's incredibly new, and I must go back to it.

Overdue shifting of externalities in the UK

Fed up with manufacturers releasing Internet-connected products for the home with inadequate security that puts everyone in the world at risk, the UK has finally cracked down:

Default passwords for internet-connected devices will be banned, and firms which do not comply will face huge fines.

The Product Security and Telecommunications Infrastructure Bill lays out three new rules:

  • easy-to-guess default passwords preloaded on devices are banned. All products now need unique passwords that cannot be reset to factory default
  • customers must be told when they buy a device the minimum time it will receive vital security updates and patches. If a product doesn't get either, that must also be disclosed
  • security researchers will be given a public point of contact to point out flaws and bugs

The new regime will be overseen by a regulator, which will be appointed once the bill comes into force. It will have the power to fine companies up to £10m [$1.3m] or 4% of their global turnover, as well as up to £20,000 [$26,700] a day for ongoing contraventions.

About bloody time, I say. Yes, people should know better than to connect open Internet ports to their home networks, but most people in the world do not understand what any of that means. We don't make people mix their gasoline with air when driving anymore for the same reasons.

Fun start to the day

My 8am meeting with colleagues in London had to wait until 9:30 because Comcast screwed the pooch this morning:

Reports indicate the system was down, or at least unsteady, in areas stretching from Chicago to Philadelphia, New Jersey, and South Carolina. Looking at DownDetector, issues had been reported earlier in the Bay Area, but it’s unclear if those are connected to the problems people saw this morning.

Comcast has released a statement regarding the outage. According to a spokesperson, “Earlier, some customers experienced intermittent service disruptions as a result of a network issue. We have addressed the issue and service is now restoring for impacted customers, as we continue to investigate the root cause. We apologize to those who were affected.” It appears that most of the people who reported problems have confirmed they’re back online. There’s still no word on exactly what caused the problem or how many people were impacted at its peak.

In Chicago, the outage affected thousands of people from about 7:30 to 9, by which time I'd already relocated to my company's Loop office.

Oh, and on the day before a trip, my bank called to let me know their fraud department killed my primary credit card. They hope the new one arrives before I leave for the airport.

Yay.

Stupid request limits

I had to pause the really tricky refactoring I worked on yesterday because we discovered a new performance issue that obscured an old throttling issue. It took me most of the morning to find the performance bottleneck, but after removing it a process went from 270 seconds to 80. Then I started looking into getting the 80 down to, say, 0.8, and discovered that because we're using an API limit with a request limit (180 requests in 15 minutes), I put in a 5-second delay between requests.

Sigh.

So now I've got all this to read...someday:

Finally, the economics of workers vs employers has taken an odd turn as job applicants have started simply ghosting interviewers. But, as Slate says, "employers have been doing this to workers for years, and their hand-wringing didn’t start until the tables were turned."

How is it 9pm already?

Quick hit list of stuff I didn't find time to read:

Finally, Alexandra Petri guesses about the books that Republican candidate for Virginia Governor Glenn Youngkin might put on your kid's AP curriculum.

Happy Mason-Dixon Day

On this day in 1767, Charles Mason and Jeremiah Dixon completed their survey of the disputed Maryland-Pennsylvania border, which became even more contentious in 1780 when Pennsylvania aboolished slavery. A group of surveyors started re-surveying the border in 2019; I can't find out whether they finished.

Meanwhile, 255 years later, politics is still mostly local:

Finally, Chicago has perfectly clear skies for only the third time this month after yesterday and the 4th, getting only 39% of possible sunshine for almost the past three weeks.

How Facebook went down today

Cloudflare explains:

BGP stands for Border Gateway Protocol. It's a mechanism to exchange routing information between autonomous systems (AS) on the Internet. The big routers that make the Internet work have huge, constantly updated lists of the possible routes that can be used to deliver every network packet to their final destinations. Without BGP, the Internet routers wouldn't know what to do, and the Internet wouldn't work.

The Internet is literally a network of networks, and it’s bound together by BGP. BGP allows one network (say Facebook) to advertise its presence to other networks that form the Internet. As we write Facebook is not advertising its presence, ISPs and other networks can’t find Facebook’s network and so it is unavailable.

The individual networks each have an ASN: an Autonomous System Number. An Autonomous System (AS) is an individual network with a unified internal routing policy. An AS can originate prefixes (say that they control a group of IP addresses), as well as transit prefixes (say they know how to reach specific groups of IP addresses).

At 1658 UTC we noticed that Facebook had stopped announcing the routes to their DNS prefixes.

We keep track of all the BGP updates and announcements we see in our global network. At our scale, the data we collect gives us a view of how the Internet is connected and where the traffic is meant to flow from and to everywhere on the planet.

A BGP UPDATE message informs a router of any changes you’ve made to a prefix advertisement or entirely withdraws the prefix. We can clearly see this in the number of updates we received from Facebook when checking our time-series BGP database. Normally this chart is fairly quiet: Facebook doesn’t make a lot of changes to its network minute to minute.

But at around 15:40 UTC we saw a peak of routing changes from Facebook. That’s when the trouble began.

So, someone at Facebook may have applied a router update incorrectly. And as of now, they've corrected the problem.

First Monday of October

The United States Supreme Court began their term earlier today, in person for the first time since March 2020. Justice Brett Kavanagh (R) did not attend owing to his positive Covid-19 test last week.

In other news:

So how did facebook.com disappear from root DNS, the day after 60 Minutes aired a segment on Haugen?

Monday lunchtime reading

Just a couple today, but they seem interesting:

And wow, did the Chicago Bears have a bad game yesterday.

Late morning things of interest

So these things happened:

And finally, break out the Glühwein: Chicago's Christkindlmarket will return to Daley Plaza and Wrigleyville this winter.