FOXTROT/ALFA: Microsoft Patches the Patch, Hacktoberfest Turns into Helltoberfest, Windows to Warn You If Your SSD is Going on the Fritz

Welcome back to the FOXTROT/ALFA tech and policy newsletter, which will, hopefully, be a daily affair again from now on! Our move to Düsseldorf went well, I’ve got internet connectivity and I’m working full-time again from this week. So here is issue 145 of the newsletter for Monday, 5 October 2020.

But before we get into what happened today, I want to recap some stories that happened while I was busy moving:

University Hospital in Düsseldorf Hit by Huge Ransomware Attack

Just as I was moving to Düsseldorf, the big university hospital in town was attacked with ransomware. I actually do know a few people who work there, too, and have some inside information on what went on.

The Aachener Zeitung newspaper carried a report from the German Press Association (DPA) that Doppelpaymer’s eponymous ransomware had been introduced to the University Hospital Düsseldorf’s network through a vulnerable Citrix product.

Worryingly, the ransomware’s loader had been lurking on the hospital’s network since December 2019, according to a detailed report handed to the provincial government of North Rhine-Westphalia and seen by the DPA. December 2019 was when a patch was issued by Citrix for CVE-2019-19781 – the same vuln exploited to hit the hospital, according to German tech news site Heise.

This is the Shitrix vulnerability, I first reported on in a January issue of the newsletter.

“(BSI) announced last week that the corresponding security gap in Citrix software had been known since the turn of the year. This was a loophole in the Citrix VPN software known as ‘Shitrix’ (CVE-2019-19781),” reported Heise, suggesting that once the loader had been planted on the network, the ransomware gang then opened a backdoor through a non-Citrix route before deploying the actual malware months later.

This story was widely publicised, because it was reported that a patient died because of the ransomware attack.

A woman in Germany died after a ransomware infection prevented her hospital from giving her emergency treatment. The unnamed patient died en route to a hospital in another city after she was unable to get treatment in Düsseldorf due to the malware affecting computer systems. A manslaughter investigation is now underway against the ransomware’s operators, who have yet to be identified.

In their very, very thin defense, the crooks behind the file-scrambling nasty turned over the decryption key to the cops when they were informed they had hit a hospital with their crimeware. Since then, the operators have gone dark. Chances are they are not in Germany and there’s not much hope for an arrest and extradition.

According to my sources, the reporting is essentially correct, but leaves out the fact that the woman’s chance of survival would have been extremely slim in either case. Basically, the University Hospital Düsseldorf (UKD) had shut its emergency room down when the attack was discovered, which is standard operating procedure for hospitals in Germany and actually happens quite a lot, including for many other reasons that have nothing to do with ransomware. This then means that ambulances are diverted to neighbouring hospitals – Düsseldorf, a city of about 600,000 inhabitants, has almost a dozen hospitals with emergency rooms. The woman in question died en route to a hospital that was further away than the UKD, but she would in all likelihood have died on an operating table in the UKD had that hospital’s emergency room been open.

That’s not to say that the UKD’s handling of this attack has been anything short of shambolic. Aside from the fact that the Shitrix vulnerability should have been patched in the first place, the attackers should never have been allowed to operate inside the network for months. And the UKD’s response once the attack was underway seemed to be incredibly slow and laboured. I’m told external emails are still being completely blocked in their networks at the moment – more than two weeks after the attack!

The Boeing 737 MAX Debacle Plods On

It seems like the proposed fix that should prevent more Boeing 737 MAXs to crash nose-first into the ground might not work.

The British Airline Pilots' Association (BALPA) has told American aviation regulators that the Boeing 737 Max needs better fixes for its infamous MCAS software, warning that a plane crash which killed 149 people could happen again. Airlines, in contrast, are broadly happy with proposed changes to the Boeing 737 Max, even as trade unions bellow at the US Federal Aviation Administration (FAA) that more needs to be done.

In public comments submitted to the FAA’s notice of proposed rulemaking (NPRM), BALPA warned that one of the proposed workarounds for a future MCAS failure could lead to a repeat of the crash of Ethiopian Airlines flight ET302.

The NPRM proposes various fixes to the 737 Max design, its software and procedures for pilots to follow in the event of a problem. One of those procedures includes disabling the airliner’s automatic trim system, operated by MCAS when the software kicks in, and having the two pilots use a manual backup trim wheel instead of the aircraft’s powerful electric motors.

BALPA said: “Requiring both crew members to turn the trim wheel simultaneously in a non-normal scenario is extremely undesirable and goes against all philosophies of having one pilot fly and one run the QRH [quick reference handbook: reading out the emergency checklist]. No flight control system should require both pilots to operate it at any stage, let alone in an emergency.” The trade union added: “It is felt that this should be reconsidered (particularly in light of the smaller diameter trim wheel as fitted to the MAX to enable the new larger screens to fit, and as per the scenario observed in the Ethiopian Airlines accident).

ET302 crashed after its pilots, who were fully aware of MCAS after the earlier crash of Lion Air flight 610 (the first 737 Max crash), tried without success to override the flawed software system. MCAS works by automatically trimming the 737 Max’s nose downwards if it senses that the aircraft is about to stall, a dangerous condition that normally comes about when the nose is pointing too high and the speed is too low. In ET302’s case that MCAS activation was false, however.

Its pilots disabled electric trim motors that had been activated by MCAS and, crash investigators believed, tried to use the manual trim wheel in the cockpit to physically undo what the software had done – following Boeing procedures published after the Lion Air crash. Thanks to the aircraft’s excessive speed, built up as MCAS forced its nose to point downwards at the ground, the pilots were unsuccessful. Aerodynamic forces on the control surfaces made it impossible for them to rotate the trim wheel and point the airliner’s nose back at the sky.

Sounds to me like Boeing is trying to save money again. They seem to be avoiding a necessary redesign of the trim wheels.

And that isn’t the only problem plaguing the 737.

Boeing’s 737 is at risk of an engine failure that could potentially leave the jet powerless after takeoff, the FAA has warned airlines, after multiple incidents were observed as previously grounded fleets return to active service. The US Federal Aviation Administration says it has four recent reports of 737-series jets experiencing a stuck valve that, if unchecked, could lead to disastrous results. The Emergency Airworthiness Directive (AD) requires all owners and operatives of Boeing 737-300, -400, -500, -600, -700, -700C, -800, -900, and -900ER series planes to check the engine bleed air 5th stage check valve. If it’s faulty – like getting stuck in the open position, for instance – it must be replaced.

“This emergency AD was prompted by four recent reports of single-engine shutdowns due to engine bleed air 5th stage check valves being stuck open,” the FAA explains. “Corrosion of the engine bleed air 5th stage check valve internal parts during airplane storage may cause the valve to stick in the open position. If this valve opens normally at takeoff power, it may become stuck in the open position during flight and fail to close when power is reduced at top of descent, resulting in an unrecoverable compressor stall and the inability to restart the engine.”

While the four reported cases have involved a single engine shutting down and proving unable to restart, the concern is that valves on both 737 engines could be affected. Were that the case, the FAA warns, it could leave the jet without power while in the air.

“Corrosion of these valves on both engines could result in a dual-engine power loss without the ability to restart,” the FAA points out. “This condition, if not addressed, could result in compressor stalls and dual-engine power loss without the ability to restart, which could result in a forced off-airport landing.”

“Forced off-airport landing” is reassuring-pilot-speak for “crash”.

Eric Raymond Thinks Windows Will Become an Emulation Layer atop Linux

Open source pioneer Eric S. Raymond thinks Microsoft will stop development of the Windows kernel and Linux will thus finally win the desktop wars.

Raymond’s argument, posted to his blog late last week, kicked off with some frank admiration for Windows Subsystem For Linux, the tech that lets Linux binaries run under Windows. He noted that Microsoft is making kernel contributions just to improve WSL. Raymond is also an admirer of software called “Proton”, an emulation layer that allows Windows games distributed by Steam to run under Linux. His next item of note was Microsoft’s imminent release of its Edge browser for Linux.

That collection of ingredients, he argued, will collide with the fact that Azure is now Microsoft’s cash cow while the declining PC market means that over time Microsoft will be less inclined to invest in Windows 10. “Looked at from the point of view of cold-blooded profit maximization, this means continuing Windows development is a thing Microsoft would prefer not to be doing,” he wrote. “Instead, they’d do better putting more capital investment into Azure – which is widely rumored to be running more Linux instances than Windows these days.”

Raymond next imagined he was a Microsoft strategist seeking maximum future profits and came to the following conclusion: “Microsoft Windows becomes a Proton-like emulation layer over a Linux kernel, with the layer getting thinner over time as more of the support lands in the mainline kernel sources. The economic motive is that Microsoft sheds an ever-larger fraction of its development costs as less and less has to be done in-house. If you think this is fantasy, think again. The best evidence that it’s already the plan is that Microsoft has already ported Edge to run under Linux. There is only one way that makes any sense, and that is as a trial run for freeing the rest of the Windows utility suite from depending on any emulation layer.”

What’s not mentioned in this is that Proton is a Valve project and Gaben traditionally hasn’t always been a fan of Microsoft, the company he left to start his own. But, it’s based on Wine and thus open source. So Microsoft could use it anyway, I guess.

Over time, Raymond reckoned, Windows emulation would only be present to handle “games and other legacy third-party software”. And eventually Microsoft will get so focused on Azure, and so uninterested in spending money on Windows, that it will ditch even the Windows emulation layer.

Third-party software providers stop shipping Windows binaries in favor of ELF binaries with a pure Linux API …and Linux finally wins the desktop wars, not by displacing Windows but by co-opting it.”

Wouldn’t that be a delicious final irony after all the hate and underhanded tactics of the ’90s and early 2000’s?

And with that prelude, let’s look at what’s been happening in the tech news over the weekend and today:

SQL Server 2019: Microsoft Patches the Patch

Microsoft isn’t having any luck with SQL Server lately…

There was good news for administrators of Microsoft’s SQL Server 2019 last night as Cumulative Update 8 emerged, fixing the borkage of its predecessor. Things haven’t been going well for the SQL Server 2019 servicing model: Cumulative Update 2 left the SQL Agent a bit unhappy (resulting in the company advising users to skip the patch if they were using the afflicted component). And most recently Cumulative Update 7 suffered from a “known issue” with database snapshots, which also affected that little-used command DBCC CHECKDB.

Cumulative Update 8 arrived with little fanfare and, like its predecessor, applies to both the Linux and Windows incarnations of Microsoft’s database engine. Those who applied the previous borked update and have hit the server with the CREATE DATABASE ... AS SNAPSHOT OF syntax will need to recreate that snapshot before applying number 8.

Microsoft is naturally keen that administrators slap on its sticking plasters as soon possible, declaring that each contains all the fixes (but hopefully not the cock-ups) of those that have gone before and all “are certified to the same levels as Service Packs, and should be installed at the same level of confidence”. n the case of SQL Server 2019’s Cumulative Updates, confidence levels may not be particularly high at this stage since two of the eight were subsequently subject to hurried “for the love of God, don’t install this one!” warnings from the Windows giant.

Windows to Proactively Warn You Before Your SSD Dies

Well, this could be a handy feature:

A new Windows Insider build – 20226 – contains a feature designed to spot impending failures of NVMe SSDs that gives users the opportunity to get their data off the things before they curl up and die.

Microsoft’s creatively named “October 2020 Update” is due to hit shortly. As it should be little more than a jumped-up cumulative update of its predecessor, one can but hope that there won’t be a repeat of its predecessor’s Surface-based teething troubles.

This would be especially handy because of that SSD wrecking bug in Windows 10.

Spotify Lets You Search by Lyrics Now

Speaking of handy features…

Spotify has rolled out a useful new feature today for iOS and Android that allows users to search for songs by its lyrics.

Hacktoberfest? More Like Helltoberfest

Well that certainly didn’t go according to plan…

Hosting biz Digital Ocean kicked off its seventh Hacktoberfest on Thursday – and managed to seriously annoy the very developers the event aims to celebrate. Launched in 2014, Hacktoberfest was founded to inspire people to get involved with the development of open-source software. It attempts to do so by encouraging programmers to submit quality pull requests to open-source repositories on GitHub with the promise of a free t-shirt in return.

The event has proven to be extremely popular though it hasn’t done anything for the quality of the contributions. Between the lure of swag and the reputational reward one gets from social interaction metrics on GitHub for any kind of activity, there’s not much that ensures contributions are meaningful. The result has been a deluge of spam that burdens already put-upon developers who maintain projects, often without much in the way of compensation or recognition.

In a blog post on Wednesday, Domenic Denicola, a senior software engineer at Google who contributes to various open source projects, lambasted Hacktoberfest for promoting what he describes as “a corporate-sponsored distributed denial of service attack against the open source maintainer community.” Denicola said that in one day, before the event had even officially begun, he and his fellow maintainers of the Web Hypertext Application Technology Working Group (WHATWG) HTML repository had closed 11 spam pull requests. “Each of these generates notifications, often email, to the 485 watchers of the repository,” he explained. “And each of them requires maintainer time to visit the pull request page, evaluate its spamminess, close it, tag it as spam, lock the thread to prevent further spam comments, and then report the spammer to GitHub in the hopes of stopping their time-wasting rampage.”

Drew DeVault, a developer who maintains various open-source projects, said as much in a post on Thursday: “As I write this, a Digital Ocean-sponsored and GitHub-enabled Distributed Denial of Service (DDoS) attack is ongoing, wasting the time of thousands of free software maintainers with an onslaught of meaningless spam.”

The discontent among developers has spurred the creation of a Twitter account with an indelicate name to track all the useless pull requests foisted upon open-source maintainers. The account contains a link that suggests the extent of the spamming: it lists over 300,000 issues created with the keywords “improve docs” – pull requests to add text to repo documentation are the easiest to make because no actual functioning code needs to be included.

“As of 2pm PST on October 1, at least four per cent of pull requests from Hacktoberfest participants have been marked ‘invalid’ or ‘spam’,” Digital Ocean said in an update on Thursday. “We’ve traced the majority of this year’s spammy contributions back to a participant with a large online audience who openly encouraged their community to take part in spammy activities, including ideas on how to game the system. However, we know the spam issues go beyond this one example.”

As a result, it’s going to help maintainers opt out of Hacktoberfest, set up a system to ban persistent offenders, and double the amount of consideration time for maintainers to 14 days.

Ouch.

Also Noteworthy

Other stories I’ve been reading today:

This is an archived issue of my daily newsletter FOXTROT/ALFA. You can find more information about it, including how to subscribe via email, on this page.