Business IT, Security

You’ve probably heard of Meltdown and Spectre by now – reports have been circulating across the Internet and even in mainstream news media since the beginning of January. These pose a catastrophic risk to businesses everywhere, but the cause may turn out to be more bureaucratic than technical.

So just how bad is this? Surely it’s being blown out of all proportion?

Unfortunately, not really. If anything there’s far less attention being paid than the issue actually warrants. Some security experts have even been downplaying the risks because these vulnerabilities don’t allow remote code execution – they need to run within a privileged user session, which means an authorised user must run the malware that’s exploiting the vulnerability, and for that to happen the system must already be compromised. Meltdown and Spectre are not magic keys that unlock PCs to access and control from afar, unlike many recent Zero-Day vulnerabilities such as the ones the NSA were exploiting with their hacking toolkits. However, those traditional zero-day vulnerabilities exploited software flaws, which, as damaging as they were, are easily fixed with OS patches – hardware flaws are rare and are almost never exploited even when they are discovered, because they’re often very difficult to implement and only work in limited scenarios, which is fortunate because they’re much harder to patch.

Meltdown can be mitigated at the software/application level, and has already been addressed by Operating System (OS) vendors and browser publishers, so it has essentially been solved. But Spectre is different, even compared with most hardware vulnerabilities, because it’s a flaw of unprecedented magnitude due to the level of access it has within a CPU. While Intel and AMD have released their microcode updates, for billions of systems worldwide we’re still waiting on patches from hardware vendors before we can protect systems, and even once they’ve been released patching is going to be labour intensive and require skills that the vast majority of consumers are not going to be familiar with. Because of the simple fact that automated patching is going to be impossible for huge amounts of consumer hardware, Spectre is extremely dangerous.

But back to that earlier point about not being a remote code vulnerability. If it needs to be run locally on a PC, then it can’t penetrate your network to get on a PC in the first place, can it? To answer this, I’d like to remind readers about another type of malware – ransomware. Ransomware doesn’t take advantage of any technical vulnerability – it can’t attack your PCs or your network, and on its own it can’t even spread. It’s just a program that runs on a computer when a user executes it. And then it encrypts anything and everything it can get access to, any file the user themselves can write to, throughout your network. Ransomware has become a massive problem because people click things and run things that they didn’t check thoroughly first – Ransomware exploits human vulnerabilities, not technical ones, and it works.

Spectre exploits can be executed in a similar way to Ransomware, except instead of locking down files and trying to get you to pay a ransom, they could allow an attacker to take over the computer entirely, but by doing so in the background they then get access to network resources the user has, potentially allowing the malware to spread. It could piggyback itself on legitimate communication between people and machines and use that established trust between humans that are familiar to each other to trick other users to run it on their devices, continuing to spread through sophisticated human manipulation rather than technical penetration. Imagine if every file any of your trusted contacts emailed you could potentially install malware that could then take over your PC within minutes, without you having any ability to detect it – all you did wrong was open a legitimate attachment that actually opened and worked as expected. If a SysAdmin logs into a compromised machine using an account with elevated privileges to perform diagnostics, that could not only hand them full control of the whole network, but all connected backups, too. Most of the potential damage of Ransomware was mitigated in large organisations by their backup and disaster recovery systems, but if you’re successfully attacked by a Spectre toolkit and you don’t have cold backup archives, you could also lose every backup that’s subsequently connected to your network.

Cybercrime is a trillion dollar industry (due to hit $6 trillion by 2021) – it is a colossal business enterprise and they employ some of the finest minds on the planet (because you’ll make a lot more money writing malware than you will trying to protect against it); they will find every possible way to extract maximum value from this and they will do their level best to attack every accessible target until they control it. To say nothing of the effort state actors (foreign governments) are pumping into their CyberWarfare divisions.

What About Antivirus?

Oh, but your antivirus/Endpoint Protection suite is meant to keep you safe? Remember, these are hardware vulnerabilities, so exploiting them will require software to operate in a way that is not immediately obvious to antivirus software – it isn’t going to act like a virus at all unless it’s bundled with other tools. Sure, when software is discovered that is exploiting these vulnerabilities security vendors will be able to create signatures for them. But heuristic analysis is unlikely to detect it, so security suites are always going to play catch up. I direct your attention again to Ransomware, as the vast majority of successful attacks were performed on systems running active security suites that were unable to detect it. Quite a few security experts have warned that antivirus software may remain totally unaware of a Spectre attack while it’s going on. That may be less accurate as time passes as security companies become more familiar with Spectre and write more sophisticated software to detect it, but in that time hacking toolkits exploiting spectre will become more sophisticated too. Yes, it’s an arms race.

Ok it Sounds Bad – But the Industry is Fixing it, Right?

On the third of April 2018 Intel announced that they’d released microcode patches for most of the CPUs they’ve produced over the past ten years or so. A week later AMD also announced the release of microcode patches for their ecosystem that will be available through hardware manufacturers. Both processor manufacturers have announced a program with Microsoft to push out updates that help mitigate Spectre, but Microsoft are taking a softly-softly approach to pushing out these microcode updates within Windows – so far only CPUs from the last couple of architecture generations have been included. It is hoped that Microsoft will gradually add additional CPU models over time, but there’s no way to know at this stage how many they will eventually support. Even though the microcode patches have been released to system board/PC/Server manufactures from Intel, they need to be tested, qualified, packaged up and released in the form of firmware updates for every single PC, server, mainboard and affected device – a different, unique firmware for every single model.

So how does that affect you? If you have a PC or a laptop, it will remain vulnerable until you’ve applied a firmware patch released by the manufacturer or your CPU model has been added to Microsoft’s list, you’re running the latest version of Windows and you’ve applied all current updates. For example, if that’s a Dell machine, you’ll have to check the Dell support page for that specific device. If the patch isn’t available yet you’ll have to keep checking back. Hopefully, eventually, Dell will release a firmware patch that you can download and apply to your machine, and then you’ll be safe. You can go here to find a reporting tool that will run a check on your device and tell you if it’s still vulnerable.

If you have a business with lots of PCs and devices, yes you’ll have to check the support pages of every one of those, then download and apply the firmware patches once they become available. Yes, that’s going to take quite a lot of time and effort, but that’s not even the biggest problem.

Greatest Risk – Will Hardware Vendors Actually Patch Legacy Systems, or use this to Drive New Sales?

Unfortunately it is likely that the industry thinks it is reasonable to not provide patches for older systems, as that will help stimulate new purchasing – customers get shiny new faster equipment, the IT industry gets fat from a new wave of global panic buying, the economy is stimulated and everyone’s happy, or so the thinking goes. However, many businesses won’t be able to afford to patch large numbers of machines all at once. Some won’t trust the analysis, some will think they can roll the dice and win against the house. (And that says nothing for cost-sensitive consumers who haven’t had a pay increase in over a decade and simply can’t afford a replacement device.)

Unfortunately all those businesses, regardless of motive, will be compromised, they will lose customer data, they will lose control of their own systems, they won’t be able to afford the clean-up bill and they certainly won’t be able to afford the law suits from their customers or the suppliers they can’t afford to pay – they will go into liquidation and if it can be shown that these decisions were made knowingly, some people will go to jail.

This situation isn’t helped at all by the extraordinary apathy that has been demonstrated by many analysts and commentators – most articles on this subject have offhandedly dismissed the overwhelming difficulty to consumers and SMBs of applying firmware patches, while they haven’t even considered the possibility that device manufacturers and mainboard vendors might not bother providing firmware patches for out-of-warranty systems. Let me put it into perspective for you – if manufacturers decide to draw a line in the sand and only patch hardware manufactured in the past three years, that will leave between two and three billion PCs and laptops unpatched. Even if they go back five years, that’s still over 1.5 billion unpatched (and unprotectable) devices.

{Update 9th May: We’ve been in touch with some of the major hardware manufacturers and it appears all devices with a non-EFI Award BIOS cannot be updated with Intel’s microcode. We’re following up to see if there are technical workarounds for this, but from the information we have at hand it doesn’t look good for older systems.

As mentioned above, Microsoft are pushing out microcode updates with the most recent versions of Windows 10 (go here to view the full list), but so far it’s for a limited range of CPUs only produced in the last couple of years. Unfortunately we have no way to know where Microsoft will draw the line on older CPUs – if they discover any architectural limitations affecting stability of any devices as a result of the microcode during internal testing it is very unlikely they will continue with that series – they will do everything they can to avoid the PR disaster that would ensue from their patch causing previously operation PCs to start crashing.}

Why Wouldn’t Manufacturers Provide Patches?

Apart from the fact that manufacturers stop releasing updates for products outside their warranty period, last year we saw a kind of trial run of this situation – Intel’s AMT flaw. AMT is a component of vPro, Intel’s business class asset management hardware that’s built into all vPro devices. AMT allows remote analysis, monitoring and even remote control of the device, and a flaw was discovered that could allow anyone full control over a device by completely bypassing the password. This was a massive deal for enterprises, but most consumers didn’t even hear about it.

Like Intel’s just done with Spectre, last year they developed and released a firmware patch to fix the problem with AMT, but it was up to device manufacturers to push that out via firmware updates to PCs, laptops and motherboards. At Xion Technology we experienced varying levels of support from manufacturers of systems we support, but one particular vendor was notable. This manufacturer didn’t include motherboard models in their ‘supported products’ document, despite having a number of affected models available. For the model that affected us, it took them three months to release a firmware patch after Intel’s announcement, while some manufacturers like Dell had patches available in just two weeks. But applying the firmware didn’t patch the vulnerability – it took phone calls to a major distributor to put us in touch with the head of RA at their Australian head office before we discovered that the actual AMT firmware was a completely separate file that they didn’t publish publicly at all – the only way we were able to obtain it and patch the affected systems was via email from the head of RA. Using normal, published support channels was useless – normal support had absolutely no idea what AMT even was. Without this backchannel those systems would still be unpatched today.

As mentioned earlier, the IT Industry could easily exploit ‘opportunities’ like this to make money – manufacturers are already pushing this as an ‘upgrade opportunity’, with the major processor vendors jostling to come out of this looking good by ratcheting up the rhetoric. But the reality is that no one looks good – all the hardware vendors completely missed this for well over a decade. Trying to make a buck out of this is a cynical exercise in corporate profiteering at the expense of customers. What hardware vendors should be doing is offering discounts and credits for customers forced to upgrade due to unpatchable systems or significant performance impacts – they should be bending over backwards to make this right, even if it means a cycle of depressed profits – sell it to your shareholders as buying good will (they’ve sure as heck got a lot to make up for right now). What they may do, sadly, could be cutting their customers loose, because it makes the short-term balance sheet look better.

{Update 9th May: As mentioned in the section above, for older systems running a pre-EFI BIOS (such as Award by Phoenix Technologies) it may not be possible to write the microcode to the CPUs. In that case the reason the device or motherboard manufacturer isn’t providing a firmware patch may simply be because they can’t.}

Can We Just Replace Affected Devices?

What about simply replacing legacy systems? Unfortunately, even brand new systems are shipping without these firmware patches today, but within a month or so you’ll be able to buy computers off-the-shelf that are already fully patched. Unfortunately, patched current generation hardware isn’t natively immune to the flaw – it has the same workaround that has been implemented in older systems, and this comes with a performance penalty. The extent of that penalty will depend on what you’re doing with the device – IO intensive tasks accessing high-speed storage like SSDs or Optane suffer the largest penalties, while tasks like gaming (which depend on the graphics card more than anything) and even fairly compute-intensive applications don’t see much impact at all.

Intel’s Canon Lake CPU refresh due towards the end of the year has supposedly been redesigned to address the speculative execution flaw that gives Spectre its name, but Intel haven’t had enough time to fully redesign Canon Lake and initial reports suggested it would take until their 9^th gen architecture (Ice Lake) before they had silicon that was designed from the ground up without these vulnerabilities. It will be at least 18 months before Ice Lake is released and we can start comparing apples with apples. Alternatively, AMD’s Ryzen platform may be worth investigating, as it is a compelling option to Intel’s Core architecture (and it supports ECC memory, which is a big plus).

Over the coming months you may find systems that remain unpatched by the manufacturer and it may be prudent to start replacing older systems at that point. But I wouldn’t rush out and start replacing your PCs today – you’re just replacing defective hardware with slightly newer defective hardware, at least for a while.

So What Can We Do?

To start with, you need a comprehensive list of all devices you have in use. Then use that list to find the Support page for that device and check if the manufacturer has released a firmware patch for it. If not, get in contact with the manufacturer and ask them when they expect to have a new firmware available to address this issue. Repeat these steps for every PC, Server and laptop that you own. Smartphones and tablets were vulnerable to Spectre, yes, but any device that can receive a security patch pushed out to it will have one (if they’ve haven’t already), and due to the way smartphone app stores operate it is highly unlikely a malicious app will be able to affect phones in any great number (because only those who install apps from unverified sources will even be able to get such software on their phones).

As this plays out over the next month or so, we need to watch manufacturers closely for announcements of which devices they have patched, and which they are intending to cut loose.

In the meantime you need to proactively educate staff – you need to ensure staff are well informed about the use of work computers, only using known, legitimate websites (preferably using stored Bookmarks rather than typing or clicking links redirecting from other sites). They need to think carefully about email attachments and file downloads – did it come from a legitimate source? Were they expecting it? Was the language used in the email natural and normal for that person, or did it seem unusual – could it be someone impersonating them? Does the sender’s address look correct, or is it from a different domain but hiding behind a recognised ‘reply to’ address? If in doubt, NEVER CLICK – step away from the keyboard and mouse and ask for assistance from senior or technical staff.

Finally, a Message for the Industry

We have a simple message for the whole IT hardware industry – do the right thing. For every architecture that Intel and AMD have released a patch for, make sure every PC, laptop, server and mainboard you’ve produced has supporting BIOS/UEFI firmware updates to get the speculative execution mitigation microcode into systems. It’s still going to be an uphill battle getting end users to patch their systems anyway, but at least you will have done the right thing. Over the next few years this is going to evolve into a significant problem even if the whole industry does everything we can to resolve it – the last thing we need is unscrupulous manufacturers trying to cut costs and maximise profit by not even bothering to patch older hardware.

At some point, politicians and law makers somewhere are going to get tired of all the reports of successful hacks, people losing their savings, businesses going bankrupt and private data being stolen, and when they realise there is a scapegoat, that they can score political points by attacking the IT Industry and manufacturers, they won’t just encourage law suits, they will look to regulate the industry. Once it happens somewhere, it will be copied everywhere, as businesses and citizens demand action from politicians. Because it’s now too important – this isn’t just hobbyists, or small businesses, it isn’t even small industries, it is the global economy. It doesn’t really matter if your particular company only supplies components for a small fraction of the market, if you choose to cast your customers to the wind, you’re shirking your responsibility to them and risking the security of every bit of data that flows through that hardware. Governments are more than happy to make examples out of smaller players.

Frankly if the IT Industry can’t self-regulate and ensure every partner in the supply chain does everything it can to adhere to a security-first approach, it needs to be legally regulated.