I Was Breached. It Was My Vendor’s Fault. What Now?
Leaders and IT departments worldwide are still working to recover from arguably one of the most prolific cyberattacks ever. If you use Microsoft Exchange Server, you were undoubtedly impacted by the hack, and if you were, you were not alone. The Wall Street Journal reported that the number of customers affected by the attack could reach upwards of 250,000.
Breaking Down the Exchange Attack
In early March, Microsoft disclosed that Chinese state-sponsored hackers, a group known as Hafnium, had gained access to organizations’ email accounts through vulnerabilities in its Exchange Server email software. Microsoft identified four vulnerabilities and issued emergency patches for the vulnerabilities for systems going back to 2010. That means flaws were present in Microsoft code for over a decade.
Microsoft was first alerted to the vulnerabilities by security company Volexity, who found the attackers had been exploiting the flaws since January. The attack does not impact Microsoft’s cloud-based email and calendar service, only its on-prem offering.
On March 5th, cybersecurity journalist Brian Krebs reported that at least 30,000 organizations across the United States were breached. This included small businesses, towns, cities, and local governments. IT teams continue to patch the vulnerabilities across the country, but as of March 14th, at least 80,000 vulnerable Exchange servers were still exposed on the internet, according to reports.
How the Attack Works
The attack works like this:
- Attackers are able to gain access to an Exchange Server either with stolen passwords or exploiting the vulnerabilities to disguise themselves as someone with legitimate access.
- Attackers create a web shell to control the compromised server remotely. A web shell is able to be uploaded to a web server, allowing remote access to the server.
- Attackers use this remote access to steal data from the organization, including email data.
Oh, But it Get’s Worse….
The flaws in the Exchange server are considered zero-days. However, Microsoft sent a proof-of-concept (PoC) code privately to members of its Active Protections Program. This code was leaked, either purposefully or accidentally, according to The Wall Street Journal, who reported that Microsoft had launched an investigation into the leak. Either way, a PoC made exploiting the vulnerabilities substantially easier and opened up the Exchange vulnerabilities, like floodgates, to a wide swath of the internet’s darker denizens. Other nation-states and cybercriminal gangs joined in on the fun. On top of that, a ransomware strain dubbed DearCry began capitalizing on the Exchange vulnerabilities.
The breach has forced those who use Microsoft’s on-prem email offering to scramble to migrate their email to the cloud. It also forced leaders to ask, did bad guys get in, and if they did, what did they get their hands on?
Patching the vulnerabilities was a critical step, but it didn’t necessarily eject criminals from internal networks. In some cases, organizations had to revert to backups and rebuild. This is a reason why it is so important to have backups. These emergency situations can arise, and you have to pull the plug immediately.
But First There Was SolarWinds
The Microsoft Exchange server attack came on the heels of another extraordinary cyber incident. This one occurred late last year and impacted 425 of the US Fortune 500, including telecommunication companies, accounting firms, and all branches of the US military. It began when cybersecurity company FireEye announced they had been a victim of a cyber breach attributed to Russian state-sponsored actors. Further investigation revealed that it was not FireEye that was the point of attack. Instead, it was SolarWinds, a widely-used IT management firm. Attackers had compromised a SolarWinds software update, so when SolarWinds customers applied the update, their systems became infected.
The incident was massive enough for Congress to call forth executives from SolarWinds and FireEye as well as Microsoft, who was impacted by the hack, for a good old fashion tongue lashing in front of the cameras. While the executives, essentially, pointed fingers at each other, the rest of us were left to wonder how we can protect ourselves from attacks we can’t even anticipate nor prevent?
What Can We Do?
When you think of third-party vendor attacks, the classic example is Target, which in 2013 has over 40 million credit cards stolen from its point-of-sale systems. The attackers were able to exploit Target’s HVAC vendor to gain access to Target’s network. This is the example that is often cited when we discuss the need to “watch your six” when it comes to giving vendor access to your networks. This was an HVAC vendor, and Target was roundly criticized for its data segmentation practices.
An HVAC vendor should not have access to any confidential, sensitive, or customer information. However, a technology vendor is a different story, especially one who helps administer your email (in the case of Microsoft) or oversees your network management (in the case of SolarWinds). These types of vendors need the highest level of access and access to essentially everything. There has to be a level of trust there that, almost to an extreme degree.
Today, it is impossible to do business…to even function without these vendors. We need them. But this reliance comes at a cost. First, we are subject to the extreme cases listed above. We will likely continue to see more of them (as cybercriminals understand how the exploitation of a technology vendor can reap enormous booty). Leaders are also limited in what they can do to protect themselves when a critical technology vendor gets hijacked.
But there are some things, as a leader, you can do.
Keep up to Date: One of the biggest things you can do is make sure you are keeping up with what is occurring when it comes to cybersecurity. You should be making cybersecurity a part of your daily feed, so you understand how these types of attacks occur and what you can do to limit the damage when they do occur. There are plenty of great newsletters and threat reports from industry insiders. One I would recommend is The CyberWire (https://thecyberwire.com/). You can sign up for their free daily newsletter. They also release a free daily podcast that runs around 20 minutes. It is a great way to stay on top of the latest in cybersecurity while upping your cybersecurity IQ.
This will also help in mitigation. In the case of the Exchange breach, it was an enormous challenge to understand how to respond because the attack was so unprecedented. The Cybersecurity & Infrastructure Security Agency (CISA) sent out an alert with some of the tactics, techniques, and procedures to help organizations examine their internal systems for evidence of maleficence. In critical moments, it is important to understand where to turn.
Keep an Expert on Speed Dial: In critical moments, it is important to understand where to turn. That is why it is helpful to have someone with expert knowledge to give you advice. Don’t seek that person or person(s) out when the emergency is taking place. Have that relationship already built, so it is there when the worst happens, and you need it most?
Keep Your Systems Up to Date: Putting aside the fact that the SolarWinds breach was caused by a malicious update, it is critical to push updates as quickly as possible when they are released. This was especially true of the Exchange server vulnerabilities, as we saw direct exploitation of vulnerabilities and Microsoft taking an unusual step of issuing emergency patches. Unpatched vulnerabilities are like a wide-open door to a bank vault. The criminals have a blueprint to get into your systems until the vulnerability has been patched. Make sure you are keeping your systems up to date.
Keep Your Backups Current: I discussed earlier the importance of backups. You should keep backups that are regularly updated and unconnected from your network. Then when a cyber incident occurs (such as ransomware), you will be able to quickly recover (and hopefully without having to pay any ransom).
Keep Your Logs in Order: Logs record events that occur within your network. With logs in place, you can detect unusual behavior, pinpoint anomalies, or forensically understand what occurred after an incident. If an attacker is downloading a large number of files, logs can alert you. If an employee is trying to access something he shouldn’t, logs can help catch him. If there is an unusual login attempt from southeast Asia, you may see it in your logs.
Keep an Event Playbook on Hand: Digital Event Response Playbooks (DERPs) are useful when an emergency digital event occurs. What these playbooks do is ensure the entire organization, every department, and every key stakeholder understands what they are responsible for during an event. This helps everyone to be on the same page or rowing in the same direction. It also can ensure that as little time is wasted as possible during the critical first hours of a digital event. Creating a playbook specifically for a cloud vendor exploitation should be on your to-do list if you haven’t already. Reach out to us because we specifically create this type of playbook for clients. We also suggest holding trainings for the playbook as well as tabletop exercises. In these exercises, key stakeholders are able to walk through a real-world scenario using the playbook. This helps the stakeholders understand their role during a digital event. It also aids the playbook owner in understanding how the playbook could be refined or improved.
When technology vendors inadvertently open holes in your network, you may feel helpless. It can feel like there is nothing you, as a leader, can do. But there are some things you can do to help your organization manage a difficult situation. A little extra preparation today can help in these extreme situations.