How a Manufacturing Firm Recovered from a Devastating Ransomware Attack

The infamous Ryuk ransomware slammed a small company that makes heavy-duty vehicle alternators for government and emergency fleet. Heres what happened.

The tiny IT team at C.E. Niehoff & Co. had been working for two weeks to run down and clean up a malware infection that had infiltrated its network after an employee clicked on a URL in a phishing email. Unbeknownst to the company as it scrambled to quell the attack, the malware, which was later identified as Trickbot, was quietly spreading among its endpoints and servers, gathering intel about the manufacturing firm and stealing credentials from the compromised machines.
It wasnt until the morning of Sunday, Oct. 14, when C.E. Niehoff IT manager Kelvin Larrue logged into the companys network from home, that it became clear to the company that the attack was something much more serious than a bot infection. A stunned Larrue could see that an intruder was running a PowerShell session on one of the companys servers, moving from server to server with stolen credentials and disabling security tools.
I could see what he was actually doing. I knew we were in real trouble and someone was in our system, Larrue recalls. They literally had the keys to the kingdom.
Larrue jumped into his car and drove the 20-minute route to the data center on the companys campus in Evanston, Ill., which houses its corporate headquarters and the manufacturing plant where it builds heavy-duty alternators for government and emergency vehicles. Racing to shut down the network in order to shut out the attacker, Larrue and his team pulled the plug in hopes of preventing the attacker from getting any deeper into the network, but it was too late.
By that time, the perpetrator had done extensive damage to our network, he says. The attacker had begun dropping ransomware: He had started routines to encrypt files on all of the servers and any workstations he happened to be on at that point, Larrue says.
What Larrue was witnessing firsthand, he later learned, was a Ryuk ransomware attack on his company. Ryuk is part of the recent generation of ransomware variants that is typically used for custom and targeted attacks on bigger and potentially more financially lucrative targets. According to Check Point Security, which has studied Ryuk and its attack methods, Ryuks authors built it with an encryption scheme that targets critical resources and assets in a victims network; for maximum impact, its payload is released manually by the attackers once they have the intel and stolen credentials they need.
When [Ryuk attackers] infect a new victim, they can stay for a while to observe the network ... and see if the infected machine or network is interesting, explains Itay Cohen, a security researcher with Check Point who tracks Ryuk. They do not automatically drop Ryuk; they drop it manually if they decide its a useful target. Thats a departure from earlier ransomware attack campaigns that were more random and automated, he says.
Ryuk has claimed several high-profile victims since the fall of 2018, including newspapers such as the Chicago Tribune and the Los Angeles Times; the city of Stuart, Fla.; and Onslow Water and Sewer Authority, which was hit with a ransomware attack in October 2018, around the same time frame as C.E. Niehoff.
Ryuk and other ransomware, such as GandCrab and LockerGoga, which crippled Norwegian aluminum manufacturer
Norsk Hydro
, are all about targeting what CrowdStrike calls
big game,
or large organizations theoretically able to pay a higher ransom than randomly infected consumers or small organizations.
Larrue says C.E. Niehoff believes the malicious URL in the phishing email that dropped Trickbot was the first phase of its attack and where the intel-gathering and credential-stealing occurred. In some Ryuk attacks on other victims, the gang has used Emotet as the bot and Trickbot as the intel- and credential-stealer in advance of dropping Ryuk and locking down the victims machines.
What was happening behind the scenes was that Trickbot got in and set up the whole command-and-control thing, and we later found out what was actually going on. They siphoned off credentials, set up the C2, and then we got hit with the big one, the Ryuk ransomware, Larrue explains.
While he and his team were chasing our tails trying to quell the infections spread, the attackers had set up a reverse-shell attack, he says, possibly exploiting an unpatched vulnerability in Java. The companys Vipre anti-malware tools didnt recognize or catch the variant.
With the stolen credentials, the Ryuk attackers then set up Remote Desktop Protocol (RDP) connections to the network and, via the PowerShell commands, set off the Ryuk ransomware payload, server by server, he says.
But what Larrue and his team didnt realize at the time was that unplugging machines from the network actually exacerbated the attack: The Ryuk attackers apparently had set the attack to corrupt the firmware of the infected machines if the ransomwares encryption process was disrupted. Larrue and his team of three IT staffers had not seen the ransom note warning them not to shut down or risk their systems getting corrupted when they frantically did the shutdown; they finally got a look at the message in the wake of the response.
They were expecting us to come in Monday morning [to the ransom message], he says. They didnt expect us on Sunday.
Unplugging the machines was a mistake on my part, Larrue adds. Part of the encryption scheme ... was if we did pull the plug, something would corrupt the firmware on all the servers, including the manufacturing firms email and ERP servers.
At that point it was totally lost. Even if we wanted to pay ransom, we couldnt, he recalls.
It turned out the ransom note had warned that only the attackers could help decrypt the files, and that resetting or shutting down systems could damage the files. It didnt include a ransom fee, but instead instructions on how to proceed in working with the attackers to get the files decrypted.
Ive had bad days in my life, but Ive never had one like that, Larrue says. I had the weight of the world bearing down on me.
Paper and Pen
C.E. Niehoff is a relatively small, privately held manufacturing firm, with 400 employees and a three-person IT department that also works on security issues. Its customers include the US military, which uses its industrial alternators for vehicles, for instance. One of the first worries in the wake of the attack was the loss of its ERP manufacturing server to the Ryuk attack.
The good news was the attackers hadnt stolen any customer or sensitive information, but the bad news was the manufacturing process had to rely on paper and existing orders to keep the shop floor open. We had enough paperwork to keep the manufacturing floor running on jobs already issued, Larrue says. The ERP system provides information to execute on the shop floor, but we can still produce without it. Production didnt come to a grinding halt.
But we couldnt see too far into the future until the ERP system was back online, he recalls.
By some stroke of luck, the companys human resources and payroll server wasnt infected with ransomware. Neither was its two backup appliances, although there were signs the attackers had tried to encrypt the Arcserve 8200 Series devices but had failed for some unknown reason. One appliance sat in Building A, and the other in Building B, on the campus, and were set to run a data backup rotation and handle file compression for terabytes of the firms data.
So this was more or less all we had, as well as some older backup tapes that only contained data for the past four years, Larrue says.
And C.E. Niehoff had not actually set up the appliances for full system recovery yet — the devices were relatively new — so Larrue had to get help from an Arcserve engineer/technician to restore the backups to the new computers, which the manufacturing firm had to quickly purchase to replace the compromised systems. A couple of the systems that had been configured for bare-metal restoration were back online quickly, he recalls, but there were challenges with several other systems that had not been configured for full restoration.
We had to more or less rebuild the machine, which took longer to restore, he says.
One way to keep backup systems safe from ransomware attacks is to keep them on a separate domain, advises Gary Sussman, the Arcserve engineer who helped Larrue restore the manufacturing companys systems. He also recommends setting them with strong credentials and ensuring that hardware encryption is turned on.
In all, it took C.E. Niehoff two-and-half weeks to get all of its systems fully back up and running, starting with its email server.
Larrue says the company since has added additional layers of security and is working on beefing up redundancy in its systems and storage, including some cloud-based storage. Ransomware threats are the new normal.
The lessons learned here is this is an ongoing campaign and its not going to stop, he says of the threat of ransomware attacks.
Related Content:
6 Things To Know About the Ransomware That Hit Norsk Hydro
Ransomwares New Normal
The Big E-Crime Pivot
Ransomware Attack Via MSP Locks Customers Out of Systems

Last News
▸ IoT Devices on Average Have 25 Vulnerabilities ◂ Discovered: 23/12/2024 Category: security	▸ DHS-funded SWAMP scans code for bugs. ◂ Discovered: 23/12/2024 Category: security	▸ Debunking Machine Learning in Security. ◂ Discovered: 23/12/2024 Category: security

**Cyber Security Categories**
Google Dorks Database	Exploits Vulnerability	Exploit Shellcodes

CVE List

Tools/Apps

News/Aarticles

Phishing Database

Deepfake Detection

Trends/Statistics & Live Infos

Tags:
How a Manufacturing Firm Recovered from a Devastating Ransomware Attack