There's not too many scarier situations in information security than ransomware incidents. With Halloween around the corner here in the US, I felt it appropriate to share a horror story in one particular run in I had with CryptoLocker ransomware, back when Windows 7 still ruled the world. Though this story is focused on information security, the lessons I learned from this experience were really valuable in terms of interpersonal relations and influencing non-technical executives to value information technology and security as a function necessary to continue operating. Shall we begin the story? Gather around the digital fireplace.
My very first paid opportunity in IT was to work with a small and very successful business in the agriculture industry. I established a relationship with them through a mentor of mine, and helped them fix several issues over several months (including helping them move off of Windows XP and upgrade from 1GB of RAM per workstation, not kidding). So they trusted me and reached out anytime they had tech related issues. That said, I still hadn’t been effective at encouraging the owner of the business to see the value in managed backups, email and security products (such as enterprise AV, spam filtering, network firewalls). He valued my services and the prompt support I provided, so I figured I'd eventually be able to convince them to adopt more important tools and services over time. Everytime I’d mention cyber threats or malware, he would say “well, who would ever want to hack us?” I explained that not all threat actors out there are specifically targeting large businesses, some are criminal organizations targeting anyone and everyone they can including individuals and small businesses in order to turn a profit. He still continued to view security as an expensive and unnecessary place to direct resources. One call I received from them changed the dynamics and their perspective on computers and security forever.
I'm sitting in a college class at the time when my phone starts ringing. I see the contact name, and immediately step out of the classroom expecting the call to be related to some tech issue I need to attend to. The call is from the general manager of the business. They say in a concerning tone: “Bob, I seem to be having trouble with my computer, the screen is black and I cant open any of my files or programs.”
Immediately my heart sinks to my stomach and my brain thinks “ransomware.” . My optimistic mind hopes it’s something else. I respond with: “Is anyone else experiencing this issue?”
The general manager states: “Not exactly, but it seems that no one is able to connect to our parts database”.
Part of their business involved buying and selling parts for farming equipment. I ask: “When did you all first start noticing this?”
The general manager mentions: “I personally noticed this before the end of the day yesterday but figured I’d wait until the morning to see if it fixed itself before calling you.”
I advise the general manager to power down their computer and wait for me to arrive as they were a non-technical user, asking for them to unplug the network cable may have led to more complications. Powering down is generally frowned upon by forensics experts as it eliminates the chance of retrieving anything in memory, such as a decryption key to ransomware. A better route would’ve been to leave the computer running, yet isolate it so any malware that was potentially running could not infect other computers on the network. My main concern was to stop a potential spread. It may sound funny but if I asked the general manager to disconnect the cable, they may have unplugged the keyboard and thought the job was done. This is one major con of me asking them to power down the computer instead of disconnecting the network cable. I quickly get to my car and travel to the business. I arrive and head back to the executive office. The first thing I notice is everyone’s emotional state. I'm seeing frustration, fear, and nervousness. I personally am feeling a bit nervous, but I try to hide it because expressing my nervousness wouldn't help the situation. I assure everyone that whatever is happening, I am going to fix it and make sure they can get back to serving their customers. They seem relieved to hear this reassurance.
I follow the general manager back to their office, unplug the network cable from the NIC (network interface controller/card) and power up the computer. I disconnected the cable to do my best to prevent the malware (if it was malware) from spreading over the network, if it hadn’t already. They log in to their domain account and sure enough the desktop background no longer shows an image of their family , it's just black. They try to open their running ledger file in Excel. They had been keeping track of all transactions in this Excel file on the desktop, this is the first time I had become aware of the existence of this file. It does not open successfully. This had all but confirmed my worst fear about this situation. We are dealing with ransomware (que spooky music). To confirm I open file explorer and start looking in directories for text files. Sure enough I see text files in almost every directory. In these text files are instructions on how to retrieve a decryption key to decrypt the data. My mind immediately jumps to considering the domain controller and file server (though not a best practice, it's common for smaller organizations to run multiple services and server roles on the same OS install).
I walk into the server room and plug in a keyboard and monitor into the computer running the domain controller (Active Directory, a centralized database of user, group, and computer accounts) and file server. The reason this is a concern is because I know the software they use to access the parts database relies on mapped network drives from the client side to the shared folder (via SMB) where the database and program files are stored for the application. I discovered that the desktop background of the server is also black and it appears as though there is a txt file with decryption instructions in every directory. The server had gotten hit and I suspect it was from the same ransomware that started on the general manager’s computer, then spread to the server through mapped network drives.
At this point, I decided to call a meeting with the executives of the company. We sit at the conference room table in the CEO’s office and I discuss my findings. The general manager admits to having clicked a link in an email that looked like it came from UPS. This came to their @yahoo.com account that they use for business and personal matters.
The seriousness of the situation gave me the courage to explain the importance of being security minded and us all working together to remain proactive in trying to prevent as well as respond to cyber threats like this.
I would go on to conduct simple training with them to become more mindful of various cyber threats they may face, and how to avoid and respond to them. The CEO realized that small businesses are also in danger of cyber attacks, and it's not always someone in a black hoodie trying to manually attack an organization. (sometimes it’s a team of people somewhere in an office writing, and/or distributing malware and just blasting it out en masse as their day job). I share my plans to build the domain controller from scratch, and attempt to use some of my data recovery tools to retrieve a version of that running ledger file from the hard drive of the general manager’s computer. And to work with the software vendor to restore the parts database and the application. (The vendor had their own custom backup solution running. thankfully).
Many hours later, with a brand new domain controller and unsuccessful attempts at recovering the running ledger file, I decided to do a fresh install on all the computers in the office just to be safe. I'm not a fan of virus removal as a long term solution, just wipe it. The CEO pulls me aside and says: “Whatever you think we should do I trust your judgement, just let me know what we need and I'll get it.”
I ended up moving them to Office 365 (now Microsoft 365, gives more control of mail traffic flow, permissions and spam filtering), setting up a cloud-based backup solution (allowing for more granular options with regard to versioning and restoration), remote access tools (allowing for remote access and quicker response time to issues) and an enterprise-grade AV solution (allowing for better alerting and reporting options).
After that incident there wasn’t a whole lot I asked for that the CEO did not approve. That said, I learned a major lesson that I still carry to this day when it comes to showing others the value of computers and security no matter how big or small the organization. That lesson is don't be afraid to disagree or have difficult conversations for the greater good. Up until this incident, I had been hesitant to have those difficult conversations with leadership. It was almost like I had feared that disagreement or coming off too strong and serious would somehow ruin the opportunity to work with them.
Now anytime I work with an organization on their IT matters, I always emphasize security first and I take it very seriously in my communications. Especially when I face resistance. Because the truth of the day and age we are in is that cyber attacks can put organizations out of business if they are not prepared and do not have adequate support.
Here are some of the other lessons I took with me:
Identify and document locations of all business critical data, including any data that may be on employee workstations.
Create a business continuity plan and disaster recovery strategy that is easy to follow and understand for technical & non-technical stakeholders.
Consider the 3-2-1 rule in your data backup strategy. Is there 3 different copies of the data, in 2 different formats with 1 copy off-site or in the cloud?
Ensure all business related communications are managed through enterprise email solutions and/or collaboration suites connected to a namespace the organization owns.
If it's in a small or midsize company’s budget, find a Managed Security Service Provider to provide ongoing monitoring of end points, server and network infrastructure.
If you are limited on hardware to use for servers, try to use hyper-visors like VMware ESXI or leverage cloud-based services (ex. AWS, Azure VMs, Azure AD or GCP) to create different OS installs to separate workloads so a file server being infected won’t lead to a domain controller also being infected.
Start using cloud-based file storage and office suites with simple documents and spreadsheets.
If much of your environment is running on equipment on-premises, look into IDS/IPS appliances that can catch malware at the network level or even allow you to deny infected systems from communicating over the network. Some newer managed network switches provide this functionality.
Learn more about the offensive and defensive side of security for your business through stories like this, and hands-on training with the latest tactics, techniques and procedures on Hack The Box through HTB Academy for Business, Professional Labs and Dedicated Labs.