Ransomware Recovery Guide for MSPs
Lessons Learned from Axcient Partner Robert Cioffi, COO and Co-founder of Progressive Computing, after a Total Ransomware Takedown
Table of Contents
How confident are you in your ability to recover one client from a ransomware attack? How about more than five clients? What if all of your clients, plus your MSP, was hit? You may think it’s an impossible scenario, but that’s precisely what happened to an Axcient partner during the July 2021 Kaseya attack. Robert Cioffi, COO and Co-founder of Progressive Computing, suffered a total ransomware takedown – but was able to recover and is telling his story.
Robert’s company was the victim of a ransomware attack where REvil Sodinokibi infiltrated their RMM platform through a flaw in the platform. Over 2,500 endpoints were encrypted, including over 250 servers for roughly 80 customers across four time zones. Thankfully, the Axcient team was ready with the assist. Robert reports, “The Axcient business continuity and disaster recovery product worked almost flawlessly. We had a couple of little hiccups here and there, but we recovered just about all 250 of those servers. We had an incident on one, but it was a known issue, and it was such a minor thing that we’re scoring this a 100% victory.”
Not only did Axcient’s backup and disaster recovery technology work flawlessly for us, but the support staff that helped us get through that process was just amazing… They were very compassionate about our situation.
Now, there were other logistical challenges to their ransomware hit besides the disaster recovery and restoration of servers and critical infrastructure. The amount of manpower required to recover quickly far outpaced what Progressive Computing had internally. Axcient, as well as the MSP community, came forward with willing engineers who could help speed up the process.
…We had restored almost 100% of our customers to almost 100% levels within 17 calendar days. And I can’t imagine doing that without a company like Axcient at our back.
Robert also says, “I think it’s really important for every MSP out there to not only vet the solutions that they’re including as part of their cybersecurity protection – and we all know that disaster recovery is a critical part of the solution stack there – but like any product, it really needs to be tested. And it’s very difficult to test these things unless you’re in that real-world scenario. And I can tell you from direct, personal, and professional experience that it worked. And it worked flawlessly.” We here at Axcient think our product is pretty darn great, too.
TL;DR (Hey, we get it…)
Ready to learn from Robert’s experience and the Axcient team’s expertise? We pulled it all together in the free eBook, “Surviving a Total Ransomware Takedown: An MSP Quick Guide to Overcoming Today’s Cyberattacks” Download it now and get a head start on applying the lessons learned, or keep reading for a summary…
Total Ransomware Takedown: Recovery Lessons Learned
Today, hackers are only getting more sophisticated, complex, and relentless in their efforts to steal data.
Despite the abundance of research, regular headline news, and the target on MSPs and their SMB clients – some MSPs are still not prepared with comprehensive business continuity and disaster recovery (BCDR). On top of that, BCDR is just one piece of the recovery puzzle. MSPs also need cyber liability insurance, an incident response plan, and disaster recovery planning and testing to protect clients from potentially business-ending data loss. Not to mention the growing legal ramifications MSPs now face following a client’s cyber incident.
Luckily, it’s never too late to create a ransomware recovery guide. Based on what we learned during our recovery efforts with Progressive Computing we’re outlining some key takeaways, such as…
- Understand the current risks and realities of ransomware so you can educate clients, justify BCDR expenses, and proactively plan for recovery no matter what comes next.
- Take a multi-layer security approach with an “it’s not if, but when” perspective to ensure that you and your clients are always prepared.
- Continuously optimize your plan for recovery according to new threats, internal changes, solution updates, insurance policies, and available support systems because data loss is inevitable.
Ransomware Refresh: A Quick Look at What MSPs Are Up Against
Ransomware has been such a hot topic in the cybersecurity landscape, especially with the spike in 2020, that many professionals have grown numb to the realities of an attack. Unfortunately, thinking, “it won’t happen to me,” is the worst response an MSP or an SMB client can take. So perk up, and let’s talk about the state of ransomware today. But, first, according to the Cybersecurity and Infrastructure Security Agency (CISA).
An ever-evolving form of malware designed to encrypt files on a device, rendering any files and the system that rely on them unusable. Malicious actors then demand ransom in exchange for decryption. Ransomware actors often target and threaten to sell or lead exfiltrated data or authentication information if the ransom is not paid.
Today’s ransomware attacks are calculated, strategic, and advanced methods of stopping businesses from running. Whether it’s holding data hostage, shutting down operational systems and communication channels, or deleting backups, ransomware is a business killer with costly consequences. Here are some of the latest stats – be sure to reuse these when talking BCDR with clients!
- 78% of MSPs reported ransomware attacks on their clients from 2020 – 2021.
- 13% increase in ransomware in 2021 – a rise as significant as the last five years combined.
- 40% of ransomware incidents involve desktop sharing software, typically used by remote and hybrid employees.
- 35% of ransomware incidents involve email, such as phishing attacks.
- 50% of ransomware demands are more than $50,000.
- 80% of businesses that pay the ransom suffer a second ransomware attack, often by the same threat actor group.
What these numbers tell us is that ransomware attacks are continuing to rise, they’re expensive, SMBs are being targeted, and business enablement tools provide an opening for hackers. All in all, ransomware still needs to be the top priority for MSPs and the services they provide to SMBs.
With that said, it’s important to note that ransomware by itself is just a way that hackers monetize the access they’ve gained. If they cannot gain access to internal systems or have so many layers of security that their ROI is upside down, a hacker will most likely retreat. If you’re thinking, “great, cybersecurity tools here I come!” not so fast. Also noteworthy is that human error remains the number one cause of data loss and is often the open door through which hackers enter. These two realities require MSPs to implement data protection solutions and prepare for an inevitable ransomware attack.
Preparation is Key, but Ransomware Recovery Is Complicated
Ransomware recovery is not guaranteed just because you have a specific solution, cyber insurance policy, or plan – it’s only possible if you have all three of these components and more. Recovery requires your entire cybersecurity ecosystem to come together quickly and efficiently in a step-by-step process familiar to your recovery team. It’s not easy, it’s not simple, and nothing is guaranteed – but it can save your business and your client’s businesses from complete shut-down.
Here are the three vital components for a complete ransomware recovery guide that can also be used in other data loss incidents and natural disasters.
#1: Business Continuity and Disaster Recovery Product
A BCDR product is the easiest layer to implement in a multi-layer security approach. Unfortunately, too many MSPs believe this is the only thing necessary to keep the bad guys out. Sure, if you choose a comprehensive solution, built specifically for MSPs and SMB end-users, and it’s regularly updated and upgraded to address new cyber threats – it can do a lot – but it’s still just one layer of a larger strategy.
Why BCDR instead of backups? Because backups are dead. Today’s cybersecurity threats, growing attacks, dispersed workers, and state regulations have put the backup-only approach to data protection to bed. Hackers knew how much we relied on backups for recovery, so they started targeting them in ransomware attacks. While hackers may have killed backups, business continuity and disaster recovery solutions are going a couple of steps further.
For example, Axcient x360Recover for BCDR comes equipped with a number of proprietary features to protect MSPs and their SMB clients from ransomware, while providing rapid and reliable recovery in the event of a disaster. Additional features include:
- One solution for multiple use cases with Direct-to-Cloud – endpoint backup, no-hardware BDR, full-service BDR, and public or private cloud to protect various client environments.
- Patented Chain-Free backups significantly reduce restore complexity for near-instant recovery – no chains to manage, no base image requirements, no consolidation, and no staging space.
- AirGap anti-ransomware and data loss technology separate backup deletion requests from the actual deletion mechanics to prevent malicious or accidental deletion. “Honeypots” or fake signals trick hackers into believing they’ve successfully deleted data – but lucky for Axcient partners, it’s stored fully intact and available in a safety archive.
Video: Robert Cioffi from Progressive Computing Talks Rapid Ransomware Recovery with Axcient BCDR
#2: Cyber Liability Insurance
Insurance in the MSP space is relatively new, but it’s become more critical in response to expensive ransomware recovery costs. Cyber liability insurance financially protects businesses after a cyberattack or other incident where company and/or client data is lost. Typically, your first call should be to your insurance carrier after discovering a ransomware attack. They will provide a Breach Attorney to guide you through the recovery process from a legal perspective, including, but not limited to the following:
- Regulatory fines
- Media liability
- Breach management expenses
- Cyber extortion and ransomware
- Social engineering
- Reputation loss
- Business interruption
- Breach response and communications
#3: Incident Response Plan
This is the meat and potatoes of your ransomware recovery guide because it encompasses everything necessary for a smooth recovery in one pre-determined, step-by-step manual. Often based on the NIST Cybersecurity Framework, it takes MSPs through the following stages of a data loss event: identify, protect, detect, respond, and recover. Although every MSP’s IR plan will vary, these are some must-haves that make responding to a ransomware attack more straightforward so you can focus on recovery.
- Incident response team: Name and provide updated contact information for everyone involved in incident response, their role, and how they will complete their duties. Assume that internal systems will be compromised and access to email and phone won’t be available.
- Breach notification hierarchy: Determine the external stakeholders that need to be notified of the incident, starting with your insurance carrier and including your BCDR vendor, legal counsel, compromised clients, and state and regulatory agencies depending on breach notification laws in your location and industry.
- Internal and external messaging: Your Breach Attorney will provide legalese for incident notification and updates – but speaking like a lawyer to your clients, instead of in the familiar tone they’re used to, can cause panic. Strike a balance between what you’re required to say and what clients expect from you to maintain relationships and support worried clients.
- Risk analysis and prioritization: If multiple clients are attacked and your MSP is attacked, you need a risk analysis and prioritization framework to streamline recovery efforts. Consider clients in highly regulated industries that may incur penalties for long periods of downtime, or clients that may be inclined to litigate – and start there.
- Decision implications: Be prepared to justify any and all decisions you make once the dust settles. No matter what you do, there will be consequences – good or bad – for how you react. Consider the implications of each option within your guide to make it easier for you to respond under pressure.
Utilize these resources to build your IR Plan:
- 5 Critical Pieces of a Good Security Playbook
- Incident Response Checklist
- Huntress and Axcient: Planning for the Next Ransomware Attack
Disaster Recovery Planning and Testing
Another critical piece of an MSPs ransomware recovery guide is the disaster recovery plan. Unlike the IR plan, which focuses on navigating recovery from a business standpoint, the disaster recovery plan, or DR plan, ensures that all critical data, IT systems, and networks can be recovered. The information necessary for DR planning ensures that businesses are operational and will avoid costly and disruptive downtime in the event of a disaster. While the focus of this post is ransomware recovery, a DR plan protects infrastructure from anything that could cause an outage – from technical glitches and system failures to accidental human error, power outages, and of course, ransomware and other cyberattacks.
Central to DR planning is DR testing. DR testing puts your BCDR solution and the DR plan into action in a safe space to make sure it works. Undoubtedly, DR testing will always reveal opportunities to update and improve your DR plan no matter how well it goes. DR testing is critical to any business, but MSPs can use it with clients to showcase the value of their solution, encourage them to create IR and DR plans of their own, open the door for cross-sell opportunities, or highlight weak spots in their current levels of protection.
>> Download Axcient’s MSP Playbook for Best Practices in Disaster Recovery Planning and Testing
And check out Axcient x360Recover for DR testing made easy with Virtual Office. Axcient partners can perform regular full-office recovery tests to ensure backups are always available and demonstrate instant recovery of production servers and workstations in the Axcient Cloud.
Practice Makes Perfect – Every Quarter!
Ransomware recovery requires continuous optimization to ensure survival. Table reads, practice drills, rehearsals, and wrap-around discussions among technicians and operational departments are required at least quarterly to keep up with everything in your ransomware recovery guide. Some regularly changing aspects you need to be aware of each quarter include the following:
- Contact information and role changes for both internal teams and external stakeholders.
- Updates to solutions, software, tools, and other systems that may impact access, recovery capabilities, and SLAs.
- Evolving cybersecurity threats, vulnerabilities, phishing scams, and other tactics can require employee training, additional layers of security, or changes to response and recovery.
- Changes in cyber liability insurance carriers, policies, and required procedures are necessary for claims payment.
- New breach notification requirements, state laws, and other regulatory oversight that impact the who, what, when, where, and how of communicating data loss incidents.
- Revisions based on the outcomes of previous drills, lessons learned, and opportunities for improvement.
Are You Preparing for Ransomware?
Hopefully, this post has provided the outline and resources necessary to move forward with an all-inclusive ransomware recovery guide that includes a best-in-class BCDR solution, cyber liability insurance, an IR plan, DR plan, DR testing, and quarterly review. As a 100% MSP-only solutions provider, Axcient is here to help, whether you’re a partner or not.
Keep this important guide handy, and get even more info:
<< Download the no-cost eBook “Surviving a Total Ransomware Takedown: An MSP Quick Guide to Overcoming Today’s Cyberattacks”
About the Author:
Cory Hinz // Sales Engineer Manager, Axcient
Cory Hinz is experienced and passionate about technologies and sales engineering with a demonstrated history of working in the information technology and managed services (MSP) industry. Cory’s professional skills include solutions engineering, network architecture, data center, software as a service (SaaS), and training. He has a strong business culture focus, and a professional mindset, and is constantly looking for better ways to improve the business-to-vendor relationship while being the catalyst between the Product and Revenue teams.