It is a scenario that every CISO hopes they never face: the network encrypted by ransomware attack and cybercriminals who claim to have stolen sensitive data which they will leak if they don’t receive a ransom payment.

Unfortunately for Zach Lewis, CISO at the University of Health Science and Pharmacy (UHSP) in St. Louis, this was the reality he faced when the organization was hit by a LockBit ransomware attack in summer 2023. The University recovered without paying the ransom, but it was a challenging time for all involved.

In a conversation with Infosecurity, Lewis opened up about how it felt to be the victim of a ransomware attack and the toll it took on his team, as well as how the organization recovered from the incident and the lessons other cybersecurity leaders can learn from this experience.

Infosecurity Magazine: When were you first aware that something was wrong?

Zach Lewis: I got the phone call about 04.30 in the morning. There had been an outage. We couldn't get to any of our on-site servers or services. I tried to hit them remotely from home, but couldn’t.

So, I got in my car and made calls to the team on the way to the office. I said we were having an IT outage, but it was probably nothing major.

We got there and were just trying to get things working again. We managed to get things functioning after a couple of days: the ransomware hadn't been triggered at that point. What we didn't know was they were in the system.

A few hours later, after that environment went down again, that's when they launched the ransomware and encrypted the files. So, when we go back into troubleshoot the same way we did before it didn’t work because the files couldn’t be read and that’s when we found the ransom note.

IM: How did you react after discovering it was a ransomware attack?

ZL: I was in my office at the end of the day, and I saw the README file. I just remember getting a sinking feeling in my stomach, knowing that it should not be there. I opened it up and saw the ransom note: ‘We've got in your system, we've owned it, if you don't pay us, we're going to come back and get you again.’

You know it’s a make or break moment for your career. It's one of the worst feelings in cybersecurity when your environment's just completely down, it's been taken out by bad guys and you can't really do anything about it. It sucks.

After the initial shock, I called in the network director and COO to discuss next steps.

We had to call insurance immediately. They needed to be our first call. Then I needed to let the authorities know. I do a lot of work with the FBI so I knew who to reach out to, they could put me in contact with the right people. Then I called my wife to let her know I was going to be late.

IM: How important was it to have the COO and rest of the university leadership involved in dealing with the ransomware attack from the start?

ZL: Prior to the incident, I had built a rapport with the executive team. You need to have a working relationship with them, you need them to understand and trust what you are saying.

Fortunately, we had run a tabletop exercise around ransomware with them just a few months prior, so they were very aware of what it was.

So, when the ransomware attack happened, when systems were encrypted and unavailable, they understood what had happened and they put full faith in me. That rapport and history together meant they were going to trust what I said and what we were going to do.

IM: How did you ensure that your cybersecurity team was not overwhelmed by the incident response?

ZL: As a leader of the team, it fell on me to handle it. I didn’t want the team to feel overwhelmed. So I took all the meetings, all the questions from the board and from the leadership team.

I wanted to shelter the team as much as possible while they were working on recovery. They were dealing with the technical aspects and I didn’t want them having to deal with questions from end users, executives or whoever it may be.

I also needed to make sure that they were getting rest and going home at some point in alternating shifts.

But as the leader, as the CISO, it was stressful. There's just no way around it, but that's part of the leadership gig. That's what got you the big C, right? So, I took hold of that.

IM: How did you restore the network without giving in to the ransom demand?

ZL: The first move was to open negotiations with the threat actor to verify their claims. Did they have the data they said they had and how much did they have? We were trying to suss that out, while also buying time and thinking about how to recover.

That time was spent getting new equipment, trying to get to our offline backups and recover from them, then verifying that the environment was good enough for us to operate with.

We restored the systems we needed to make essential things work and we felt pretty good about that, so we weren’t going to pay the ransom.

But then they pivoted. That's when their whole negotiation tactic changed from ‘pay us to get a decryption key’ to ‘pay us a ransom to make sure we don't post your data on the dark web’.

We went back to the negotiations with them to ask what files they had. And at this point, we had to gamble on whether or not there was sensitive data there or not based on a handful of file names.

But we felt confident there wasn't going to be. So we just ran the clock out until we got the data drop. They just posted it because we wouldn't answer or respond. They said they had 200GB of data, but it was only 2GB.

Then with the data dump we went through all the data and sent an email to students, staff and alumni that said if there was anything of note released, they would be contacted individually. And that anyone we didn’t contact had nothing to worry about.

In the end we had to contact four people and I called them on the phone to let them know.

IM: Post incident, what strategies or tools have you applied to bolster your network security and prevent this from happening again?

ZL: Prior to the incident, we had been doing a big push to move all of our applications and infrastructure to cloud and SaaS platforms. Now almost everything we have is SaaS, so we're not storing data on site in any meaningful way.

We had looked at the enterprise browser Island before the incident. We liked it, but we just couldn't justify the cost out of our budget. But the incident happened, and all of a sudden, the board's asking, ‘What do you need?’ And I said what I need and why we need it.

Everything our users touch is in the browser via logins. So why was I trying to secure people's own devices, their home devices or their research individually when I could do it all from one place in the Island browser? The security is in there and they can still easily get to all their SaaS platforms.

So that was one big piece for us. We could centralize identity, we could centralize what users could see, where they could go. That's the bread and butter of our environment now.

That was a very big piece for us. And the second piece was just figuring out what our data was and where it was. We didn't want to be lost again.

IM: What is the key lesson you learned from dealing with this ransomware incident?

ZL: You're never 100% secure, there's always a way in. So, always test your environment and keep up to date with what is happening with new technologies and threat intelligence.

IM: How do you hope to help other security leaders by detailing how your organization was hit by a ransomware attack?

ZL: I hope that by sharing this story, it helps people talk about this issue. We should be talking about how our defenses fail, what controls fail, where the weaknesses are, how we bolstered them, and share that together.

The bad guys talk. They know who's hit who, and they go back and they hit the same people over and over again because they're sharing that information.

There's all kinds of different reasons why we as an industry don't talk about incidents. But if we work together we can beat this thing.