EW 051 – How To Successfully Deal With A Cyber Attack By Being Well-Prepared – Winning with Jesse Hill, President of Tier3 IT Solutions

Gordon Sheppard (00:00.993)
Jesse, welcome to the show.

Jesse Hill (00:03.608)
Thanks for having me, glad to be here.

Gordon Sheppard (00:05.333)
Well, it's so good to have you here because there's not many folks that do what you do for decades. There's a lot of come and go in your industry. There certainly are not reliable partners given that the IT work that you do and beyond, there's so many changes. So this multi-decade thinking to have someone on, know, in your leadership role in that industry with all of that pedigree in behind you, you've been through all the different eras. And so the client services that you would bring to a given situation,

are super strong. And I think I just am excited about hearing what your win's gonna be. But before we kinda get going, I think maybe a lot of people haven't met you. So if you had to describe yourself, what would you say?

Jesse Hill (00:44.814)
I mean, I think first of all, we start off with, you know, father of three, you know, husband of one. And then of course we get into the business side of things. And so I am, as you mentioned, second generation in this business. I've been involved in it one way or another since 1990. So we're going on, we just had our 35th anniversary. Of course,

My parents started it and I got more involved when I graduated from college, took over operations in the early 2000s and then acquired the business in 2014, 2015. And as you mentioned, we have gone through every single iteration you could imagine when it comes to technology, computers, networking, the cloud, and now of course moving into the world of AI.

And so constant reinvention, it's one of the things that I love about this business, this industry is that there is never a dull moment. There's always something new and exciting. And truthfully, it gets us engaged with other business people and gives us the opportunity to see how we can affect their operations, their people, their teams to help them reach their goals.

Gordon Sheppard (02:06.967)
How lucky to have you in the room with that level of experience and confidence, because it's funny about seeing those different things that come along. In the end, if I describe this wrong, I'm going to say it's tools. That in the right hands with the right user, with your guidance, those tools can work. There's a lot of hype around the AI thing right now, but in fact, I think what I boiled it down to is it's one of the tools, and you're either going to kind of get it right in a sense that ties into your overall business and strategy goals.

Jesse Hill (02:17.912)
Hmm.

Gordon Sheppard (02:34.773)
rather than it being the solution to fix all things. Is that the right direction for that conversation?

Jesse Hill (02:39.688)
You know, and a hundred percent, and I find myself talking about the tools less and less as time goes on. You know, of course, the analogy that I use is that you and I and a professional can all walk into the grocery store and buy the same ingredients, go to our kitchens and bake a cake. And mine, I don't know about you, but mine and yours probably are not going to look like the professionals. And

It's not because we didn't have the same ingredients, we didn't have the same pots and pans, we didn't have an oven. It's because we don't have the knowledge, the expertise, the skills, the discipline, the process, the people that go into producing a high quality product. so the same thing is in technology. Again, I find myself talking to prospects and clients about this all the time is that

Every IT company can buy the same firewalls and antivirus and patch management solution and ticketing platform and all of these things. So when you remove all of that, what is it that they actually are looking for? And they're looking for expert guidance to help them align technology with their business priorities.

Gordon Sheppard (03:55.391)
And I love it because you're talking second generation. So you grew up on the hip of somebody going through and please take me back to the eighties. did data management cards in high school. They put me through one of those awful moments in time to tell me learn what I don't, why I don't do you, what you do. And there's that kind of thing. Funny. You take me back to an MBA analogy that a professor of mine used to say, said, what level of complexity are we dealing with here? Are you baking a cake? Are you flying to the moon? A little tougher, but we can get her done. Or you're raising a child. And I think often.

you would be in that position of being the advisor at that higher level of complexity where there's a lot of moving parts in a business. Maybe the strategy is not there, but you're able to bring in then your lived experience, your work experience and confidence to help them find a real IT solution. Is that about right?

Jesse Hill (04:40.686)
100%. It's funny that you brought AI into the discussion already. was just chatting with a colleague of mine this morning and we were talking about how AI has changed the conversation and changed the selling and the buying side of the relationship. We got into using AI a year a year and half ago to help us summarize meetings, prepare follow-up questions, craft

RFP responses and all those types of things. And now we're starting to see buyers using AI. And what's challenging with it is we sit in meetings with them and we realize that they don't understand technology. They have a grasp of where their business wants to go. And then, you know, we leave and then an hour later we get this list of questions from them that clearly generated by AI that based on the conversation we just had with them, they don't understand.

And I actually put a LinkedIn post up about a week or two ago where I said, know, dear prospect, the questions you're asking aren't going to solve the problem you're facing. And so I think that's one of the challenges that I'm seeing in AI right now is that, you know, both buyers and sellers are using it, but it's not getting to the root of the problem that they're trying to solve. And it's making it harder for them to figure out is this the right

company is this the right advisor because it's so focused on tools and whatever's going in on the back end side.

Gordon Sheppard (06:18.422)
And again, it's them in a, just like what you said earlier, right? So that level of expertise is not there. So when they need someone with three decades of experience and the confidence to sort of help them meet their business objectives, you've been in it that long, that's the only thing you've been doing, certainly for the long standing clients that you have. And I'm so glad that that would be at the core of the thinking as you sort of pull through these crazy times and what's after AI.

You know, like it's funny, there's going to be, there'll be the next thing. If we, you know, if we, if we're old enough to go dot com bubble, you know, and whatever these things are, it's great. Well, we could talk all day long about the nice wide ranging conversation in your area, but this podcast is about picking a win. So if you had to choose one of your biggest wins in the work that you do, what would that win be and how would you describe it?

Jesse Hill (06:54.754)
We could.

Jesse Hill (07:03.148)
It was tough when you asked me the question and the one that I thought about was actually coming out of the most challenging moment that we had in this business. And so as you can appreciate, our world has become much more connected and dependent on technology than ever before. And we are, even as an IT services company, we are dependent on the technology partners that we choose. And

The difficult time that we found ourselves was back in 2021, a certain piece of software that we, along with tens of thousands of other IT companies around the world use. A gentleman in Belarus discovered a vulnerability in that software. And it's what's called a zero-day vulnerability, meaning that it has been known about for zero days. is not, nobody is aware of it except

you know, this one particular fellow and he saw an opportunity to leverage that and to push malicious software out using the tool. And so, you know, I don't want to get into all the details, but we were unfortunately one of the impacted parties, which means our clients were impacted by it. And, you know, it caused just a massive amount of, you know, chaos and challenge in our business.

and that of our clients. The reason I wanted to bring it up as a win is that that was 2021. We're now sitting here December of 2025. We survived the incident. In fact, we actually came out it afterwards stronger in the relationships that we had with our clients. We had many clients who came to us afterwards and said, my gosh, we are...

so thankful and so lucky that we had you as our partner to go through that. I don't know if we would have survived if we weren't working with you, which I found really interesting because we chose the platform that caused the problem. But when I think about it as a win, the reason I chose it is that we came out of it in that way because of the work we had done before.

Jesse Hill (09:26.702)
And so again, not to go too much into detail, but in October of 2020, we brought our team together, we locked ourselves in our boardroom and we ran an incident simulation exercise. And I will never forget it because we clicked play on the video we were watching and the facilitator starts off and says, imagine this, it's the Friday of the Independence Day long weekend.

You know, your lead technician is off grid fishing. You get a call from a client. They've got this message and then another one and then another one. And we basically locked ourselves in that room for five hours and we would just pause it every 15 minutes and say, okay, what would we do? What are we missing? What do we need to put on our priority list? And we started chipping away at all of those items. And on July 2nd, 2021, so the Friday,

Canada day long weekend, but it actually was Independence Day long weekend because July 4th would have been the Sunday in the States. We ended up living the exact scenario that we had role played eight months earlier. And because of the work that we had done from starting that simulation exercise to it actually occurring was the reason that

our clients came to us afterwards and said, thank you. You are the ones that helped us get through this. And I think when I tell people about it, what I tell them now is that we developed a very unique skillset that I can confidently say when I enter a room that I know more about cyber insurance.

cyber incident response, recovery, and all of the things that go into a scenario like this than most other people that you're going to meet.

Gordon Sheppard (11:31.508)
Because it's lived experience. I mean, there's so much good stuff to unpack in there. One, never waste a good crisis, right? That's a really important lesson that a lot of leaders kind of need to learn. And in fact, these things are going to happen. Like, it's funny, I think if you had a 10 year stretch or a 20 year stretch in a business, you're going to deal with something. But did you hire the folks that invested in firefighting prevention, right? Like in terms of what you're discussing, which is that thing of, you're the...

Jesse Hill (11:55.352)
you

Gordon Sheppard (11:58.902)
you're the provider that actually invested in that simulation. That could actually be on an interview card right now to say, have you put the time in? Not only do you have lived experience and have you reacted to this before, but an, oh, you were proactive about it. Well, why were you proactive about it? And that really speaks to getting into what I call the anatomy of a win to help break down this conversation where we talk about the strategy, the work plan, the people, the operations and results. So there's something in strategy, vision, mission, values, values. The value here is that you want to be ready. So you took the time.

to get your staff into a boardroom, to have that simulated moment in, know, prior to this even possibility, what was it that helped you to make that investment? Because a day of training is pretty precious to a company, you know, any size, doesn't matter what it is, but why did you make that investment? Was it from a strategic point of view?

Jesse Hill (12:48.034)
Yeah, you what, you brought up the word proactive, I think in there, or you talked about being reactive firefighters. we had joined a peer group in the start of 2020, right when everything was changing in the world. And the, I guess, core element of this program, this peer group that we got into was

you know, really establishing what it means to be proactive in running an IT company. And I tell people this all the time, if you'd have met me six years ago and asked me, I would have talked about tools, good antivirus, good backups, good patch management. And if you have a problem, give us a call. We've got a great team of technicians that'll, you know, ring the alarm and rush over and solve your problem. And the perspective that we got when we got into this peer group was that

Proactive actually, like the tools are just kind of base level stuff. You have to do that. Proactive is about having a plan, having people in defined roles that are reducing the number of interruptions your clients have from technology. And they have a whole video library. And so we were legitimately just chipping our way through this training and we got to this incident response thing and we went, I mean, we've never done it. We...

we should learn about this. And so in a sense, we almost stumbled into that exercise. Now it's part of our essential, know, recurring process that we take our team through it.

Gordon Sheppard (14:25.383)
I love it. And I remember running into a set of serious executives one time and they had done it in binder days, not even in sort of online days when that binder could walk out of a burning building and that binder was an hour away from having them up and running again. And then they could be back up and running operations within, because they had spent the time to be ready. And isn't that great that you've got ready yourselves so that then when this moment occurs, cause there's, there's a first moment, like you just said, like it's one thing to be in the sim.

It's another thing, was there a first email? Was there a call? Like what started the chain of events in 2021 that you wouldn't have known right away, because you said it was a zero, and thank you for that term, a zero day concept. So it couldn't even possibly been on your radar. What was the first moment when you said, uh-oh, what's something's going on?

Jesse Hill (14:55.662)
Thanks.

Jesse Hill (15:14.318)
10.35 a.m. into the brain. Yeah, we had one of our support techs, you know, took a call from a client and the client basically reported they've got a pop-up, you know, talking about ransomware and that was odd in itself because again, we have, you know, processes and tools and things in place to avoid this. And then someone else said, oh, I've got a call as well.

hey, can we get some help on the phones? And I remember, again, I was working from home that day, that I popped into our phone system software and I looked at it, I was like, we have 18 callers phoning us right now. And I mean, we get 18 calls a week, right? Most people communicate by email. And so I hopped into the queue and took a phone call and went, okay, this is bigger than...

than one or two users. And so I immediately messaged my leadership team and said, okay, we need to figure out what's going on. Let's meet at the office. And I remember my wife, it was a beautiful Friday morning. She was out in the backyard and I went, I gotta go to the office. I don't know what's happening. I'll see you when I see you. And when we got here at 1105.

we determined that it was an incident that we need to enact the plan that we started in October. And this was the magic was that we pulled up that document and said, okay, JP, you're handling this, Roland, you're handling this, Cam, you're handling this. I am calling our breach council, which again, I had interviewed in April and put on retainer.

and was able to just pick up the phone and kick that process off because we had lived it previously in practice.

Gordon Sheppard (17:13.621)
That is just remarkable. So the moment 1035 occurs, you're in the office, it's 1105, and you crack open the procedure, which not a lot of other companies, I'm gonna guess, across this, even this incident alone, would have made that investment in because so often they put things off. But let's back all the way up into your own strategy, which you say you kind of fell into that training, but I don't think so because you joined a peer group.

So the peer group is a strategic thing to do to get into a peer setting. Am I allowed to ask you what the name of the peer group is?

Jesse Hill (17:44.419)
Yeah, it won't mean much to most of the world, but it's called True Methods. It's led by Gary Pica and Peter Bryden. They've become great friends and mentors of ours now, and I recommend it to everyone.

Gordon Sheppard (17:57.161)
Yeah, love it. And so true, so you make that investment and it's often, those peer groups are often a substantial, at least one time investment for what they are. But in the meantime, I don't know if you've done the math on it, on the payback, right? Right, it could be a thousand times in terms of what you're talking about. So now we've got, we do have a strategy because there's the readiness strategies generally, you kind of get into this area, but then you're ready. then now the incident is happening and you've got your way to go, you're able to point.

Jesse Hill (18:09.737)
off.

Gordon Sheppard (18:26.037)
who does what in the given moment. So your response time is probably unbelievable in terms of at least working through what's not a pleasant set of issues to work through, except you're the ones now working it through. And the win is the clients coming to you after saying, thank you for being so ready when this occurred. So we've got a strategic sort of highlight figured out there. The second step in the anatomy of a win is to take a look at what was the work plan. So is the work plan in this case from start of incident to resolution, it?

minutes that we're talking about here? Is it days? How long did it go from 1035? Was it three months? Like what was the sort of timeline in a sense where we can pick off some highlights in the work plan?

Jesse Hill (19:03.81)
I'll throw in one more interesting bit of the timeline from that day because we've already outlined 10.35, 11.05. It wasn't until, I believe about 2.30 in the afternoon that the software vendor sent an email acknowledging that there might be an incident that's affecting multiple partners around the world. And to answer your question about the timeline,

I mean, we just went into all hands on deck 24 seven, how do we get through this? And ultimately, it was two weeks later to the day that we left the office at 5pm and looked at each other and said, like, okay, we don't have to come to work tomorrow. We've done it, the big work is done. There was a whole bunch of cleanup that happened for the next three months.

But that was the amazing thing was that because we had practiced it, we were able to bring our team together, have a meeting later that afternoon, tell everybody what we knew, what we needed. And I mean, I'm telling you, Saturday, 8 a.m., everybody was online. Everybody was ready to work. We were calling clients, we were investigating, we were doing everything that we possibly could. And a large number of our clients,

were operational again before the end of the weekend. It wasn't pretty. There were things that were still messy and needed to be cleaned up. But because they came in and they worked, we were able to do so. I will also point out one really important thing that I remember about the practice session that we did was in the simulation, they talked about the importance of

food and rest. And it was really, really impactful because when you're in a situation like that and you're facing down your absolute worst nightmare, which it was my absolute worst nightmare, you just want to work as long and as hard as you can. And that Friday night, I don't know what time it was, probably 10 p.m., we just looked at our core people.

Jesse Hill (21:25.186)
And we just said, you need to go home. You need to sleep. You can't make good decisions if you're tired and hungry. So we'll see you at eight.

Gordon Sheppard (21:36.615)
Isn't that such a non-Wall Street approach in terms of what we're supposed to do here, right? So that's really interesting. Now we've got a strategy feel. We've got sort of these moments of up and running by the end of the weekend in the ugliest way, but they could function within two weeks. The heavy lifting is done, let alone the of the fallout through those next few months and whatever that looks like. Which in this world, in these types of crisis is probably a pretty quick response.

to this type of thing for, like you said, something that nobody could have known about, which is, I think our perpetual world is what I'm just guessing overall, that you can't do 100 % guarantee in your world just because of all those things, like you said, from third-party vendors and stuff that are beyond your control, but you can do what you did do, which was uphold these three decades of reputation, uphold this sort of investment in getting ready and doing those things as you kind of go forward. Wow, that's really interesting. So strategy.

work plan, we've got to feel for that. There's always people involved. And I'll just go back and you were the type of leader in a sense, or the leadership team at that time, is able to say, hey, we can't be good decision makers here right now. That's the type of leadership. There's a communication element to this of what's going on. How would you describe, I'll start with you, but I loved what you were pointing. like, and you could see that you're pointing to the people in that moment where you had this name and that name and this name and how important they were. How would you describe yourself and your attitude? It's 1035.

Life is pretty good. You said it was a summer day or whatever it is, things are fine. And then, right, how would you describe your own attitude, the peaks and valleys through next 24 hours, next, yeah, how would you, yeah.

Jesse Hill (23:02.296)
Yeah.

Jesse Hill (23:08.726)
Well, okay, so this is a really important part and again something that a lot of people don't maybe appreciate until they're going through it. I sure didn't. I think of myself as quite a capable, resilient person. I lost 10 pounds in the first four days. I could not eat. I tried my hardest to make sure that I sat at the breakfast table with my family, with my daughters and my wife and just tried to make that seem normal for them.

And I remember one of my daughters saying, daddy, aren't you hungry? And I'm like, yeah, I am starving, but I I physically could not eat. And in the simulation, they actually talk about that often the owner or the CEO is not the best person to lead an organization through that type of scenario because it is so personal and so, so challenging. I actually,

in the fall of that year talked to another MSP, an IT company who was impacted down in the States. And he just said, yeah, he turned his phone off. He went and got his guns and gave them to his neighbor. And he just completely ignored the problem. And then on Tuesday, he changed his voicemail on his phone to tell all of his clients to

just pack their computers up, bring them in, they're gonna erase everything and start from scratch. And so that was the contrast after the fact that we had of what we had actually accomplished through our preparation. So my role, I think as a leader, in that is to make sure that our team is prepared and equipped and supported. And being part of the peer group, again, was a huge benefit because

On this Sunday, I was able to call a meeting with nine other IT companies who are all over the world at that point and tell them what was happening and ask them for help. And they were able to provide technical resources and licenses for tools and all kinds of other things. And so I found myself over the next couple of weeks often

Jesse Hill (25:31.609)
thinking about the choices I was making of saying, can sit and I can work on a computer and I can try to fix one user's problem, or I can try to find a solution that will solve 100 users problems. And so it really caused me to think about my leverage and how I can fight that fire, right? I can be one guy spraying a hose or I can go...

make a deal to get a fleet of bulldozers to come and build a fire break. And I think I actually used that exact analogy when I was talking to my team. So it was really about supporting and enabling, monitoring them and working to keep them healthy so that they could get back in the next day and fight the fight again.

Gordon Sheppard (26:23.796)
It's just such a great set of lessons in lived leadership. I got ready. We actually went through the simulation to kind of get ready. And then I was able to lead with sort of above the trees in a sense, like you said, and with bulldozers instead of a garden hose in terms of what you were dealing with. That's really interesting along the way. In the meantime, internally, you're losing your mind. Like, you know what I mean? Like this is not easy stuff in terms of sort of putting on that brave outward face that

that leaders sort of need to do in these moments of crisis. So there's a price to be paid in a sense for your own, like you said, losing weight and those things as well. But you got that kind of, does that make sense?

Jesse Hill (27:02.176)
100%. It's only really been probably the last six months that I've taken to talking about this publicly. And there are instances where, so I now run incident response simulations for our clients and industry groups and those types of things. And I always start off with a slide that says, why should you listen to me? And I tell them this story. And almost every time I get to the part about the,

emotional challenges. I still tear up in front of a room full of people about the emotional impact. And so it is very challenging and impactful, but important to share with people because they need to make decisions about how they're going to handle the same kinds of stresses.

Gordon Sheppard (27:54.004)
Well, and I can't believe the value that you're bringing to this interview for all leaders, because all leaders have to go through your world. Whenever I'm on the general advisory side, working with leadership teams, I'm like, IT has to be in every conversation. They're like, why? I'm like, they have to be in every conversation because you can't do anything without them. So I'm so glad to hear about your approach, but what do we say? We have a cyber incident response and you're talking about asking for help, building capacity, taking good care of yourself. I mean, these are, you can see how much bigger it is than

Jesse Hill (28:05.806)
Mm-hmm.

Gordon Sheppard (28:23.701)
Let's just put in the code and stamp out the problem or find the bad guy. Right? It's way bigger.

Jesse Hill (28:27.309)
Yeah.

way bigger. And I will tell you, in that incident, had, I mentioned having meetings with our team, talking about what we needed. We had one person who had planned holidays at that time, and he was offline. He didn't join any of the efforts. And we respect that. He had it booked. He did his thing. What caused challenges after the fact was he made a comment.

that he was happy that he wasn't there during it because of what he would have perceived the stress. you can imagine that when the rest of the team sacrificed parts of themselves to get through this, to hear somebody make a comment like that, that was challenging. And I do believe that we ended up losing one person afterwards from the stress. I should...

We lost the employment of one person.

Gordon Sheppard (29:32.924)
And right in terms of interest and here we are talking about, I'm not gonna say PTSD, there's a hangover, there's emotional, like you said, even when I go to relive it today, it was that impactful that you're talking about. It impacted your company work culture. It impacted attraction and retention. The type of person now that wants to come to work for you would hear that reputation and the type of person that needs to leave your organization needs to hear that, wow, they're so, like again, I don't know that the insurance companies are budgeting for,

therapy down the road, right? But they should be because that's the level of impact when you, you know what I mean? And again, if I don't even, I don't even mean to hazard a guess, but if we're talking about dozens or hundreds of clients of yours at a given moment in time, you're actually then the node for economically a billion dollars worth of business. Do you know what I mean? Or whatever, hundreds of millions of dollars worth of business reasonably, it wouldn't be hard to rack up those numbers pretty quickly and say, well, you're the node to that. And that all stopped.

Jesse Hill (30:03.49)
Mmm.

Gordon Sheppard (30:30.888)
because of this fella, bad actor, somewhere else in the world. Wow, this is so lucky to be getting this out, because I think anybody that hears this will hear the value now of, well, what am I investing in? Is it just going to be a patch? Is it 30 bucks a month? And you're like, no, you know what it is? We're going to talk about your whole business and why being prepared is so critical. again, if I catch wind in the news now and again in terms of who's being impacted, but I don't think, again, all the way down to little individuals, all the way up to those major organizations that are getting hurt.

this is a really valuable conversation for them to hear. Strategy, work plan, there's people, sorry, I need to know, there's your team that pulled together. Any critical people that you need to mention, not necessarily, you mention them by name if you need to. There's your family that was supporting you, normalizing your breakfast at least for an hour a day or whatever that was. There's some element there, there's your clients. Anybody that you need to point out on the people side that really got this win to occur and take your time, like yeah.

Jesse Hill (31:28.066)
Well, first and foremost is our whole team. There are core people, my leadership team, I mentioned before, JP, Roland, Cam, Kerry, they were really central to the strategy. But then I really look back and I think about what it would have been like to be a support technician.

you know, a little bit more of a frontline, you know, staff member. Because when we did the exercise in October, it wasn't the whole company, right? I mean, it's, as you mentioned, it's difficult to do that. So some of them hadn't had the practice. They had heard that we'd done it, they had heard about some of the plans, but for them, they're getting thrown into the deep end. And...

As I mentioned, we called them in to work on Saturday, on Sunday. There was overtime, there was all this stuff. They were compensated for it and all of the things that you might expect, but it was more stress than any of them ever thought they would have signed up for, I'm sure. Again, like you, hope that they, whether they're still here working with us or they move on to something else,

that they always are able to tell the story about what they learned and how it was valuable to them. We mentioned those outside advisors, so the peer group. The fact that I had them, a relationship at that point over a year where, again, I was able to get a couple of guys from Australia and a bunch all over the US and one in New Zealand to get on a call on Sunday morning.

and share openly. That is the biggest thing with the peer group is that because we're geographically separated and we have this trust and relationship, they know as much about my business as I do. And so they were able to help out. And then just having those advisors, good lawyers, insurance contacts, those types of things that were critical.

Jesse Hill (33:48.238)
to it were important.

Gordon Sheppard (33:49.726)
So many different factors. And I'm just imagining those frontline technicians or whoever was on the phone or you on the phone, along with clients that are like, I can't get my email. I can't get my payroll out. can't, I can't, and they would just be again, everything from pulling their hair out to, whatever it is they need to get through it as well. And then nice that there was a bunch there that stick stuck with you through the tough times in a sense, and we're able to reflect and go, wow, the win, it really was a win.

for something that was really critical as things, because the real time rollout is tough because nobody knows what's going on. And then you get to those weeks later and they were able to reflect in that moment. So again, we could examine all that, but that's really an interesting strategy, work plan, people, operations. So you've described the operations of getting this win quite a bit. We went through the sim, we had the playbook, we had sort of a checklist.

90 % completed or whatever that is. We had ways to go forward and sort of check the boxes. Would you describe any other sort of operational highlights to pull this off?

Jesse Hill (34:50.104)
Well, I'd say part of the thing that we haven't really talked about here is that while we're fighting this fight and trying to get everything back to normal, we still have a business to run.

Right? I mean, we still have clients who have, you know, the regular issues they needed help with. We still have to manage our receivables and cash flow and, you know, our payables and vendors and all of those types of things. And, you know, it's actually one of the things that's kind of interesting. was chatting with one of our team members a couple of weeks ago and reflecting, like, right.

he started with us in August of 2021, which means I was screening and interviewing and recruiting while we were going through all of this stuff. And so I think that that's one of the things from an operations perspective that a lot of times gets overlooked is that we now have two jobs and each of them need

your attention.

Gordon Sheppard (36:02.259)
And you're in the regular services business, right? Like you're in the day-to-day, the week-to-week, the maintenance, the updates, the whatever those things, all the different things that you do along the way. So now you got lucky, well, not lucky, unlucky, you had two businesses to run at the same time. Again, I don't know if all the readiness was there to do it, but that demand on your people and that push, that's a really good point that you're saying, well, we had the operational playbook, we did those steps, and then in the meantime, we had our other thing to run too. So everyone's going.

100 hours a week or whatever it is, just to try and get those futons out on the floors and make it happen, right?

Jesse Hill (36:38.706)
100%. And it's actually part of when I run the incident simulations for groups, it's part of what we've built into our model is that when you make a decision and you say, the CEO is going to fill in the blank, whatever that is, well, that person is now occupied. They can't also meet with insurance and also meet with the bank and also run a meeting with your clients and also

do these other things, you are limited. We are all restricted by our capacity.

Gordon Sheppard (37:13.723)
And leadership wise, you brought that out earlier, which is what is my role in a crisis? Because I think when we think about leaders who want to be fixers or solvers, who end up taking on too much in the regular tour of duty around a crisis, they might think, they can't be everywhere at once. So then that gets back into how prepared were you with the team building that you did, how prepared were you? And so this is really interesting because this, funny enough, we could port this structure into other areas of a business.

for readiness and because I think there's a lot of really strong fundamentals in it that are super well rounded and how nice to have that preparedness approach and so detailed and that investment that was made by the peer group leaders that you got that you've now gotten to adapt and with your lived experience make even better it would be iteratively better and then of course iteratively we'll get new trouble on the horizon which then you have to adapt to any ways you want. Great strategy, work plan, people, operations. Finally there's results. Quantitative results, qualitative results. Quantitatively

you got through the crisis with X number of clients and solved problems. Do know what mean? Any other quantitative highlights that you'd like to point out?

Jesse Hill (38:17.742)
quantitative. Well, what I will say, the outcomes from it were first of all, we didn't lose any clients as a result of the incident. None of the clients paid the ransom or had a need to pay the ransom because truth be told, the protections we had in place largely worked. The third thing that I think is really important was that, as you mentioned, this is impacting

a very large enterprise, a billion dollar plus of collected revenues and people. And we didn't have a single statement of claim filed against us as we went through it. So that told us a lot. The other thing is that it actually enabled us to have really positive conversations with our clients about what types of investments and changes should be made into the future.

Truth be told, we were able to go to our client base and enhance the services that we were providing to them.

Gordon Sheppard (39:26.631)
Yeah, mean, look at the alternative costs or opportunity costs, the opposite of opportunity costs, the negative costs, the negative impacts. So ransomware, had one ransomware been paid out? I don't know where they're at these days other than whole hostages of whatever we could pick company A and put it into that. the ransomwares are not cheap these days. And I actually heard from a consultant one time, said, those bad guys are reputable. Pay the ransom.

Jesse Hill (39:38.968)
Yeah. Yeah.

Jesse Hill (39:47.864)
No.

Gordon Sheppard (39:54.055)
because they'll, and they'll give you back your stuff. And it's like, my God, to think that there would have to be reputation amongst bad guys. But in the sense you didn't get that no statement of claim that's zero day is something I learned here today. I didn't understand that. No statement of claim is pretty strong reputationally to think that you got through with that. And again, it's maybe not something that you could guarantee in the future, but you could say we've got the best shot at it in a sense going forward. when someone's laying down to do that, plus you've got now, in addition to your regular

Jesse Hill (39:54.092)
Ha ha ha ha.

Jesse Hill (40:04.782)
Yeah, yeah.

Jesse Hill (40:09.39)
Mmm, that's good.

Gordon Sheppard (40:23.93)
services, you've got this recommended, but strongly recommended way to go in the future for their company to make sure that they are bulletproofing themselves as much as possible. Fair enough?

Jesse Hill (40:34.606)
Yeah, yeah. I thought of one more there that I would share and that is that, again, we ran the exercise, made a list of things to change. And one of the things that we changed was our insurance provider. Ultimately, through the simulation exercise in October, we've always had this gut feeling that they weren't quite understanding our business. And so in January of 2021, we switched insurance providers.

And as a result of that decision, the entire cost for us to do all of the recovery exercises and everything that we did were covered by our insurance. And so we actually didn't end up being out of pocket for that. Now that absolutely stunk for the new insurance carrier. We're still paying premiums with them today and they've...

I recommend them to everybody because of how they treated us. But again, we came out of it financially unscathed because of the decisions that we'd made in preparation.

Gordon Sheppard (41:46.919)
There's strong reputation moments all the way along here. So again, don't waste a good crisis, even the free, I heard there's a lawyer in the background there. You're like, I had speed dialed to this lawyer. We were ready. The lawyer was ready and understood. I had the insurance provider who understood while it was painful for them. We're still with them because you've built out partnerships, the collective partnerships and costs of prevention, which now people, it's dawning on people that way outweigh, the cost of an incident, right? So this is what you've simply,

you've proven it out and you're not even in the lab. Like you're out of the lab, you're in the wild and you're proving it out step by step. And again, it'd be fun to talk to you in two or three years, because you would have nuance-y things you've bolted in. You'd say, then this happened and we'd learned that thing. Then we did this proactive thing. Then there's all that iterative aspect to your world that would go on. qualitatively, man, it's a great story. Qualitatively, how's it feel to describe this story?

Jesse Hill (42:42.392)
Well, mean, it's obviously one of our big wins. know, again, I feel like we're in a place now where we feel comfortable talking about it and sharing with the world what we went through, how we got through it, what we learned. And you're right, I think that it positions us in a way that is unique.

Gordon Sheppard (43:06.994)
Well, and there's the feelings that come up around it and I'll just, you know, listening, I always recommend this. Do you have Libby on your phone? Libby is an app where, I'll just say this so everybody can hear it. Get Libby on your phone, L-I-B-B-Y, put your free library card in and you can get access to the library. I'm saying that for anybody, especially with kids. Why am I saying that? It's because I listen widely and I read widely through that app. And one of the things I came across recently was a Brene Brown book talk about daring greatly and the concept of shame.

Jesse Hill (43:21.709)
Mmm.

Gordon Sheppard (43:37.363)
and the feeling of shame and one of the things that maintains shame is silence. There's a lot of shame, I think, that goes into an incident like this because the company person said, well, I should have hired the right IT company to be responsible for this. I'm responsible for my people. And if you don't do what you're doing right now in this conversation, then that shame holds silence. And we keep it in that deep feeling so that we don't get a chance to react to it. And in a sense, to follow her logic, it gets into saying to be vulnerable around this thing. But you've taken us through all aspects of this.

and that we didn't even get into the technical solves that you would have done, which I'm sure are miraculous and thank you, it's not my area, but you talked about the leadership roles, the asking for help, the relationship building that you had done proactively, the proactive things that you had in there. I mean, that is just fantastic. Well, we could keep talking. It's so good to have you on here. Yeah, go ahead, you got one more thing, yeah.

Jesse Hill (44:25.486)
I want to share one thing I listened to a lot of things to and one of the podcasts I listened to how I built this with Guy Raz, he always wraps up and asks, how much of where you got was because of the hard work and your grit and your intelligence and how much of it was luck? And I do have to always remind people there's luck, there's unlucky in getting caught up in the thing. But one of the things that from our clients perspective was really lucky

for us was that the day after the incident happened on the Saturday, President Joe Biden delayed getting off of Air Force One for about an hour so that he could be briefed on the scope of this cyber incident. And the reason that that was so incredibly lucky and valuable for us was that to your point about, know, people could have said, you know, it was tier three that caused this, tier three made this decision, it's, you know, it's them.

We were able to, fairly early in the process, send our clients links to an article from CNN, basically saying, hey, this is something much bigger than any of us. And I often think back to that and I wonder how different the outcome would have been if it hadn't have been front page, if it hadn't have attracted that kind of attention.

And so yes, all of the things that we did, I think, contributed to the win, but we got lucky in a couple of things too.

Gordon Sheppard (46:00.307)
Yeah, and it's funny, I would put luck at about 5 % of what you've described, like in terms of what it is. So I appreciate that you got to have that moment in time and that enhancement. It's funny, I'm glad to hear about that recommendation about the last moment of someone else's show. I always like to say, if you had two or three sentences to say to a leader about dealing with a crisis, what would you leave behind with them?

Jesse Hill (46:05.39)
Yeah.

Jesse Hill (46:29.282)
Well, I think so much of what we've talked about is about preparing. And any opportunity you get to prepare you and your team, think is critical.

People overestimate how in sync they are with their team about how they would respond to a scenario. And so if you can find the time to stop and practice, or if you can make the time to stop and practice, highly recommend it.

Gordon Sheppard (47:02.35)
Make the time to stop and practice. And I'm also going to literally summarize from what you said. Stop and practice, build strong relationships over time. Have a multi-generational family business who's been dealing with the IT industry for 30 plus years. Have strong relationships with your clients to the point where we can go through a crisis together and still be together. Make sure you have the proactivity to find great dotted line to all your vendors and those people that are around you. Build a great team, take them through all that sort of stuff. So

Thank you so much for being lucky and being prepared. It was just so great to have you on the show.

Jesse Hill (47:37.038)
Yeah, I appreciate it. Thanks for having me.