Adam Levin:
So I don’t know if you guys have heard this, but there are some people who are tax counsel. They’re telling their clients that, “If you make a payment in connection with a ransomware attack to basically get your systems released, get your files decrypted, that it might be deductible.”
Beau Friedlander:
No.
Adam Levin:
Yeah. Apparently the IRS, so they say is considering it. What do you guys think about that?
Beau Friedlander:
I know. I think it’s terrible. The only thing I can think of that is maybe on that level of dumb is, I don’t know, making a chicken nugget smoothie. It’s stupid. Travis.
Travis Taylor:
Well, ransomware has become so widespread at this point that it’s almost going legitimate. There are companies that do a cybersecurity and a ransomware remediation, that they will actually pay the ransom on your behalf. So there’s sort of a functioning as the bag man for you. And they do the negotiation with the bad actors.
Beau Friedlander:
Yeah but Travis, they’re not making ransomware legitimate, they’re legitimizing ransomware maybe. There’s a business sector rising up around the prevalent, the ubiquity of these ransomware attacks. But Adam, I can’t think of any… That’s feeding the raccoons around your house. It’s just dumb.
Adam Levin:
Well, the other thing too is that insurance companies have reimbursed companies also when they’ve made ransomware payments. Depending upon what policy people have. Although, there are a number of insurance companies that are starting to pull away from that. So-
Beau Friedlander:
Well, [crosstalk 00:01:41] testimony to the fact that ransomware’s everywhere.
Adam Levin:
Now ransomware, as you said, it is ubiquitous. It is terrifying. It could be an extinction level event for certain organizations. And there are enormous amount of dollars spent by companies to recover from ransomware. But I also believe that they’re actually… the government should consider giving tax credits to organizations that do what they need to do to protect themselves against breaches and ransomware attacks.
Beau Friedlander:
A 100%, I think that’s an awesome idea.
Adam Levin:
I think that that’s where part of government focus should be, because we’re all in this together.
Beau Friedlander:
Yeah. But I also think, Adam, your idea is so good because there’s the one side you’re saying, “Well, we should get a tax credit for having gotten hit.” And the other side you’re saying, “We should get a tax credit from making ourselves harder to hit.” There’s a very big difference.
Adam Levin:
And I think the concept of getting a tax credit for making yourself harder to hit is important, because the ramifications of being hit can be devastating.
Beau Friedlander:
A 100%.
Adam Levin:
I’m Adam Levin, former director of the New Jersey Division of Consumer Affairs, founder of CyberScout and author of Swiped: How to Protect Yourself in a World Full of Scammers, Phishers and Identity Thieves.
Beau Friedlander:
And I Beau Friedlander cyber strange warrior of bad actors.
Travis Taylor:
And I’m Travis Taylor, resident tech guy. And can’t really follow up on a Beau after that one, so.
Adam Levin:
So today we have Dan Beeler on our show and he’s the CIO of Boyne Resorts. And he has a really intriguing story for us. And it’s a situation that has really been the bane of the existence of so many businesses and government agencies and individuals throughout the world. And so rather than give anything away, let’s get to know Dan a little bit better. First of all Dan, where do you live?
Dan Beeler
I actually just recently moved down to Panama City Beach, Florida. Our company is headquartered out of Boyne Falls, Michigan. So up until about three months ago, I lived up in Michigan. A little better weather.
Adam Levin:
Yeah. I have warm and squishy feelings by Michigan. I went to University of Michigan Law School. And so I did spent a lot of time in Ann Arbor, spent a lot of time in Michigan: is a beautiful, beautiful place.
Dan Beeler
It is.
Adam Levin:
So now tell us about your company.
Dan Beeler
So Boyne Resorts is an experienced company, where we have a lot of ski resorts and championship golf resorts in our portfolio. We have resorts in Maine, New Hampshire, Michigan, Utah, Montana, Washington and then British Columbia. And then we also have an attraction down in Gatlinburg called Gatlinburg Skylift and the SkyBridge. But the company’s been around for a very long time. It’s a really awesome place to work. And we just specialize in making great memories for our customers.
Beau Friedlander:
Adam and I had a little debate before this call. And if you make me right, I’m going to make five bucks. Do you offer zip lining anywhere in the summertime?
Dan Beeler
Of course we do. Yeah, that’s a big thing that we do. Yeah.
Beau Friedlander:
I’m a rich man.
Dan Beeler
We have lots of activities around everything from zip lining to Segway tours, to trout fishing, to just about anything you can imagine: our resorts and involved in it. Just we love the outdoors. We like our people to be out in the outdoors.
Beau Friedlander:
It’s fun. The only time I was ever on a Segway, I did a Mr. Bean move and I ended up sort of eating it. But they’re fun.
Adam Levin:
I have a nine year old, so we’re always looking for experiences. So we’re going to have to definitely check you guys out.
Dan Beeler
Yeah. See the people at our resorts and they go through a little training session on the Segways and then we have the sort of mountain ice ones with the big tires. And then you just see them zooming around the resorts. It’s pretty fun to watch
Beau Friedlander:
So Dan you do work with experiences, but I understand as the CIO of your company that you had a less pleasant experience. Not so fun. But first can you tell our listeners, what a CIO does?
Dan Beeler
Yes. That’s a great question. I would say that for our company, I liaise a lot with our business. I’m responsible for all of the technology in our organization.
Beau Friedlander:
So you’re the cybersecurity guy.
Dan Beeler
I am. We’re a fairly small company comparatively, so we don’t have a specific CSO. So in my job, I am responsible for that. I also work with the business in a lot of capacities. My job is to sort of take technology and help the business leverage technology, so we can be more competitive, we can get some economy of scale. I’m also responsible for security or at least the people on my team are.
Beau Friedlander:
And you had a security event.
Dan Beeler
We did. We had a very big security event. So October of 2020, we got hit with some malware. One of the guys from our resorts, one of our resorts in Michigan called me. It was about 3:30, my family and I were out hiking the Grand Canyon and called me and said, “Hey, we’re having something going on.” He was notified by our night auditor that when she was logged in and trying to do work, icons on her computer were starting to change. And she was starting seeing things pop up and all the files were started changing to an easy to lock dot extension.
Dan Beeler
So we got everybody involved. I called all my vice-presidents and all the technical people. We jumped on a quick call. We quickly identify this as malware, as you can imagine. We have an incident response plan. We started working on the incident response plan. We soon realized that the gravity of this event was much greater than our incident response plan. So we had to slow things. So we had to sort of stop where we’re at, had to reevaluate what we’re doing. And then as it basically included the executive team and made some decisions immediately. We started shutting down all this stuff that we could at each of the resorts. Our resources are sort of decentralized. So we have everything based out of Michigan, but each resort has its own technical stack. So we basically shut down everything at all the resorts and then started working on a plan to start recovery. But we were hit by Evil Corp.
Beau Friedlander:
Evil Corp. So they ask for…
Speaker 6:
$1 million.
Dan Beeler
Just went sharks with lasers on their hit. They actually did ask for a million dollars. They wanted a 90 Bitcoin. So at that time, Bitcoin was roughly what it equated to at least shy of a million dollars.
Beau Friedlander:
Wow. That’s Dr. Evil money.
Dan Beeler
It is.
Adam Levin:
Dan to back it up, once you started doing your analysis, I’m sure you also brought in probably a third party forensic team to take a look at this. Did they figure out how it started? Was there a… What was the root of this whole problem?
Dan Beeler
Yeah. It’s really, really crazy that it happened like this. So we did engage with a CRA, which is Charles River Associates. They work in this forensic space, sort of help you come in and figure out what happened. We started working through everything there. And we were able to go back and we realized that they had been in our system since September 29th. The event happened on October 18th. So they had sort of set into our system, ideally for around three weeks. Not ideally, I should say, without manifesting themselves. What happened is our resorts are decentralized. We had a user at a resort who had unnecessary privileges. She had local admin on her machine. And ultimately comes down to one of our administrators out there being lazy and not trying to figure out what she needed to actually do her job. But took the easy way out and gave her local admin.
Dan Beeler
She was out doing a research on COVID law. She was at covidlaw.com I think. And she clicked a upgrade for Chrome. So she wanted to upgrade the version of Chrome. She clicked on that since she had local admin to the machine, it installed a hidden payload, included MimiKatz. And once that was installed, they had access to that machine and they started harvesting credentials. And then they just jumped around our organization. So we had taken some safeguards to sort of… We had an air gap our backups, but they used some pretty interesting techniques to get that data. I can go into detail if you want, but it’s… They were just willing to wait. And they waited for roughly three weeks until they got access to our backup servers, because this would have not… If they would have encrypted everything, it would’ve done the no good.
Beau Friedlander:
And what do you think they were waiting for?
Dan Beeler
So I think what they’re waiting for is another… So that user ended up having to have an admin assistant and some stuff. So when that admin logged back in they had MimiKatz installed, they were able to grab the credentials from that. And then they were able to navigate over to our air gap to backup solution. And they weren’t able to do anything with the solution that we had in place. What they ended up doing was actually formatting the disk array. So they ran a format on the disk array so we can not get any of that data back. Because there’s ways to get that data back with your backup solutions. However, in this case, they formatted it. We actually sent the arrays out and we could not recover any of the data. So they didn’t do a basic format, they did sort of an industrial grade format of this disk.
Adam Levin:
Hey there canvass. Look, if you have a story about being a victim of a hack, we’d love to hear about it. Give us a call at 6232521828. That’s 6232521828 or email stories@WhattheHackpod.com.
Beau Friedlander:
Does Travis have any questions?
Travis Taylor:
Yeah, just going back to MimiKatz, if I recall correctly, that’s a type of malware that gathers credentials. Is that accurate?
Dan Beeler
Yeah. I don’t know if you would call it malware, it’s a piece of software somebody wrote 20 years ago to sort of show vulnerabilities with Microsoft. And it’s been used in hacking toolkits for last whatever… Actually, of course you know what happen to us. It’s not something I usually go play with, but when it happened to us, I went out and did a little bit of research and there’s a whole market out there for it. And there have different versions for depending on which versions of Windows you’re running. And these guys were pretty astute and they were able to get it installed. Once they had it installed, they started crawling our network, created a bunch of fake accounts and they did the stuff that all these guys do.
Travis Taylor:
Yeah. So it’s a pentesting tool, but it’s one that if you want to use it for criminal ends, I guess it’ll be pretty effective.
Dan Beeler
And it was complicated too or exacerbated, I should say. Because we didn’t have some of the general tools that you might have in some of these other places. I came from a background working for some very large companies. Where we had some of these… We had an EDR and some tools on the outside. We didn’t have those here. Again, one of the things I think is unique about how these guys operate is they take the lowest hanging fruit there is. In this case, they were able to go to a company where we didn’t have an active EDR or MDR and that really slows our response. And it makes it easier to get… Once you get into the system, it’s very easy for them to jump around and to exploit us. And I think if we would have had some of the solutions in place, which we do now, it would have been a little bit more challenging for them.
Travis Taylor:
And you said that this was a WastedLocker, right?
Dan Beeler
Correct. Yeah.
Travis Taylor:
Okay. One thing that I’ve heard before about WastedLocker, that I wouldn’t say that you got lucky, but a WastedLocker is known for being just a single extortion ransomware. So did you ever have any kind of followup threatening to sell your data or anything?
Dan Beeler
That escalated a little bit as it went on, but we never… Our research was the same in that these guys were basically just in encryption. They come in and do what they do, and they don’t take any data ex-filtration. We did have a sim and we didn’t see any data ex-filtration. And the company that we’re working with was listening on the dark net and they didn’t see anything of our data ex-filtration either. In that regard, it was good. The bad thing about these guys are is I think we would have… We did consider paying the ransom. Now we have cyber insurance is one of those things we talked about, but these guys were on the OFAC, the Office Foreign Assets and Control.
Dan Beeler
Basically, these guys are sort of state sponsored. And if you do business with them, we would have gotten sanctioned by the government. So it was no longer an option. It’s crazy, crazy story too, because we originally thought this was a BitPaymer malware attack and we thought, “Okay, cool. We can… Not that we would want to, but we can recover quickly.” We’d actually changed… We engaged with them, we gave them a couple… We asked for proof that they had the decrypter, we’d done it on a couple pieces of a couple of files and proved that it worked. We were within two hours of doing the transfer. And one of my guys actually looked at it and said, “I’m pretty sure this is not a BitPaymer, this is a different variant.” We did some more research. We engage with the folks at CRA and there’s, “Yeah, you know what? It looks like you’re right.” Ended up being WastedLocker and Evil Corp.
Travis Taylor:
And the BitPaymer was also from Evil Corp.
Dan Beeler
It was, I believe it was. Yeah. But I think what happens is these guys matured. I read an article recently that said, these guys are actually using tools to sort of cover up that it’s them now. And they’re actually pretending to be other players in the space because they’re on the OFAC, and it’s not legal for American companies to pay them.
Adam Levin:
Yeah. It’s always good to pin it on the other guy.
Dan Beeler
Yeah, exactly. Especially in this case.
Adam Levin:
So how did it ultimately resolve itself for you? How’d you get the data back?
Dan Beeler
So this is a good question. We didn’t really get the data back. We had been in the process of living a bunch of our stuff to the cloud. We’re in the Azure cloud space. And we had been working on that. We ended up going back to a copy of data that we had in the cloud and my amazing team and the amazing folks I work with, basically spent the next five weeks rebuilding all of that information to get us back and running. The thing that happened is this event happened in mid October, we really start hitting our stride as a company for skiing and we start opening back up in late November around Thanksgiving. We get super busy in December, January, February, as you can imagine. This event happened probably at the most opportune time for us. There’s never a great time to have your business taken away, but this was sort of one of the better times that could have happened.
Dan Beeler
So we had a five week runway to rebuild our ski solutions, to rebuild our payroll solution, to rebuild everything. And we did. Like I said, my team did a great job. And we did have some stuff in the cloud that we could recover. We used some of the tools out there from some of our other partners to sort of roll back and get the encryption out of the environment. And we rebuilt everything from the ground up, all new Ada environments. We didn’t want to risk having any of that stuff in place. So we took a lot of steps to make sure that we were not going to everything’s back on to still have the bad guys in our network.
Adam Levin:
Have we heard from them again?
Dan Beeler
We have not. So the sort of the way this sort of works is, at least in our case, we engage with a legal company. They helped us. They put us in touch with the folks of the forensics team and the forensics team at CRA handled most of the work when it came to interacting with the bad guys. We didn’t really do that. We have a conversation, they eventually lower the request down to 38 Bitcoin. They started at 90. So roughly 40% of what they were asking before. Just because they wanted to get paid. Again, these guys are all about the volume. Travis was right. They’re not really extorting you for credit cards. At least at that time, they weren’t. They were just coming to do… We talked to the FBI, they said they do between four to seven a week. And that’s just about numbers for them.
Beau Friedlander:
Nowadays, I think we’re seeing more ransomware attacks where it’s kind of a hunter that uses every part of the animal. They’re ex-filtrating data. They’re selling it, they’re using it. They’re keeping things. They’re giving you back data after they’ve already used it. Are you hearing about that out in the wild?
Dan Beeler
We are, yeah. That’s exactly. Even these guys, a lot of the guys used to just do the encrypt and hold your data ransom, have migrated into additional facets of that attacks scheme. So they’re doing… They’re holding your data. They’re also selling your credit cards. Any PII they have, they’re selling that. So it’s definitely evolving.
Adam Levin:
Now we’ve even seen cases where they not only sell the data, but then they go to the people whose data it is. When there’s consumer data or employee data involved and they say, “What’s it worth to you now for us not to sell your data?” Even though they’ve already sold the data, they’re going to the individual. There was a case of some plastic surgery practices. Where first they hit the practice. Then they went to the patients and said, “How would you like your before and after pictures distributed on the web?” So there is… It’s sort of, “Have you no shame?” They have no shame. It’s basically all about the money and it’s about how they can be as efficient as possible in extorting as much as possible, as quickly as possible.
Beau Friedlander:
Well, yeah. It’s turning into… The ransomware as a service is really blossomed into a full, fully formed multi silo business that they’re operating. And so it’s… Do you agree with that from what you’re seeing Dan?
Dan Beeler
Absolutely. Yeah. And that’s what we heard too from the folks that we spoke with at the FBI and from CRA: it’s all about a service now. Anybody with a little bit of technical knowledge can go out and do this extorted stuff with this service. Especially if your targets are small. If you’re going after cosmetic surgery places or you’re going after the little mom and pop stores whose data’s important. But you can go in there and hit those places pretty easily because they don’t have a lot of defenses.
Adam Levin:
And they will pay. Most of them will pay too.
Dan Beeler
They almost have to.
Adam Levin:
Yeah. So what’s been the learning experience from this? And I know it was a painful, but how do you feel you’ve learned from this?
Dan Beeler
Great question. If you don’t learn something from this, I think you really missed an opportunity. For us, we take security a lot more seriously now. Not that we didn’t before. Like I said, I came from some places where we had dedicated teams to do that. Being a smaller company, it’s a little harder to dedicate an FTE around that. Subsequently, we’ve got a full-time person doing security. We actually have a couple people doing security. We participate in a lot of round tables and sharing of data. These types of calls for me are super important because I want to make sure everyone out there doesn’t get caught in the same situation we were.
Dan Beeler
Additionally, we’ve taken a lot of steps afterwards. And just as an organization, we put in a new XDR solution. We use MFA for everyone. We got around some of these things by not doing some of the… We’re trying to save a little bit of money by not doing all these things, that is out the window. Now, everything is… Anything we can do to sort of decrease the threat surface we do. As long as it’s not exorbitantly expensive, but everything from data leaks to share data. We have a sock that helps us identify threats and to address them as quickly as possible. We’ve expanded our incident response plan. So our incident response plan is no longer… I built it when I first started here. I’ve been here for about three and a half years, and it just didn’t encompass everything that we saw happen to us in the malware event. So it’s a much more sophisticated now. We do a lot of tabletop exercises around how do we recover and what can we do in that space?
Adam Levin:
No. And that’s very important. Because just as the sophistication of the hackers evolves and the sophistication of the malware evolves, so that each time somebody figures out a solution they tweak it just one more way in order to evade anyone discovering them. The sophistication of the breach response plans have to also evolve, because unfortunately every day is a new adventure. And a lot of companies I think, have also learned that you could be totally secure at 9:00 AM and at 9:01, somebody clicks on the wrong link or somebody gets access to credentials that would allow them to get deeper and deeper into the company. That’s all you need and then you’re there. And it’s a nightmare. And without question, the call that you received while you were in the Grand Canyon, it made you feel you were falling into the Grand Canyon. And it’s a terrifying call to receive. I know myself with the companies that I’ve been involved with over the years, the one thing nobody ever wanted to get was that phone call. Ever. And it usually came in the middle of the night.
Dan Beeler
Yeah. It’s an awful feeling. I’ve been doing this for almost 30 years and-
Beau Friedlander:
I have a question for you Dan. Are you conducting any kind of anti-phishing work at your company? Do you test your employees periodically and try to phish them and see if they fall for it?
Dan Beeler
We do. So we put that in after we had our events. So we use a tool now and we phish them at least a couple of times a quarter, when they have to go through training when they start. And the people who continually get phished have to go through remedial training.
Beau Friedlander:
Yeah. How is the phishing? Has it been pretty good catching?
Dan Beeler
It’s been pretty good. Yeah, unfortunately. This is not the kind of phishing that you want to be good, but it’s been pretty good.
Adam Levin:
Now, often they say that… Someone will say, “Great news. I’ve got it to the point now where 93% of our people don’t fall for it.” The answer is all you need is one.
Dan Beeler
Yeah. We have to be right all the time. They just have to get lucky.
Adam Levin:
That’s it, that’s it. And that’s the tough part for any defender. Is how can you be right all the time? No one, other than my mother is right all the time. So therefore.
Dan Beeler
We sort of feel that way. Hopefully we don’t have an attack at the gravity we had the last time, but I don’t know that you can say that we’re never going to get attacked again. And like I said, we’ve done a lot of things to make that more difficult. But in reality the right approach to this is to keep them away as long as you can, restrict what happens when they get into your environment and recover as quickly as possible. So we’ve taken a lot of steps in that space to make that happen. We use immutable backups now. We have a lot of things in place now if and when they get into our environment, we’ve used a lot more segmentation. We have some east west firewalls. We do a lot of things to prevent them from jumping around an organization, sort of a submarine. Where you can cordon off an area of the sub if you get hit by a torpedo. That part’s damage, but the rest of it’s still intact and you’re still be able to move forward. That’s sort of the approach we’re taking right now.
Adam Levin:
Well, and you have to compartmentalize things because the bottom line is, breaches have become the third certainty in life behind death and taxes, unfortunately.
Dan Beeler
Absolutely. If I had one thing to say, I would just say, make sure that you’re looking at your plan continually. Making sure that you have the right story to the executives in your company or to your leadership to make sure you get the right funding you need. And there’s a good story to be told there, sort of like insurance, people never want to buy it until they have an accident. And then they were, “Oh, I wish I had that.” That’s how security is. It’s the same adage we’ve heard a million times, but it’s real important.
Travis Taylor:
A study came out recently finding that 80% of ransomware victims have ended up getting hit again. So it sounds like you’re doing all the right things in terms of really just amping up your defenses against that.
Dan Beeler
Yeah. Like I said, we’re doing as much as we can to defend. But if something does happen, our plan is to have a really quick recovery response to our… Recovery time objectives are pretty aggressive because we do a lot of online business. So it’s important to have this, our systems up and available. So we’re doing our best to make sure we have that. But we’ve put a lot of things into that if something does happen, we can recover quickly.
Adam Levin:
Well, it sounds like you’re doing a great job. And I fully understand exactly the nightmare that you went through. I have total empathy for what you went through. I, and I think everyone here would like to congratulate you on the work that you’re doing and the effort that you put in. And more importantly, the fact that you’re willing to share your story so that other people can not go through the nightmare that you had to go through.
Dan Beeler
Well, I appreciate you. This is a great vehicle. I love the idea of what you guys are doing. So people understand what’s going on out there, understanding the risk. Talking about it is as important as anything. Commend you guys on that as well.
Beau Friedlander:
First step is admitting you have a problem, right?
Dan Beeler
It is, yeah.
Beau Friedlander:
Well round of applause man. Thank you so much for joining us. This really was eye-opening.
Dan Beeler
My pleasure.
Adam Levin:
Take care now.
Dan Beeler
Thank you.
Adam Levin:
Thank you.
Travis Taylor:
What the Hack is a Loud Tree Media production in partnership with Larj Media, that’s L-A-R-J Media. You can find What the Hack, wherever you get your podcasts. Be sure to follow us on social media and find more information @AdamLevin.com.
Speaker 7:
Loud Tree.
