Well, here we are at the eve of Memorial day weekend. So we’re delighted to have all of you guys joining us. Our theme for today is all about automation. And the title of today’s TECH Talk was good, better, and best, and really where are you in terms of using automation to help harden or shore up your security posture.
And as I was preparing for this, because Shahin always throws these topics at me, or we talk about them and then I have to go spend a bunch of time learning to catch up with him. All of the materials I was reading talked about automation as it relates to investigation response, or investigation and containment, or alert enhancement or enrichment. And so there’s a lot of automation going on on the defensive side or on the responsive side, I guess. So after an event has happened. And we really wanted to talk to you about where the technology is today, where the tools are today and really where we believe there’s a missed opportunity happening.
And so for those of you who might be using one of the 3000 tools or many of the 3000 tools out there, who might even have deployed some automation, we really want to help you start thinking about where you are in terms of that good, better, and best, and give you some things to think about in terms of how automation might be helping you even more.
So Shahin, I’ll turn it over to you. And why don’t you talk to us a little bit about where automation is today and is it really only effective on that responsive side? Or is there just something we’re missing?
When you think about automation, it’s all about repetitive actions. So what is a repetitive action? A repetitive action is something gets triggered, that trigger initiates a set of steps that do stuff. And those steps that do stuff have to be something that is consistent and repeating and recurring. And that makes the world of automation smaller and smaller and smaller because as time goes on, as the adversaries evolve and they create better and better attacks. And they figure out how to get around the tools and technologies that we’ve put in place, the controls that we’ve put in place, the things they do, the triggers, change.
And so we have to constantly be adjusting what triggers we’re looking for, and we have to constantly be adjusting what actions we take when those triggers happen. So when we started talking about this I got a little heretical and this is funny coming out of me, because anybody who’s worked for me over the last 30 years knows that the very first thing I talk about is how do we automate this? I don’t want to ever do it again. I want to do it once and never touch it again.
And I’ve built my career on how to get as much efficiency through automation, whether that be DevOps, DevSecOps, or whatever new acronym we’re going to throw at this automation space, it’s not new, we just keep coming up with new words that talk about the way we do things.
So the heretical side of what I’m going to say is that all of the tools that we invest in, all of the technologies that we invest in to protect ourselves are reactive in nature. And the problem with that is they don’t do something until something happens. What’s wrong with that? That sounds like a great thing, right? That’s beautiful. There’s a bad guy doing something and I’m going to stop him. Sounds fantastic.
But the bad guy does not do something. The attack does not start the day that the first action is noticed. The attack started six months before that. The hacker’s been lurking inside your network for six months investigating, doing due diligence, doing all their reconnaissance, figuring out where the crown jewels are, figuring out where the best place to install software, the best accounts to compromise.
That effort of reconnaissance takes months. And on average, it takes 200 days across organizations, according to the industry. So if they’re in there for 200 days and they don’t pull the trigger until day 200, yes, it’s fantastic that we have automated tools that respond when they pull the trigger, but wouldn’t it be so much better if we caught them in two minutes, not 200 days. And that’s really the focus of is automation really working.
That’s what generated this conversation. And the heretical talk is we keep hearing everybody’s AI focused. Everybody’s doing all kinds of great things. Everybody has autonomous responses. Those are all good things and you don’t want to do without them, but is it enough?
Well, and that’s what you see in the industry. Anything you read, any reports you’re looking at among all of these security tools everyone is promoting, hey, we’ve got some element of machine learning. We have some element of automation, which in theory is fantastic, but as you bring your tools together, how does that work across each tool? And as you work across each tool, those are more alerts you’re getting, that’s more information you have to manage.
And so even just on the responsive side or reactive side, you are creating a lot of work with the automation that someone’s got to figure out. If you’re not watching it, if you’re not correlating it, then all of that automation and all of the things the automation is telling you might not be heard.
And the problem with anomaly detection is there are a lot of anomaly tools out there that try to sniff out and look at behaviors that are anomalous from the norm. And then they use augmented intelligence, machine learning, whatever else to try to identify what to do with that augmented data or anomaly data. The problem is the hackers get smarter and smarter and make their anomalies look more and more like human behavior.
The issue is those human behaviors are clearly not human behaviors when you have a seasoned analyst looking at the logs and figuring out, wait a minute, this doesn’t make sense that they did this right after this. And anomaly detection is only as good as the learning models we give it, because they’re all based on machine learning. And so when you’re based on machine learning, you have to teach the machine how to identify an anomaly.
How does that happen? It’s the same way all of our traditional security tools and controls have happened. We give it a set of examples that this looks like an anomaly, but when we really sit back and look at it, those examples are already happened examples, things that happened in the past. So those are behaviors and activities and steps that are repeatable if the hacker does those same things again.
What we know in the security space is they typically don’t, unless they’re a script kiddie who bought a kit. And in that case, there’s not a lot of risk there because they’re not sophisticated, they’re running automated tools to attack. Your automated tools are going to do fine. They’ll stop them. They’ll still be in your network, they’ll still be doing reconnaissance. Targeted attacks are where it really becomes a problem.
I was listening to some peers in the industry and they talked about three types of attack, the hit and run, the hit and stay, and the hit and steal, if you will. So the hit and run is ransomware. You’re coming in, you’re doing a hit, you encrypt everything and you drop your ransom and their attack is over. That attack, by the way, still took anywhere up to 200 days inside your network to figure out. The hit and stay is they’ve done something, they’re in your network, they’ve been there, but they’re waiting for the best opportunity. They’re waiting until they find something unique.
And these attacks can last year’s or these infestations can last years. So wouldn’t the focus of security be better served if you were looking for people who are sitting in your network and those hit and stays are not script kiddies. Those hit and stays are intelligent people who are looking for an opportunity to make a large sum of money, not 20, 30, $50,000, or even the ransoms that get up to even 500,000. These guys are looking for the millions of dollars, the big hits.
Then the extortion type, which is steal your information, the extraction data. That is what effectively impacts your business system significantly. You now have the risk of you’ve lost customer data so customer trust goes away. You’ve lost IP so your competitors can now compete against you. And you fill in that blank line. You’ve lost, blah, and the impact is blah in your industry.
And that is all tied to intelligence staff, intelligent security staff or nation states that are doing stuff. There’s a lot of recent attacks that have been nation states doing things and their focus was – like China’s been involved in a few recent well popularized attacks and their focus was we’re going to come in through one of your supply chain, get into your network, and from there, we’re going to sit and monitor and learn and then do something. And then they do.
The Capital One breach was a perfect example of that. It came in through the supply chain. It stole one of the largest numbers of personal records and credit card records. And it was published out to the internet so that all the script kiddies can go off and use your credit cards and do things. And imagine if you were Capital One, you just now lost everybody’s credit cards that trusted you.
And translate that to your own business, and that’s where we have to get to the point where the trigger is not the event. The time of event is not the event. We have to find the adversary in the 200 days that are there sitting dormant, doing tiny, tiny evaluations of what your environment is to avoid detection. We often say, and we’ve been getting more and more salesy at these TECH Talks, Kirstin, but we often say that what we do is we identify the adversaries in your network in six minutes, not six months.
The reason behind that is we not only rely on the automated tools and the autonomous responses and the security orchestration and all those things that we talk about, we also take into account the fact that there’s tons of data on your network from packet captures to flow data, to the log data from all of your routers, switches, firewalls, your active directory information, your DNS information, your endpoint telemetry. And we correlate all that along with your vulnerabilities to determine what is a baseline in your network. And when something starts to stray from that baseline, we’re able to identify it very quickly. And that is the security that you ought to be focused on.
In this last comment that you made, we were kind of getting to better or best. So as we talked earlier and, and those of you who’ve implemented tools, security tools, and who have some degree of automation to help with the back end of an attacker getting in, good. You want to have that. You need to have that. And you’ve done well to do that.
So Shahin, as we talk about better and best, when I hear you talk through the orchestration and the intelligence and the people, if I’m an organization who is out there, I don’t know, I’m a construction firm, I’m a law firm. Whatever business I’m in and I start hearing intelligence, security people, I start thinking about, wow, that’s cutting into my revenue, that’s cutting into my budget.
And obviously, we’re a managed security services organization, so we can talk about that. But let’s keep it right now at if you’re an organization trying to do this yourself, in my mind I say, is this even achievable? I know there are global organizations who have hundreds of security people who have over 140 tools, there was one we talked to this week. They are still uncertain about what they’re doing. They still think that they have not – they’re somewhere in the better, but they’re not near best. So how is an organization to think about this?
So if we take that evolution of good, better, best then talk about it, like you said, good is go out and buy solid security controls, good EDR solution, good email security solution. And implement tools that are detecting something bad and stopping them. So, number one. That’s good. That’s the minimum. That’s the barrier to entry, cross that barrier. So what is better? Better is you now need to also collect the telemetry from your firewalls, from your end point tools, from whatever, and start correlating them. So that translates to a SIEM, got to have a SIEM.
And the reason that’s better is because when the attack happens, you have to go back and do an investigation to see where it started so that you can eradicate it and stop it. Otherwise, every time you whack the mole down, it’s going to pop up somewhere else. And so having the SIEM gives you the ability to go back and do forensics. So forensics helps you identify how it started, where it started, and what things do we need to do from a response perspective to eradicate this infestation.
Best is having a SIEM without a SOC is like having a guard tower without a guard. You’ve all heard me say this repeatedly. It’s pointless. It’s really just a responsive tool and your entire security portfolio rather than being proactive became responsive. So the SOC is what makes you proactive. People sitting in seats, looking at those logs, identifying anomalies, investigating those anomalies and doing threat hunts. So that is the end state.
Is it achievable? Of course it’s achievable. The problem is it’s expensive, like you said, and it’s going to come out of capital budgets and recurring operating budgets. We often do TCOs for our customers to compare what would it take for you to build this versus consume the services we offer? And on average, we’re running 50 to 60% cheaper than it would be to build it. So absolutely anybody can go and build what we’ve done. We’re not rocket scientists here. We have an awful lot of experience.
I’ve been a CISO for 30 years and I bring that experience to our customers. And my team, on average, has about 20 years of experience in this space. So we bring that level of depth and experience to the table to build something great, but it’s not magic. Nothing we do is magic. And I hate the words common sense, but it’s leveraging the common sense of IT security to implement the right controls, the right procedures, and the right people to be able to identify, investigate, and respond to events before they become the trigger point of an attack.
Sure. Well, and to be fair, it’s our core business. And it’s not the construction firm’s core business. It’s not the law firm’s core business. And so clearly we’ve got an edge there than someone building it themselves, because if you’re going to make that your core business, that’s going to compete with your legal clients or whatever.
What I heard you say when you talked about good, better, best, better with the SIEM. We went to best and said, well, if you don’t have a SOC, the SIEM doesn’t matter. It almost sounds like you’re saying you either have the option of good or best, because if you invest in better, but you can’t operationalize better, better doesn’t matter. Did I hear you right?
Sort of. The SIEM does help. It is incrementally better than not having the SIEM. Let me walk you through a scenario. If you get attacked, and the attack that you notice, if you don’t have the proactive controls we just talked about, is the moment of trigger, the moment the hackers started to do something. Your EDR tool did its darndest, maybe even succeeded in blocking it from encrypting your end point. And blocked it from having some malicious software run on your end point.
But how did that malicious software get to that end point? And if you clean up that malicious software, are you comfortable that it’s not going to come back to that end point again? And I gave an example a few weeks ago on one of these TECH Talks about an incident response we did where the ransomware spread quickly to 700 machines. Our EDR solution stopped it, blocked it, we were able to recover and get them back up and functioning.
And that happened at midnight, by the way. Who’s watching the shop at midnight is the other thing to talk about. Within five minutes, 700 systems were infected. It laterally moved to 700 systems. Within about a half hour we had everything stopped and began the rollback process. And by noon the next day, everything was recovered. But that was on Saturday it started, Saturday at midnight. Sunday at noon it was recovered. Monday at 10:00 AM it started again. And it started spreading all over again.
And at that point we realized it’s a system that isn’t under protection. Otherwise it would have been caught and captured and cleaned. Fortunately, the customer had a SIEM. They were not on our SOC service, but they had their own SIEM. So we were able to log on to the SIEM and identify where this was coming from. The hacker had been in their environment for four months, compromised an active directory credential and used that credential to compromise the domain admin credential. They captured the hash from that domain admin, and they used that to move laterally.
So even if they change the domain admin’s password after it was compromised, it wouldn’t matter because they have the hash and that Kerberos key will allow them to continue to use that domain admin account. So no matter how many times we would whack those moles, they would’ve kept going back and forth. So we were able to completely recover and help them recover and get up and functional and get everything going.
But it was – they didn’t have to pay ransomware. They didn’t have to build huge infrastructure in parallel to decrypt. Nothing got encrypted. They only had to restore one server. So all positive. It sounds good. We stopped it, but it would have been even better if we stopped it four months prior.
Exactly.
And that’s the difference between best and better. Without the SIEM we wouldn’t have known the linux box that was compromised and be able to clean up that linux box. That would have been literally finding the needle in the haystack. So with the SIEM we were able to identify where it came from, so that is better than just the end point solution. And then the final layer is best is not just – and we have a five layer security model that we talk about.
Email is your number one place where inbound attacks come from. So 93% of all attacks start with email. Email protection is critical. 80% of malware that gets on the end points needs to have DNS to function well. So you saw the DNS connection issue where malware can’t connect to command and control, and now you’ve blocked 80% of malware. So now you’re dealing with only 20% of the malware that gets to the endpoint.
Then the EDR solution is where that comes in. That 20% is going to do something, so now you’re reactive. But the first two steps were proactive, let’s prevent it from happening to begin with. A couple of other factors are 50% of the companies out there have a ransomware attack. Out of those 50%, three out of four are successfully encrypted. Out of the 50% really, 80% of those companies are attacked again.
It’s not like they hit you once and move on. They realize, hey, this is an easy target, we’re going to do this again. So it’s important not to just fall back on automation. And that’s really the topic of this, you can’t get comfortable that my EDR tool is one of the best in the market and it will stop the attack. It doesn’t always because the hackers keep getting smarter and smarter, and they change the way they do things. A lot of these tools leverage the MITRE ATT&CK matrix as the foundation for what an attack looks like.
When we started our security business, there was only 100 controls in the MITRE ATT&CK matrix. I think it’s up to about 160 controls now or tactics and techniques. So they keep finding new tactics and techniques and procedures that hackers are using and they add them, so that database keeps growing. And the real question is great that you use MITRE, but how often are you updating the ttps for MITRE? And are you looking at vulnerabilities with the CVEs that also come from MITRE? They curate the CVEs which are the common vulnerabilities in the environment.
So good is be able to block and tackle. Better is be able to identify and do some investigation about what happened, where it started and all that, so that you can clean up. Best is catch them before they do anything. And really the only way that happens is a tremendous amount of telemetry and people looking at the alerts and events and anomalies that are happening.
Yeah. Do you see, as we are out there talking to customers and just watching the market – and we’ve talked about this before, right, is there a level where a customer or organization needs to start investing in this? If you are smaller or if you’re in a certain industry and you say, well, you know what, I’m good. I’m good with good. That’s all I need. Or really is this something that everyone needs to be thinking about?
Everyone needs to be thinking about this. We have – before we started our security services, this is five and a half years ago now, we had six customers in six months get compromised with ransomware. And those six customers ranged in size from about 50 employees to about 3000 employees. So people think I’m not in an industry that’s targeted. They don’t care about us. It’s not true.
There’s three approaches, the hit and run players are doing shotgun blast across everybody. There’s databases on the dark web with millions. So the issue that we have with compromised accounts from Capital One, from Target, from whatever, all of those databases are on the dark web. These hackers go in and grab them. And those databases include the passwords associated with an email address.
So they go and they pick abc.com and all the passwords, and they attempt to log in to your websites with that username and that password. And guess what? Your users probably use the same password they did at Target that they do for a lot of your systems, because you don’t have a password management system in place. And so they get in, that’s how they start. And from there, they start infiltrating your environment.
The other option is inbound through email, which is again 93% of the attacks start with email, because as much as we love to rely on our employees for that first level of defense, everybody makes a mistake and clicks on the link. I just got a really good phishing attempt from PayPal. They said, we just – and I was looking at, can’t even remember what the service is now, but I was looking at this service and it said – oh, it was a Bitcoin thing.
I was looking at Coinbase and I was evaluating something in Coinbase. And it said, we just transferred $275 to your Coinbase account from PayPal. And I was like, whoa, did I click something wrong? And here I am supposed to be an expert in this. And then I slowed myself down and I said, I wouldn’t click something wrong. And I started looking and the email address was a Gmail account. The link did not go to PayPal.
It was [inaudible 00:26:19]. And of course me being me, I clicked through a couple of things and sure enough, the landing page looked just like the PayPal login page and it was a very well done attack. And they didn’t say $100,000, they said 275, which was reasonable. They’re getting better at phishing. And they’re getting better at phishing and our people are getting fooled by phishing.
So we believe that email security is a broken space. We take a unique approach to protecting users from phishing by crawling every Office 365 inbox and extracting the threats before they ever see them. Most phishing solutions which are good, are gateway solutions, which only deal with inbound and outbound messages. And most of them, the companies don’t turn on outbound scanning, they only focus on inbound scanning.
Sounds like maybe a topic for another TECH Talk.
Oh, yeah, absolutely. But it’s that five layered approach, email, [inaudible 00:27:23] pinpoint, and then network security is one of those things we’ve talked on in multiple TECH Talks. We talked about our ZTN offering and micro-segmentation. Segmentation is probably one of the oldest and most broken facets of network security in the IT industry. It’s very difficult to do unless you come to us. And we have a – it is the key thing that will prevent lateral movement. And it’s the way to stop an attack from getting its foothold in as many systems and causing as much grief.
So I would say it is possible, to answer your question, for a company to do these things, but they have to take advantage of a lot of tools, a lot of [inaudible 00:28:11] and have security analysts, and it becomes very expensive. And the bigger challenge is if you do it yourself, the security talent is hard to find, and it’s a turnstile. That becomes a problem that you leave to a company like us, where it’s our job to recruit, retain, and continue to fill those seats. So it becomes very expensive to constantly recruit and it’s exhausting, honestly, for a security administrator and manager.
Well, and it really comes down to where do you want your investment dollars to go? And do you want to build this out and is that for some reasons strategic to you? Or is it just very important to you, but having someone whose business it is – well, you have a time advantage too. If you’re going to build it, there’s a lot of time that goes into building it versus consuming it. So that’s obviously another factor to think about.
So I would say, to summarize what we’ve talked about, because I went all over the place. Autonomous response is not all it’s baked up to be. It’s really a solid tool in your toolbox, but it cannot be what you rely on as the answer or the savior from a security perspective.
Right. Well, and I will invite our listeners as we always do, we’ve got the spectrum, good, better, best. And as you think about your security posture, as you think about the things that might keep you up at night or the things that might make you nervous, we are happy to talk to you. We would love to have a conversation with you. We do have the complimentary security health check, easy for me to say, that will help you determine where you are, good, better, best, and what it might take to get you where you feel your business needs to be.
Those things that are important to you to make sure whether it’s investors or supply chain or whatever, that you know, that you have heat and lights on that you know need to be secure. So as always, we invite you to reach out. We will see you again in June. Happy Memorial day to everybody. And Shahin as always, thank you so much.