Transcript

Listen: The concept of mercenaries, hired soldiers and specialists working privately to fight a nation’s battles, is nearly as old as war itself.

In our fourth episode of “Patching the System,” we’re discussing the threat cyber mercenaries pose to individuals, governments, and the private sector. We’ll examine how spyware used to track criminal and terrorist activity around the world has been abused by bad actors in cyber space who are hacking and spying activists, journalists, and even government officials. And we’ll talk about what’s being done to stop it.

Our participants are:

• John Scott-Railton, Senior Researcher at the Citizen Lab at the University of Toronto's Munk School
• David Agranovich, Director of Global Threat Disruption at Meta.
• Ali Wyne, Eurasia Group Senior Analyst (moderator)

GZERO’s special podcast series “Patching the System,” produced in partnership with Microsoft as part of the award-winning Global Stage series, highlights the work of the Cybersecurity Tech Accord, a public commitment from over 150 global technology companies dedicated to creating a safer cyber world for all of us.

Subscribe to the GZERO World Podcast on Apple Podcasts, Spotify, Stitcher, or your preferred podcast platform, to receive new episodes as soon as they're published.

Podcast: Cyber Mercenaries and the digital “wild west"

Disclosure: The opinions expressed by Eurasia Group analysts in this podcast episode are their own, and may differ from those of Microsoft and its affiliates.

John Scott-Railton:You go to a growing number of mercenary spyware companies and surveillance companies that basically offer you NSA-style capabilities in a box and say, "Look, you can pay us a certain amount of money and we're going to send you this stuff." You're seeing basically the direct proliferation, not only of those capabilities, but actually national security information about how to do this kind of hacking moving its way right into the private sector.

David Agranovich: They fill a niche in the market, nation states that lack surveillance capabilities themselves, threat actors who want deniability in their surveillance activities and clients like law firms or litigants who want an edge on their competition. In reality, the industry is putting a thin veneer of professionalism over the same type of abusive activity that we would see from other malicious hacking groups.

INTERVIEW

Ali Wyne: Welcome to Patching the System, a special podcast for the Global Stage series, a partnership between GZERO Media and Microsoft. I'm Ali Wyne, a Senior Analyst at Eurasia Group.

Throughout this series, we're highlighting the work of the Cybersecurity Tech Accord, a public commitment from over 150 global technology companies dedicated to creating a safer cyber world for are all of us. And today we're talking about mercenaries and the concept is almost as old as warfare itself. Hired guns, professional soldiers used in armed conflict. From Germans employed by the Romans in the fourth century to the Routiers of the Middle Ages, to modern day security firms whose fighters have been used in the Iraq and Afghanistan wars, as well as the current war in Ukraine.

But our conversation today is about cyber mercenaries. Now these are financially motivated private actors working in the online world to hack, to attack and to spy on behalf of governments. And in today's world where warfare is increasingly waged in the digital realm, nations use all the tools of their disposal to monitor criminal and terrorist activity online.

Now that includes spyware tools such as Pegasus, a software made by the Israel-based cyber security firm NSO Group that is designed to gain access to smartphone surreptitiously in order to spy on targets. But that same software, which government organizations around the world have used to attract terrorists and criminals has also been used to spy on activists, journalists, even officials with the U.S. State Department.

Here to talk more about the growing world of cyber mercenaries and the tech tools they use and abuse are two top experts in the field, John Scott-Railton or JSR, he's a Senior Researcher at the Citizen Lab at the University of Toronto's Munk School and David Agranovich, who now brings his years of experience in the policy space to his role as Director of Global Threat Disruption at Meta. Welcome to both of you.

JSR: Good to be here.

David Agranovich: Thanks for having us.

Ali Wyne: JSR, I'm going to start with you. So I mentioned in my introductory remarks, this Pegasus software. So tell us a little bit more about that software produced by the NSO Group and how it illustrates the challenges that we're here to talk about today?

JSR: So you can think of Pegasus as something like a service, governments around the world have a strong appetite to gain access to people's devices and to know what they're typing in and chatting about in encrypted ways. And Pegasus is a service to do it. It's a technology for infecting phones remotely, increasingly with zero-click vulnerabilities. That means accessing the phones without any deception required, nobody needs to be tricked into clicking a link or opening an attachment. And then to turn the phone into a virtual spy in the person's pocket. Once a device is infected with Pegasus, it can do everything that the user can do and some things that the user can't. So it can siphon off chats, pictures, contact lists but also remotely enable the microphone and the video camera to turn the phone into a bug in a room, for example. And it can do something else, which is it can take the credentials the user and the victim use to access their cloud accounts and siphon those away too and use those even after the infection is long gone to maintain access to people's clouds.

So you can think of it as a devastating and total access to a person's digital world. NSO, of course, is just one of the many companies that makes this kind of spyware. We've heard a lot about them, in part, because there's just an absolute mountain of abuse cases. Some of them discovered by myself and my colleagues around the world with governments acquiring this technology, perhaps some of the rubric of doing anti-terror or criminal investigation but of course they wind up conducting political espionage, monitoring journalists and others.

Ali Wyne: David, let me come to you. So I think that we should just, before we dive into the deeper conversation, getting a little bit into semantics, a little bit into nomenclature. But let's just start with some basic definitions. When most folks hear the phrase cyber mercenary, some of them might just think it's any kind of bad actor, hacker, others of them might draw parallels to real life, analog kind of mercenaries, so sort of hired soldiers in war. So how do you define the phrase cyber mercenary? How does Meta define the term cyber mercenary and why?

David Agranovich: So maybe just to ground ourselves in definitions a bit. My team at Meta works to coordinate disruption and deterrence of a whole ecosystem of adversarial threat actors online. And so that can include things like info ops, efforts to manipulate and corrupt public debate through fake personas. It can include cyber espionage activity, which is similar to what we're talking about today. Efforts to hack people's phones, email addresses, devices and scaled spamming abuse. When we're talking about cyber mercenary groups, I think of that within the broader cyber espionage space. There are people who are engaged in, as JSR talked about, surveillance, efforts to try and collect info on people to hack their devices, to gain access to private information across the broader internet. These are private companies who are offering surveillance capabilities, which once we're essentially the exclusive remit of nation state intelligence services, to any paying client.

The global surveillance-for-hire industry, for example, targets people across the internet to collect intelligence, to try and manipulate them into revealing information about themselves and ultimately to try and compromise their devices, their accounts, steal their data. They'll often claim that their services and the surveillance ware that they build are intended to focus on criminals, on terrorists. But what our teams have found and groups doing the incredible work like Citizen Lab is that they're regularly targeting journalists, dissidents, critics of authoritarian regimes, the family of opposition figures and human rights activists around the world.

These companies are part of a sprawling industry that provides these intrusive tools and surveillance services, indiscriminately to any customer, regardless of who they're targeting or the human rights abuses that they might enable.

Ali Wyne: What strikes me just in listening to your response is not only how vast, how sprawling this industry is, also how quickly it seems to have risen up. I think that just comparing the state of this industry today versus even 10 years ago or even five years ago. How did it rise up? What are some of the forces that are propelling its growth and give us a sense of it, the origin story and what the current state of play of this industry is today?

David Agranovich: As we see it, these firms grew out of essentially two principal factors. The first impunity and the second a demand for sophisticated surveillance capabilities from less sophisticated actors.

On the first point, companies like NSO or Black Cube or those that we cited in our investigative report from December last year, they wouldn't be able to flagrantly violate the privacy of innocent people if they faced real scrutiny and costs for their actions. But also, to that second point, they fill a niche in the market, nation states that lack surveillance capabilities themselves, threat actors who want deniability in their surveillance activities and clients like law firms or litigants who want an edge on their competition. In reality, the industry is putting a thin veneer of professionalism over the same type of abusive activity that we would see from other malicious hacking groups.

Ali Wyne: So JSR, so I want to come to you now. So David has kind of given us this origin story and he has given us a state of play and has really given us a sense of how sprawling this industry is. So, I guess, for lack of a better phrase, there are jobs here, there are jobs in this space. Who's hiring these cyber mercenaries and for what purposes? Who are they targeting?

JSR: There are a lot of jobs. And I think what's interesting, David pointed out the problem about accountability. And I think that's exactly right. Right now, you have an ecosystem that is largely defined only by what people will pay for, which is a seemingly endless problem set. So who's paying? Well, you have a lot of governments that are looking for this kind of capability that can't develop it endogenously and so go onto the market and look for it. I think even after the Snowden revelations, a lot of governments were like, "Man, I wish I had that stuff. How do we get that?" And the answer is increasingly simple. You go to a growing number of mercenary spyware companies and surveillance companies that basically offer you NSA-style capabilities in a box and say, "Look, you can pay us a certain amount of money and we're going to send you this stuff."

And as David points out, a lot of it is done under the sort of rhetorical flag of convenience of saying, "Well, this is stuff for tracking terrorists and criminals." But actually at this point, we probably have more evidence of abuses than we do confirmed cases where this stuff has been used against criminals. Who's doing the work? A lot of the people who go into this industry are hired by companies with names like NSO, Candiru and others. Many of them come out of government, they come out of either doing their military service in a place like Israel in a unit that focuses on cyber warfare or they come out of places like the CIA, the NSA, Five Eyes and other countries' intelligence services.

Which in itself is really concerning because you're seeing basically the direct proliferation, not only of those capabilities, but actually national security information about how to do this kind of hacking moving its way right into the private sector. And we've seen some really interesting cases in the last year of people who came out of The US intelligence community, for example, doing exactly this kind of thing and then pretty recently getting indicted for it. And so my hope is that we're beginning to see a bit of accountability around this but it's a really concerning problem set in part because the knowledge is specialized, a lot of it relates to countries' national security and it's now flowing into a big, sprawling unregulated marketplace.

Ali Wyne: So David, let's build on what JSR just said. So we have this big, sprawling, it seems increasingly unregulated surveillance ecosystem. It's more democratized, they are more individuals who can participate, the surveillance is getting more sophisticated. So I want to go back to your day job. Honestly, you have a big purview, you head up Global Threat Disruption at Meta, which is responsible for a very wide range of platforms. Which groups do you see in your personal capacity, in your professional capacity at Meta, which groups do you see as being most vulnerable to the actions of cyber mercenaries?

David Agranovich: So I think what's remarkable about these types of cyber mercenary groups, as JSR has noted I think, is just how indiscriminate their targeting is across the internet and how diverse that targeting is across multiple different internet platform. When we released our research report into seven cyber mercenary entities last year, we found that the targets of those networks ranged from journalists and opposition politicians to litigants and lawsuits to democracy activists. That targeting wasn't confined to our platforms either. One of the most concerning trends that we saw across these networks and which Citizen Lab has done significant amount of investigative reporting into is the use of these types of technologies to target journalists, often in countries where press freedoms are at risk and the use of these types of technologies, not just to try and collect open source information about someone, but really trying to break into their private information to hack their devices.

Some of the capabilities that JSR mentioned about the Pegasus malware for example, are incredibly privacy intrusive. Ultimately the problem that I see here is these firms effectively obscure the identity of their clients. Which means anybody, authoritarian regimes, corrupt officials, any client willing to pay the money, can ostensibly turn these types of powerful surveillance tools on anyone that they dislike. And so to answer your question, who's most vulnerable? The reality is that anyone can be, it's why we have to take the activities of these types of firms so seriously.

Ali Wyne: So you both have given us a sense of, again, this really sprawling surveillance ecosystem, the growing range of targets, the growing democratization of this kind of nefarious activity. Can you give us a sense of what tactics you've seen lately that are new? I mean, when I think back to some of the earlier conversations we've had in this podcast series, some of the guests we've had have said, look, there are basic precautionary measures that all of us can take, whether we are a technology expert, such as yourselves or whether we're just a lay consumer.

So use different passwords for different platforms, taking basic steps to safeguard our information. But obviously I think that the pace at which individuals can adapt and the pace at which individuals can take preventative measures, I think is invariably going to be outstripped by the speed with which actors can adapt and find new ways of engaging in cyber mercenary activities. So in your time at Meta, have you seen new tactics being used by these groups in recent years and how are you tracking those and identifying them?

David Agranovich: So maybe just to ground our understanding of how these operations work.

Ali Wyne: Sure.

David Agranovich: How do these tactics fit across the taxonomy? We break these operations down into three phases, what we call The Surveillance Chain. The first phase called reconnaissance is essentially an effort by a threat actor to build a profile on their target through open source information. The second phase which we call engagement is where that threat actor starts to try and build a rapport with the target, with the goal of social engineering them into the final phase, which is exploitation. That final step, which most often happens off of our platform is where the target receives malware or a spearphishing link in an attempt to steal their counter data.

Generally, the way we see the tactics throughout these three phases play out is we'll see these operations use social media early in their targeting to collect information to build a profile in the reconnaissance phase or to try and engage with a target and build a rapport in the engagement phases. And then they'll attempt to divert their target to other platforms like malware riddled websites, for example, where they might try to get a target to download a Trojanized chat application that then delivers malware onto their device or other social media platforms where they'll try and exploit them directly.

David Agranovich: I think the most consistent trend we see with these types of operations is adversarial adaptation. What that means is when we do these take downs and when our teams publish reports on the tactic we're seeing or when in open source investigative organizations or civil society groups find these types of networks themselves and disclose what they're doing, these firms adapt quickly to try and get around our detection. It ultimately makes it really important, one, to keep investigating and holding these firms accountable. And two, to essentially follow these threats wherever they may go, tackle this threat as a whole of society problem. That's going to require more comprehensive response if we want to see these types of tools used in a responsible way. But those are, I think, some of the trends we've seen more broadly.

JSR: Mm-hmm (affirmative).

Ali Wyne: And JSR, let me come to you, just in responding to David. So in your own work at Citizen Lab, what kinds of trends are you observing in terms of either targets and/or tactics?

JSR: Well, the scariest trend, and I think we're seeing it more or less wherever we scratch, is zero-click attacks. So it used to be, you could tell people and be Buddhist about it, "Look, detached from attachments. Be mindful of links that can bite." There's a way to do that and in fact, I'm not just pulling that out from nowhere. We worked many years ago with a group of Tibetans who were looking for a campaign of awareness raising to reduce the threat from Chinese threat actors. And so we used this very Buddhist concept of detaching from attachments, stop sending email attachments to each other. Which resulted in a real drop in the efficacy of these Chinese hacking groups as they were trying to find new ways to get people to click on malware. It took a while.

Ali Wyne: Got it.

JSR: But ultimately, per David, we saw adaptation. In general, I think the problem is twofold. One, human behavior is fraught with what we call forever day vulnerabilities, you can't patch. People are vulnerable to certain kinds of things, certain kinds of deception. And so we need to look at platforms and technologies to do part of that work of protecting people and to try to prevent attacks before they reach the level of a victim, having a long, drawn out conversation with somebody. The other thing, of course, that's really concerning, NSO and many others at this point are selling their customers ways to infect devices, whether it's laptops or phones that don't require any user interaction. And obviously this is pretty bad because there's nothing you can do about it as a user, you can keep your device updated but you'll still potentially be susceptible to infections. So you can't really tell people, "Look, here are the three things and if you just do them right, you'll be fine."

The second problem set that it creates is that it makes it a lot harder for investigators like us to find traces of infection quickly. It used to be the case a couple years ago even, that when I would run a big investigation to find cases of, say, NSO targeting, the primary process of investigation would involve finding text messages, finding those infection messages. Even if the forensic traces of the infection were long gone, we could find those. But now we have to do forensics, which means that for defenders and researchers and investigators like us, it creates a much bigger lift in order to get to a place where we understand what's going on with an attack. And that to me is really concerning. People in the government side talk about concerns around encryption causing criminals to go dark. My biggest concern is hacking groups going dark because it's a lot harder to spot when the infections happen. Of course, the harm remains and that's really what we're talking about.

Ali Wyne: I suspect that this will be a phrase that will be new to a lot of listeners or fellow listeners such as myself but when you said, "Detachment from attachment," and I said, "It's such a nice turn of phrase," and I didn't actually realize until you related this anecdote, I didn't realize that it was actually grounded in a professional experience that you had.

JSR: Yeah.

Ali Wyne: But I think it's a compelling mantra for all of us, "Detachment from attachment." I do want to be fair and I want to make sure that we're giving listeners a full picture. And so David, let me come back to you. And so one question I imagine some listeners will have, is that in theory, cyber mercenaries could be used for good? Are there some favorable or at a minimum at least, some legitimate ways that cyber mercenaries can, and/or should be employed? I mean, are there places where they're operating legally? Are there places where they're doing good work? So maybe give us a little bit of a perspective on the other side of the ledger?

David Agranovich: So I'll certainly try but I should preface this by saying, most of my career before I joined Meta was in the National Security space.

Ali Wyne: Right.

David Agranovich: And so I take the security threats that I think some of these firms talk about very seriously. The reality is that law enforcement organizations and governments around the world engage in some of this type of surveillance activity. But what's important is that they do that subject to lawful oversight. And with limitations on their legal authorities, at least in democratic systems. What makes this industry so pernicious and so complicated is, at least as far as we can tell, there's no scalable way to discern the purpose or the legitimacy of their targeting. What's more the use of these third-party services obfuscates who each end customer might be, what they are collecting and how the information is being used against potentially vulnerable groups.

There's essentially just a fundamental lack of accountability or oversight in the surveillance-for-hire industry that makes it hard to determine whether any of this targeting could be considered legitimate. If we wanted to develop a whole-of-society approach to the surveillance-for-hire space and answer your question, we would need to, one, create the oversight and accountability that surveillance tools should receive. Two, hold these companies accountable for how they of tools are used or misused. And three, align through the democratic process on how much these firms should be allowed to do. Until we answer those questions, the surveillance industry will be ripe for abuse.

So one of the interesting things I like to think about is people think that the problem with the mercenary spyware industry is that it sells to autocrats and authoritarians. And of course, it's true. That is part of the problem with the industry because you can guarantee that autocrats and authoritarians are probably going to use this technology in bad ways, in ways that are anti-democratic and problematic. But we now have a couple of year’s experience looking at what happens when big, sprawling democracies from Mexico to India to Poland, get their hands on Pegasus. And what we see is abuses there too.

And so I like to think of the problem set as actually being one, that there are very few customers that you could sell this kind of technology to, that you could sell this really sophisticated surveillance capability to that wouldn't be likely to abuse it. And to me, you have to situate this within the broader problem set, which is authoritarianism is resurgent around the world. And unfortunately, this technology has come time when lots of authoritarians and want-to-be authoritarians are looking for technological ways to get into the heads and phones of their subjects and people around the world. And it's just a very unfortunate thing that these two things are happening at the same time. But I think we can look around the world and say, the mercenary industry is absolutely increasing the speed of authoritarianism in certain country contexts, including in certain democracies that are sliding towards authoritarianism. Hungary would be an example, El Salvador is another, both big Pegasus scandals, both on paper are democratic, but really moving in a concerning direction.

Ali Wyne: I think that context you provided, that geopolitical context is a really helpful backdrop for or an overlay on our broader conversation. Up until now, we've been talking about trends in the digital space and I think you're bringing in this geopolitical element and you put the two together and there's a real prospect of not only resurgent authoritarianism but resurgent authoritarianism imbued with ever more sophisticated technology. So I think that you've given us…You've given us a sense of that digital geopolitical nexus and really a scale of the problem. I want to have you both react just given the scale of this problem, JSR as you've outlined it, I want to get you both to react to a conversation or a snippet of a conversation I recently had with Annalaura Gallo. She's the Head of the Secretariat of the Cybersecurity Tech Accord. And here's what she had to say about cyber mercenaries.

Annalaura Gallo: So the issue here is that we have a private industry that is often legal, that is focused on building very sophisticated, offensive cyber capabilities because these are sometimes even more sophisticated that states can develop. And then they're sold to governments but also other customers. And essentially they're made to exploit peaceful technology products. We know they've also been used by authoritarian governments for surveillance and to crack on political opposition in particular. And we think that all this is extremely concerning because first of all, we are witnessing a growing market. There is a proliferation of these cyber capabilities that could finally end up in the wrong hands. So not only governments but also malicious actors that use these tools to then conduct larger scale cyber attacks. So we don't see how we can just continue in a framework where there is no regulation of these actors because this would just put not only human lives at risk, but also put at risk the entire internet ecosystem.

Ali Wyne: So David, let me come to you. if this nexus of issues is so large, who needs to begin to take responsibility and how? You speak as a representative from a major industry player, Meta. What can the private sector in particular do to mitigate the impact of cyber mercenaries? And maybe if you could just give us a sense of some general industry principles that you'd recommend.

David Agranovich: There's a responsibility, I think, spread across governments, tech companies, civil society and the surveillance industry itself. Governments have the most power to meaningfully constrain the use of these tools. They can hold abusive firms accountable and they can protect the rights of the victims that these firms target. This industry has thrived in a legal gray zone. So the lack of oversight, the lack of regulation has enabled them to grow and appropriate oversight and regulation would go pretty far in curbing some of the worst abuses. Tech companies like ours also need to continue doing what we can to help protect our users from being targeted and to provide people with the tools to strengthen their account security. We need to make it harder for surveillance companies that are part of this industry to find people on our platform and to try and compromise their devices or their accounts.

We routinely investigate these firms. And when we do, we take steps to curb their use of fake accounts, we work to reverse engineer their malware. And then when we do, we share threat indicators or indicators of compromise with other industry players and with the public. So we're also working to help notify the victims when we see them being targeted and that also can help take steps to mitigate the risk. Because these operations are so often cross-platform, they might leverage applications, they might leverage social media websites, they may leverage websites controlled by the attacker. If we see someone being targeted on one of our platforms, we believe that by sending them a notification that we think they are being targeted. And in that notification, giving them specific steps to follow to lock down their cybersecurity presence, hopefully that doesn't just protect them from being targeted on our platform, it also might cut off avenues of attack if a surveillance company is trying to get at them on another way.

Third, civil society also has an important role to play, in particular, in determining what the norms in this space should be. What's acceptable? What's going too far? And how to start creating those expectations more broadly. And then finally, I mentioned, the surveillance industry has responsibilities here. You can see these firms claim, as JSR has noted, that they're just in the business of targeting terrorists and criminals. That's just not what our investigations find.

JSR: I agree with David. I think you have to have consequences and accountability and we are getting there. One of the most interesting things that happened in this space the last couple years was The Commerce Department choosing to list NSO. Now this, of course, limits the ability of American companies to do business with NSO Group. But it had an immediate and radical signaling effect on investors in NSO and the value of NSO's debt plummeted. I think what's interesting about that is that it shows that the industry and the people who are interested in investing in it kind of know how far offsides they are from basic norms and ethics and risks. And the issue is just that for too long there haven't been consequences.

To put this into a bit of a historical perspective. We've been reporting on the mercenary spyware industry for a decade. Things really started changing only in 2019 when WhatsApp and Meta chose to sue NSO Group. That was the beginning of a different phase. Up until that point, NSO had been like the bully on the playground and civil society groups and people working with victims were like the bullied kids. NSO was just a bigger company, more powerful, pouring millions into PR and lobbying.

Suddenly things got a little more complicated for NSO. And then in the last two years, we've seen not only a string of scandals around NSO coming from a place of investigations and research, but also Apple and others joining legal actions against NSO. And then signals from the US Government, both around NSO specifically and more generally towards the mercenary spyware industry. So I think we have a model for what's needed. It looks like legal consequences and accountability for abuses. It looks like serious leaning in by players like Meta, Apple and others using all the tools available, not just technical control measures. It also looks like making sure that governments do their bit and they protect their own citizens and they also make sure that companies that are really the worst offenders, fueling proliferation, are not able to make a big success at it.

And I think we're still learning how some of these things play out but it's been essential to have big platforms leaning in. I see it a little bit like a stool, you have civil society, you have government, and you have the private sector. And we have two legs now, private sector and civil society and that third leg I think is coming. I'm very excited, for example, that the European Union is on the cusp of opening up a Committee of Inquiry into Pegasus and the mercenary spyware industry, more generally, they have a pretty broad mandate. And I just hope to continue to see more governments taking action.

I think when we see that happen, we're also going to see a real shift in the norms of the debate. Because the problem here is not just the tech, it's really the proliferation of that tech. And you solve that problem in the same way that you would solve the proliferation of other kinds of technology that can be used for war and instability. One bug I want to put in the ear of your listeners is this. So we talk about this stuff, as we're talking about the harms that come directly from an attack. So, the harms to an individual or the person that they're in contact with when they get hacked or even to the chilling effect on democracy and civil society somewhere, if all the journalists are being bugged by a greedy autocrat.

But the problem space is actually much larger, as I think some of this conversation has pointed out. If the US Government cannot ensure that its cyber weapons stay outside of the hands of criminal groups, what's the likelihood that mercenary spy war players selling to governments that absolutely cannot get their act together like Togo, for example, is going to prevent these very sophisticated zero-day vulnerabilities and other flaws from being used in a much more vigorous way by cyber criminal groups and others that may get their hands on them? To me, that's one of the biggest concerns because we've been playing fire with this problem since the beginning and mark my words, it's only a matter of time before we see really serious, bad happening here.

Ali Wyne: You mentioned that three-legged stool and you mentioned that we have two prongs of that stool but we need to work on the third one. Obviously a lot of work to do but really grateful that the two of you are involved in that work. John Scott-Railton, Senior Researcher at the Citizen Lab at the University of Toronto's Munk School. David Agranovich, Director of Global Threat Disruption at Meta. Thanks so much for this really terrific conversation.

JSR: Thank you so much.

David Agranovich: Thank you, Ali.

Ali Wyne: That's it for this episode of Patching the System. Next time we'll wrap up this series with a look at the Cybercrime Treaty negotiations underway at the United Nations, and what it could mean for cyberspace globally. You can catch this podcast as a special drop in Ian Bremmer's GZERO World feed anywhere you get your podcast. I'm Ali Wyne, thanks very much for listening.