When AI Becomes the Hacker

Bruce Schneier explores the potential dangers of artificial intelligence (AI) systems gone rogue in society.

For the past couple of years, renowned technologist and researcher Bruce Schneier has been researching how societal systems can be hacked, specifically the rules of financial markets, laws, and the tax code. That led him to his latest examination of the potential unintended consequences of artificial intelligence on society: how AI systems themselves, which he refers to as AIs, could evolve such that they automatically - and inadvertently - actually abuse societal systems.
Its AIs as the hacker, he says, rather than hackers hacking AI systems.
Schneier will discuss his AI hacker research in
a keynote address on Monday at the 2021 RSA Conference
, which, due to the pandemic, is being held online rather than in person in San Francisco. The AI topic is based on a recent essay he wrote for the Cyber Project and Council for the Responsible Use of AI at the Belfer Center for Science and International Affairs at Harvard Kennedy School.
The core question Schneier asks is this: What if artificial intelligence systems could hack social, economic, and political systems at the computer scale, speed, and range such that humans couldnt detect it in time and suffered the consequences?
Its where AIs evolve into the creative process of finding hacks.
Theyre already doing that in software, finding vulnerabilities in computer code. Theyre not that good at it, but eventually they will get better [while] humans stay the same in their vulnerability discovery capabilities, he says.
In less than a decade from now, Schneier predicts, AIs will be able to beat humans in capture-the-flag hacking contests, pointing to the DEFCON contest in 2016 when an AI-only team called Mayhem came in dead last against all-human teams. Thats because AI technology will evolve and surpass human capability.
Schneier says its not so much AIs breaking into systems, but AIs creating their own solutions. AI comes up with a hack and a vulnerability, and then humans look at it and say, Thats good, and use it as a way to make money, like with hedge funds in the financial sector, he says.
The irony here, of course, is that AI starts with human input and programming. Frankenstein analogies aside, the core problem is that AI doesnt have the same human cognitive functions like empathy or a gut check to know where to draw the line. Schneier notes that while theres plenty of research on incorporating context, ethics, and values into AI programs, its not a built-in function of todays AI systems.
Even so, he says, humans will employ AI to find loopholes in tax codes, such as a major accounting firm doing so to find a new tax dodge to sell to their customers. So financial firms arent likely to program in rules that thwart their ability to monetize AI knowledge.
The biggest risk is that AIs will find a way around a rule without humans knowing - that AIs will figure out something that will hack the rules and we wont realize that, Schneier says.
Schneier points to the Volkswagen scandal in 2015, when the carmaker was caught cheating on emissions control-level tests of its vehicle models after engineers programmed the cars computer systems to activate emissions-curbing only during tests, not in its normal operations.
There it was humans attacking the rules and not AI itself, he says, but its a good example of what AI could ultimately do to cheat a system if left unchecked to learn ways around it.
In
his essay, The Coming AI Hackers,
Schneier describes it this way: If I asked you to design a cars engine control software to maximize performance while still passing emissions control tests, you wouldnt design the software to cheat without understanding that you were cheating. This simply isnt true for an AI; it doesnt understand the abstract concept of cheating. It will think out of the box simply because it wont have a conception of the box, or of the limitations of existing human solutions. Or of ethics. It wont understand that the Volkswagen solution harms others, that it undermines the intent of the emissions control tests, or that it is breaking the law.
A Wake-up Call to Action
Schneier admits the concept of AIs as hackers is super speculative for now, but its an issue that needs to be addressed.
We need to think about this, he says. And Im not sure you can stop this. The ease of this [AIs hacking] happening depends a lot on the domain [in question]: How can we codify the rules of the system?
The key is harnessing AIs for defense, like finding and fixing all vulnerabilities in a software program before it gets released.
Wed then live in a world where software vulnerabilities were a thing of the past, he says.
The downside is the transition period would be vulnerable: Legacy or already-released code could be at risk of attack by AI tools abused by adversaries, he says.
The risk is AI systems hacking other AI systems in the future, and humans experiencing the fallout, he says.
Schneiers latest AI research evolved out of his study of
how the hacker mindset and skills could be applied to securing societal systems
, which he first presented at the 2020 RSA Conference in San Francisco. This concept, which he coined hacking society, would mean ethical hackers helping fix the US tax code and legislation to avoid inadvertent or deliberate loopholes, for example.
His big idea boils down to this: Can we hack society and help secure the systems that make up society?
Meanwhile, keep an eye on AIs hacking society.
Computers are much faster than people. A human process that might take months or years could get compressed to days, hours, or even seconds. What might happen when you feed an AI the entire US tax code and command it to figure out all of the ways one can minimize the amount of tax owed? he wrote in his essay.

Last News
▸ Veritabile Defecte de Proiectare a Securitatii in Software -> Top 10 Software Security Design Flaws ◂ Discovered: 23/12/2024 Category: security	▸ Sony, XBox Targeted by DDoS Attacks, Hacktivist Threats ◂ Discovered: 23/12/2024 Category: security	▸ There are plenty of online tools for reporting bugs. ◂ Discovered: 23/12/2024 Category: security

**Cyber Security Categories**
Google Dorks Database	Exploits Vulnerability	Exploit Shellcodes

CVE List

Tools/Apps

News/Aarticles

Phishing Database

Deepfake Detection

Trends/Statistics & Live Infos

Tags:
When AI Becomes the Hacker