top of page

The Rise of AI Viruses

Updated: Dec 14, 2021



According to What is the economic cost of covid-19?, the cost of the Covid pandemic could amount to $10tn in forgone GDP over 2020-21, making it the most impactful biological virus (so far). The accolade of most impactful computer virus according to The Top 10 Worst Computer Viruses in History | HP® Tech Takes is MyDoom which cost the global economy $52bn (inflation adjusted) and even now accounts for 1% of all phishing emails. Can an AI virus wreak similar damage? Or is AI immune?


Evidence shows that AI is very much susceptible to attack vectors (viruses). Famously In 2016, Microsoft’s Racist Chatbot Revealed the Dangers of Online Conversation when Twitter turned Microsoft’s innocent research chatbot into a horrible persona (so fitted well in the twittersphere). Today, you can download a makeup look from CV Dazzle: Computer Vision Dazzle Camouflage to defeat (a specific) CCTV face detection algorithm. In fact, AI may be more susceptible to viruses than conventional software.


The attraction of ML models that learns from the data presented (Unsupervised learning) is clear: labelling data accurately is labour-intensive and is normally done once at training so cannot react to model drift (IBM Watson Studio - Model Drift). If a model can learn from the data it processes, then we don’t have the cost of labelling, and we can continuously retrain our model to the latest data. By definition, the data is interpreted as software as it programs the model. Data poisoning is our first attack vector for an AI virus, and is extremely effective as Microsoft found out. It’s effective (and terrifying) because you don’t have to understand how the ML model works to be able to subvert it, so there is a trivial barrier to execute such an attack.


If you do understand how the model works, then so much the better. If you understand which features contribute most to the facial recognition, then you can design makeup that obfuscates that feature. If you understand which features contribute most to the ML model, then you can tailor the presentation of your data to achieve the decision that you want. Presenting counterfactual data (or lying) is our second attack vector, albeit one with a much higher barrier to execute because the lies need to be targeted to the specific ML algorithm.



Explainability is AI anti-virus


As with software or physical viruses, the first step in combating the virus is to understand the attack vector. This in turn requires understanding how the ML model arrives at its prediction. This is one of the reasons why we’re focused on Engineering Explainability into Oscar Enterprise AI; not just to quantify how a decision is made, but also what happens in the decision space leading up to the decision boundary.



By understanding the contributing features, AI practitioners can:

  • demonstrate which features solve the specific problem under consideration (and only utilise those features);

  • communicate how the problem is solved to users and regulators as well as fellow practitioners;

  • account for how these features solve the problem, and govern their utilisation.


It is this accounting and governing which is the next step in an AI anti-virus. When we know where the model is most vulnerable (the grey area), we can be vigilant to the effect of training data in this grey area, and perhaps add additional decision validation for grey input data.


We can therefore see that there are approaches that can be developed into tools to protect us from malicious misuse of ML models. I find it comforting that they are the very same tools that we need to build Responsible AI. It turns out that responsible AI is safer AI.

339 views20 comments

20 Comments


Lawrence Jonish
Lawrence Jonish
Dec 28, 2023

The rise of AI viruses is a fascinating, albeit concerning, technological evolution. It's critical to stay informed and proactive as we navigate this new digital frontier. Understanding how to stop feeling sick fast in the digital realm is crucial just as we've learned ways to fight conventional illnesses. Stay curious while remaining safe!


Like

David Williams
David Williams
Dec 12, 2023

I appreciate you allowing me to see this helpful knowledge. This is crucial for successfully conducting business or attempting to do so.If you are looking for best logistics then Supply Chain Consulting Firms In USA is the best

Like

Ty Brooks
Ty Brooks
May 29, 2023

It was very beneficial! Sincerely, I've only recently begun learning about this, but I'm learning a lot already. I'm getting to know it better! Thank you so much, and continue the fantastic work. https://glockempire.com/collections/all

Like

The information you gave truly flabbergasted me. I gained a ton from your composition, and I'm anticipating perusing your forthcoming passages. Deck Designers


Like

Exchange Savvy
Exchange Savvy
Mar 20, 2023

I discovered this blog web page and was pleasantly surprised by the material. From now on, I want to investigate a more notable aspect of your website. www.exchangesavvy.com/

Like
bottom of page