“The main difficulty with detecting trojan attacks is that deep learning models are often opaque black box models,” Ruixiang Tang, Ph.D. student at Texas A&M University and lead author of the paper, told The Daily Swig.
“Unlike the traditional software that can be analyzed line by line, even if you know the model structures and parameter values of a deep neural network, you may not be able to extract any useful information due to multiple layers of nonlinear transformations.
“Hence predicting whether a model contains trojans has become a very challenging task.”
The "attack" is to replace the model with your own model that you have designed to make particular mistakes on command. Hard to detect, yes, but you could pull the same trick with almost any other...
The "attack" is to replace the model with your own model that you have designed to make particular mistakes on command. Hard to detect, yes, but you could pull the same trick with almost any other component of the system.
Yes, it's an insider attack. Although organizations try to hire trustworthy people, they don't want to depend only on that. Compare with changing ordinary computer code, where the code is...
Yes, it's an insider attack. Although organizations try to hire trustworthy people, they don't want to depend only on that. Compare with changing ordinary computer code, where the code is typically reviewed by another team member and it's also tested. Here, there's nothing to review (it's an opaque pile of numbers) and it will pass the tests.
It's true, though, that organizations trust their employees in other ways that aren't as easily reviewed as source code.
From the article:
The "attack" is to replace the model with your own model that you have designed to make particular mistakes on command. Hard to detect, yes, but you could pull the same trick with almost any other component of the system.
Yes, it's an insider attack. Although organizations try to hire trustworthy people, they don't want to depend only on that. Compare with changing ordinary computer code, where the code is typically reviewed by another team member and it's also tested. Here, there's nothing to review (it's an opaque pile of numbers) and it will pass the tests.
It's true, though, that organizations trust their employees in other ways that aren't as easily reviewed as source code.