Adversarial machine learning is the study of the attacks on machine learning algorithms, and of the defenses against such attacks.
Machine learning techniques are mostly designed to work on specific problem sets, under the assumption that the training and test data are generated from the same statistical distribution (IID). However, this assumption is often dangerously violated in practical high-stake applications, where users may intentionally supply fabricated data that violates the statistical assumption.
Most common attacks in adversarial machine learning include evasion attacks, data poisoning attacks, Byzantine attacks and model extraction.
History
At the MIT Spam Conference in January 2004, John Graham-Cumming showed that a machine-learning spam filter could be used to defeat another machine-learning spam filter by automatically learning which words to add to a spam email to get the email classified as not spam.
In 2004, Nilesh Dalvi and others noted that linear classifiers used in spam filters could be defeated by simple “evasion attacks” as spammers inserted “good words” into their spam emails. (Around
…
Source: Wikipedia
No Comments yet!