Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

VoteTRANS: Detecting Adversarial Text without Training by Voting on Hard Labels of Transformations

About

Adversarial attacks reveal serious flaws in deep learning models. More dangerously, these attacks preserve the original meaning and escape human recognition. Existing methods for detecting these attacks need to be trained using original/adversarial data. In this paper, we propose detection without training by voting on hard labels from predictions of transformations, namely, VoteTRANS. Specifically, VoteTRANS detects adversarial text by comparing the hard labels of input text and its transformation. The evaluation demonstrates that VoteTRANS effectively detects adversarial text across various state-of-the-art attacks, models, and datasets.

Hoang-Quoc Nguyen-Son, Seira Hidano, Kazuhide Fukushima, Shinsaku Kiyomoto, Isao Echizen• 2023

Related benchmarks

TaskDatasetResultRank
Adversarial Text DetectionIMDB
F1 Score97.7
25
Adversarial Text DetectionIMDB (test)
F1 Score97.8
24
Adversarial Text DetectionAG-News
F1 Score96.7
24
Adversarial Text DetectionYelp
F1 Score97.4
15
Adversarial Text DetectionRTMR
F1 Score86.9
11
Adversarial Text DetectionYelp (test)
F10.974
7
Adversarial Text DetectionAG News (test)
F1 Score95.5
6
Adversarial Text DetectionRTMR (test)
F1 Score83.8
3
Adversarial AttackAG News (test)
Attack Success Rate0.004
3
Adversarial AttackIMDB (test)
Success Rate4.3
3
Showing 10 of 10 rows

Other info

Code

Follow for update