Malicious Tool Call Detection on TRUST-BENCH reconstructed comparison set curated 1,970-episode (test)

100AMR

ToolShield

Updated 15d ago

Evaluation Results

Method
ToolShield 2026.05	100	0	0	100	0
GuardAgent 2026.05	100	0	0	100	0
LLM-as-Judge 2026.05	100	0	0	100	0
Keyword Heuristic 2026.05	93.3	6.7	1.5	98.5	0
Gradient Boosting (10 scalar feat.) 2026.05	43.8	56.2	36.3	63.7	0
GPT-4o Judge (features) 2026.05	25.2	74.8	80.9	19.1	0
GPT-4o Judge (full) 2026.05	22.2	77.8	80.4	19.6	0
TF-IDF + MLP (same input) 2026.05	14.2	85.8	25.2	74.8	33.9
TF-IDF + MLP (raw text) 2026.05	12.2	87.8	23.8	76.2	39.9
TF-IDF + SVM (same input) 2026.05	11.7	88.3	23.2	76.8	41.7
TF-IDF + LogReg (raw text) 2026.05	10.1	89.9	23.4	76.6	44.6
VISTA-Guard (Qwen2.5-7B) 2026.05	9.6	90.4	10.6	89.4	64.9
TF-IDF + LogReg (same input) 2026.05	9.1	90.9	17.7	82.3	55.2
BERT-base (same input) 2026.05	8	92	48.1	51.9	21.9
VISTA-Guard (Qwen2-8B) 2026.05	7.6	92.4	11.6	88.4	67.6
VISTA-Guard (OLMo-2-7B) 2026.05	6.6	93.4	15.1	84.9	64.2
TF-IDF + SVM (raw text) 2026.05	6.1	93.9	30.9	69.1	41.5
VISTA-Guard (Meta CWM-32B) 2026.05	5.2	94.8	40.4	59.6	39
VISTA-Guard (Llama-3.1-8B) 2026.05	4.6	95.4	18.8	81.2	62.6
VISTA-Guard (Mistral-7B) 2026.05	4.2	95.8	5	95	84.2