Defense against Indirect Prompt Injection on Filtered QA dataset

97.65ASR (Naive)

Reminder

Updated 4d ago

Evaluation Results

Method
Reminder 2024.11	97.65	97.95	100	100	100
Instructional 2024.11	95.55	95.75	99.95	100	100
Spotlight 2024.11	94.35	96.45	100	100	100
Spotlight 2024.11	94.35	96.45	100	100	100
None 2024.11	92.45	95.9	100	100	100
None 2024.11	85.9	91.1	81.7	95.25	92.3
Reminder 2024.11	79.05	77.3	71.75	84.35	80.65
Isolation 2024.11	77.8	88.85	99.35	99.7	100
Isolation 2024.11	76.75	85	89.75	91.7	88.75
Instructional 2024.11	60.15	68.35	88.1	84.7	84.85
None 2024.11	10.55	53.35	88.25	75.3	86
Reminder 2024.11	10.55	39.9	67.5	37.85	51.2
Spotlight 2024.11	8.8	32.85	76.35	74.45	56.6
Instructional 2024.11	6.95	35	80.1	64.45	62.75
Sandwich 2024.11	4.8	6.15	14	34.2	34.6
Sandwich 2024.11	2.5	3.05	22.9	3.35	9.55
Isolation 2024.11	2.2	33.75	83.35	67.4	77.75
Fake Completion Defense 2024.11	1.75	2.45	8.75	0.8	0.6
Escape Defense 2024.11	1.45	1.7	0.75	0.68	4.95
Escape Defense 2024.11	1.25	2.7	1.05	0.9	1.65
Ignore Defense 2024.11	0.85	0.7	0.8	0.95	4.1
Sandwich 2024.11	0.45	9.35	49.55	7.3	21.25
Fake Completion Defense 2024.11	0.3	0.7	0.55	0.45	0.3
Escape Defense 2024.11	0.25	1.7	1.05	0.55	1.45
Ignore Defense 2024.11	0.25	0.3	0.35	0.45	1.1
Fake Completion Defense 2024.11	0.1	1.8	17.7	0.05	0.1
Fake Completion Defense with Template 2024.11	0.1	0.25	0.2	0.15	0.05
Ignore Defense 2024.11	0.05	0.35	0.3	0.1	1.35
Fake Completion Defense with Template 2024.11	0.05	0.05	0.3	0.05	0.05
Fake Completion Defense with Template 2024.11	0.05	0.25	0.2	0.15	0.05