Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

MQUAKE

Benchmarks

Task NameDataset NameSOTA ResultTrend
Instruction FollowingMQUAKE
Accuracy82.5
24
Knowledge EditingMQuAKE-Story 1.0 (test)
Fact Accuracy (Easy)100
14
Knowledge EditingMQuAKE Story
Fact Accuracy (Easy)100
14
Knowledge EditingMQuAKE-CF 1.0 (test)
Fact Accuracy (Easy)99.9
14
Multi-hop Knowledge EditingMQUAKE-T (All edited)
Accuracy78.16
12
Multi-hop Knowledge EditingMQUAKE-T (1 edited)
Accuracy97.7
12
Multi-hop Knowledge EditingMQUAKE-CF-3K (100 edited)
Accuracy56
12
Multi-hop Knowledge EditingMQUAKE-CF-3K (1 edited)
Accuracy67.27
12
Multi-hop Question AnsweringMQuAKE
MHQ Accuracy31.6
10
Multi-hop Knowledge EditingMQUAKE-CF-3K All edited
Accuracy45.87
10
Knowledge EditingMQUAKE
Average Accuracy0.7589
8
Sequential Knowledge EditingMQuAKE
Efficacy97.4
8
Multi-hop Knowledge EditingMQuAKE CF v2
2-hop Score69.1
6
Showing 13 of 13 rows