Item crafting on TextCraft (test)

71Success Rate

Fed-SE

Updated 3mo ago

Evaluation Results

Method	Links
Fed-SE 2025.12		71
Local 2025.12		70
Fed-SE 2025.12		67
Local 2025.12		63
Centralized 2025.12		63
FedIT 2025.12		61
Centralized 2025.12		59
ADaPT (Backend: GPT-4o) 2025.12		58
ADaPT (Backend: Mistral Small 3) 2025.12		58
Fed-SE 2025.12		58
SCOPE 2025.12		56
FedIT 2025.12		54
FedIT 2025.12		54
Centralized 2025.12		54
ADaPT (Backend: GPT-3.5) 2025.12		52
Fed-SE 2025.12		52
FedIT 2025.12		49
Centralized 2025.12		48
Local 2025.12		46
FedIT 2025.12		46
Fed-SE 2025.12		46
ADaPT (Backend: GPT-4o mini) 2025.12		43
Local 2025.12		42
Centralized 2025.12		36
Local 2025.12		33
Pre-Trained 2025.12		15
ADaPT (Backend: DeepSeek-R1-Distill-Qwen-32B) 2025.12		13
Pre-Trained 2025.12		4
Pre-Trained 2025.12		1
Pre-Trained 2025.12		1
ADaPT (Backend: Claude-3 Haiku) 2025.12		0
Pre-Trained 2025.12		0