Share your thoughts, 1 month free Claude Pro on usSee more

Constraint Following on ConstraintRot (test)

0Violation Rate

DeepSeek-V4 (control)

Updated 1mo ago

Evaluation Results

Method	Links
DeepSeek-V4 (control) 2026.06		0
DeepSeek-V4 (defense_pin) 2026.06		0
DeepSeek-V4 (defense_pin_attack) 2026.06		0
GLM-5.1 (control) 2026.06		0
GLM-5.1 (compaction) 2026.06		0
GLM-5.1 (defense_pin) 2026.06		0
GLM-5.1 (defense_pin_attack) 2026.06		0
Qwen3.6 (control) 2026.06		0
Qwen3.6 (defense_pin) 2026.06		0
Qwen3.6 (defense_pin_attack) 2026.06		0
Kimi-K2.5 (control) 2026.06		0
Kimi-K2.5 (defense_pin) 2026.06		0
Kimi-K2.5 (defense_pin_attack) 2026.06		0
Claude-4.6 (control) 2026.06		0
Claude-4.6 (attack_volume) 2026.06		0
Claude-4.6 (attack_inject) 2026.06		0
Claude-4.6 (defense_pin) 2026.06		0
Claude-4.6 (defense_pin_attack) 2026.06		0
GPT-5.4m (control) 2026.06		0
GPT-5.4m (defense_pin) 2026.06		0
GPT-5.4m (defense_pin_attack) 2026.06		0
Gemini-3.5f (control) 2026.06		0
Gemini-3.5f (attack_inject) 2026.06		0
Gemini-3.5f (defense_pin) 2026.06		0
Gemini-3.5f (defense_pin_attack) 2026.06		0
Gemini-3.5f (compaction) 2026.06		4
GLM-5.1 (attack_volume) 2026.06		7
Claude-4.6 (compaction) 2026.06		19
Gemini-3.5f (attack_volume) 2026.06		19
GLM-5.1 (attack_inject) 2026.06		22
Qwen3.6 (attack_volume) 2026.06		26
Qwen3.6 (compaction) 2026.06		30
Qwen3.6 (attack_inject) 2026.06		33
Claude-4.6 (floor) 2026.06		33
DeepSeek-V4 (floor) 2026.06		37
GPT-5.4m (attack_volume) 2026.06		37
GPT-5.4m (attack_inject) 2026.06		37
Kimi-K2.5 (attack_inject) 2026.06		41
GPT-5.4m (compaction) 2026.06		41
GLM-5.1 (floor) 2026.06		44
Kimi-K2.5 (attack_volume) 2026.06		44
DeepSeek-V4 (attack_volume) 2026.06		48
Qwen3.6 (floor) 2026.06		48
Kimi-K2.5 (floor) 2026.06		48
GPT-5.4m (floor) 2026.06		56
DeepSeek-V4 (compaction) 2026.06		59
DeepSeek-V4 (attack_inject) 2026.06		59
Kimi-K2.5 (compaction) 2026.06		59
Gemini-3.5f (floor) 2026.06		63