Share your thoughts, 1 month free Claude Pro on usSee more

Counterfactual Reasoning on CounterBench

80.8Basic Score

Deepseek-V3

Updated 4d ago

Evaluation Results

Method	Links
Deepseek-V3 2025.02		80.8	70.4	76.4	77.6	63.5	74.2
GPT-4o 2025.02		80.4	72.4	80.8	81.6	60.5	75.8
Gemini-1.5-flash 2025.02		75.2	65.6	67.2	76	53	68
Gemini-1.5-flash 2025.02		72.4	70.8	72.4	78.4	58.5	71
Gemini-1.5-flash-8b 2025.02		66.8	67.2	65.2	65.2	58.5	64.8
Claude-3.5 (Haiku) 2025.02		60.4	65.6	67.2	66	61	64.2
Claude-3 (Sonnet) 2025.02		59.2	52	64.4	60	65.5	59
GPT-4o mini 2025.02		57.2	66.4	60	63.2	50	59.8
GPT-3 (Davinci-002) 2025.02		56.8	50.2	48.8	51.6	52.5	51.9
GPT-3 (Davinci-002) 2025.02		51.2	41.9	51.2	51.6	50.5	49.3
GPT-4o 2025.02		50.4	54.4	50.4	54.8	54	52.8
Claude-3 (Sonnet) 2025.02		50.4	48.8	50	50.8	59.5	51.6
Deepseek-V3 2025.02		50.4	50.4	50	50	60.5	51.9
GPT-3 (Babbage-002) 2025.02		50	50	50	50	47.5	49.6
GPT-4o mini 2025.02		50	50	50	50	52.5	50.4
Gemini-1.5-flash-8b 2025.02		50	50	50	50	52.5	50.4
GPT-3.5 2025.02		49.6	51.2	50.4	50	52	50.6
GPT-3.5 2025.02		43.6	50.4	53.6	50	47.5	49.1
Claude-3.5 (Haiku) 2025.02		28.4	24	43.6	54	51	39.8
GPT-3 (Babbage-002) 2025.02		3.6	7.6	1.2	19.6	18.5	9.8