Share your thoughts, 1 month free Claude Pro on usSee more

Language Modeling and Reasoning Evaluation on Open LLM Leaderboard

82.8ARC

DPO

Updated 1mo ago

Evaluation Results

Method	Links
DPO 2024.06		82.8	50.6	74.3	52	65.8	63.1	-
KTO 2024.06		82.7	48.7	74.9	53.9	65.4	63	-
AOT paired 2024.06		82.5	50.8	74.4	53.1	66.1	62.9	-
AOT unpaired 2024.06		82.5	51.1	74.4	51.8	66.2	62.8	-
IPO 2024.06		82.4	46.5	74	52.3	65.1	63	-
Merlinite-7B 2024.06		81.6	42	73.9	45.2	63.2	62.6	-
FT 2024.02		73.34	47.07	74.03	16.22	79.95	46.55	56.19
LoRA 2024.02		72.07	44.84	75.85	15.01	78.6	44.42	55.13
RED 2024.02		72.04	47.59	72.85	11.9	77.86	42.27	54.09
FP16 2024.04		59.4	37.4	76.6	22.5	82.1	55.7	55.6
FP16 2024.04		59.4	37.4	76.6	22.5	82.1	55.7	55.6
FP16 2024.04		59.4	37.4	76.6	22.5	82.1	55.7	55.6
CherryQ 2024.04		58.6	38.8	77	21	82	54.6	55.3
QAT 2024.04		57.9	38.9	75.7	19.6	81.9	54.2	54.7
GPTQ 2024.04		57.4	36.1	76.8	20.4	81.5	54.6	54.5
CherryQ 2024.04		57.3	38	76.2	18.4	81.1	53.5	54.1
CherryQ 2024.04		56.7	38.9	75.4	17.8	81	52.5	53.7
GPTQ 2024.04		56.5	36	74.4	16.4	79.2	52.4	52.5
QAT 2024.04		55.5	39	75.1	16.8	80.7	52.9	53.3
QAT 2024.04		55.3	38.8	75.5	16	80.7	51.9	53
GPTQ 2024.04		54.1	34.9	75.4	15.6	79.1	50.3	51.6
FP16 2024.04		53.2	38.8	74	14.5	78.6	46.7	51
FP16 2024.04		53.2	38.8	74	14.5	78.6	46.7	51
FP16 2024.04		53.2	38.8	74	14.5	78.6	46.7	51
QAT 2024.04		52	39	72.2	10.6	77.5	43.7	49.2
GPTQ 2024.04		52	39.1	72.9	11.1	77.6	43.8	49.4
CherryQ 2024.04		51.5	39.5	73.5	12.9	77.8	44.4	49.9
CherryQ 2024.04		50.6	38.6	71.8	10.4	77	43.9	48.7
CherryQ 2024.04		49.7	38.1	72.4	8.8	76.3	41.6	47.8
QAT 2024.04		49.2	37.3	71.6	7.3	75.5	40.6	46.9
GPTQ 2024.04		48.6	38.8	71.7	8.1	73.9	39.4	46.8
GPTQ 2024.04		48.6	39.1	70.8	5.4	72.9	38.2	45.8
QAT 2024.04		48.2	37.7	70.8	6.7	75.4	39	46.3