Our new X account is live! Follow @wizwand_team for updates

Generative Question Answering on Bolmo Evaluation Suite GenQA 7B

81.6GenQA Average

Llama 3.1 70B

Updated 4d ago

Evaluation Results

Method
Llama 3.1 70B 2025.12	81.6	88.4	91.7	79.6	92.4	78.3	84	53.1	92.9	73.9
Marin 32B 2025.12	80.3	87.2	90.5	76.7	91.1	76.5	80.5	55.1	94.4	70.7
Olmo 3 32B 2025.12	79.8	84.8	90.3	75.7	93.5	80.9	75.3	49	94.5	74.1
OLMo 2 32B 2025.12	79.1	87.5	89.4	77	88.7	76.3	79.1	51.4	94	68.7
Mistral 3.1 24B 2025.12	78	86.2	90.8	79.3	91.9	74.9	80.3	45.1	92.6	61.1
LLM 360 K2V2 70B 2025.12	77.1	87.6	88.9	76.8	90.6	69.7	79.8	47.6	91.2	61.5
Seed 36B 2025.12	76	84.8	89.3	76.1	96	76.1	77.4	30.7	89.1	64.4
Marin 8B 2025.12	75.9	84	88.6	73.9	85.6	73	72.7	42.6	93.4	69.5
Gemma 2 9B 2025.12	75.6	81.8	88.8	76.3	89.3	68.2	75.1	40.4	88.8	71.5
Apertus 70B 2025.12	75	84.5	87.7	74.8	87.5	56.3	77.2	43.1	90.7	72.8
Gemma 3 27B 2025.12	73.5	86	91.3	77.5	94.9	75.9	82.1	49.2	92.4	12.4
Llama 3.1 8B 2025.12	73.1	81.5	87.3	75.5	88	59.5	70.9	36.7	89.2	69
Gemma 2 27B 2025.12	72.9	86.7	90.8	76.9	93.2	73.2	80.7	47.1	93	14.9
Olmo 3 7B 2025.12	72.5	77.7	85.7	68.9	89.5	71.5	60.4	32.6	93.5	72.8
OLMo 2 7B 2025.12	72.4	82.2	87.4	70.5	82.2	61.5	70.8	37.4	91.5	68.3
Nemo-Nano 9B 2025.12	71.8	80.2	86.2	67.9	91.4	71.4	64.9	31.2	92.3	60.4
MiMo 7B 2025.12	71.4	80.6	86.5	73.1	89.7	69.3	65.6	33.1	90.3	54.4
Qwen2 7B 2025.12	71.1	80.5	86.4	73	93.5	57.2	65.1	33.8	89.2	61.6
Apertus 8B 2025.12	69	81	85.8	70.9	83.8	37.1	70.1	35	89.6	67.4
Qwen 2.5 32B 2025.12	68.5	86.3	87.5	76.2	94.2	53.7	74	39.3	64.9	40.4
Granite 3.3 8B 2025.12	67.8	83.7	89.4	76	88.7	38.4	69.7	37	89.6	37.8
Qwen 2.5 7B 2025.12	67.5	81	86	70.3	91.4	56.7	63	31.2	87	40.5
Gaperon 24B 2025.12	65.3	75.2	80.3	58.3	83.2	59.4	58.9	33.5	89.3	49.8
Gaperon 8B 2025.12	63.3	73.9	76.4	67	80.5	54.9	55.5	28.8	86	46.7
Olmo 3 7B 2025.12	0.724	0.778	0.857	0.68	0.9	0.715	0.603	0.326	0.935	0.727
TFree-Hat 7B 2025.12	0.713	0.828	0.882	0.705	0.896	0.486	0.683	0.343	0.886	0.711
Bolmo 7B 2025.12	0.709	0.788	0.855	0.711	0.896	0.652	0.568	0.286	0.916	0.705
BLT 7B 2025.12	0.684	0.811	0.882	0.728	0.845	0.388	0.673	0.292	0.852	0.686
EvaByte 6.5B 2025.12	0.414	0.701	0.782	0.629	0.827	0.078	0.131	0.054	0.359	0.167