Share your thoughts, 1 month free Claude Pro on usSee more

Language Modeling on LAMBADA (Acc, Ppl)

86.9Accuracy

PaLM-2 L

Updated 8d ago

Evaluation Results

Method
PaLM-2 L 2023.11	86.9	-	-	-	-	-	-
Falcon-180B 2023.11	84.4	-	-	-	-	-	-
PaLM-2 M 2023.11	83.7	-	-	-	-	-	-
PaLM 2023.11	81.8	-	-	-	-	-	-
PaLM-2 S 2023.11	80.7	-	-	-	-	-	-
Falcon 2023.11	79.8	-	-	-	-	-	-
Inflection-1 2023.11	78.5	-	-	-	-	-	-
PaLM 2023.11	77.9	-	-	-	-	-	-
Chinchilla 2023.11	77.4	-	-	-	-	-	-
LLaMA-2 2023.11	77.4	-	-	-	-	-	-
Falcon 2023.11	77.3	-	-	-	-	-	-
MT-NLG 2023.11	76.6	-	-	-	-	-	-
GPT-3 2023.11	76.2	-	-	-	-	-	-
Llama3 8B 2025.09	75.6	3.1	-	-	-	-	-
Llama3 8B 2026.02	75.6	3.1	-	-	-	-	-
Llama 3.1 8B 2025.08	75.4	3.13	-	-	-	-	-
Falcon 2023.11	74.9	-	-	-	-	-	-
LoRA-Drop (50%) 2026.01	74.6	-	-	-	-	-	-
Gopher 2023.11	74.5	-	-	-	-	-	-
Matrix PCA 2025.08	73.9	3.35	-	-	-	-	-
CoSpaDi 2025.09	73.8	4.3	-	-	-	-	-
FlexiDepth 2026.01	72.1	-	-	-	-	-	-
COMPOT 2026.02	70.5	4.6	-	-	-	-	-
Llama 3.2 3B 2025.08	70.5	3.94	-	-	-	-	-
SVD-LLM 2025.08	70.5	4.63	-	-	-	-	-
Matrix PCA 2025.08	69	4.39	-	-	-	-	-
LoRA-Drop (25%) 2026.01	68.4	-	-	-	-	-	-
DLM 2026.01	66.58	-	-	-	-	-	-
DLM + extra token 2026.01	66.41	-	-	-	-	-	-
Dense 2026.02	65.55	-	-	-	-	-	-
SVD-LLM 2025.08	65.1	5.57	-	-	-	-	-
COMPOT 2026.02	64.3	6.6	-	-	-	-	-
SpanNorm 2026.01	64.2	5.8	-	-	-	-	-
SpanNorm 2026.01	63	6.5	-	-	-	-	-
Llama 3.2 1B 2025.08	62.9	5.73	-	-	-	-	-
SparseGPT 2026.02	62.76	-	-	-	-	-	-
PreNorm 2026.01	61.7	6.7	-	-	-	-	-
PreNorm 2026.01	61.4	6.9	-	-	-	-	-
CoSpaDi 2025.09	61.3	9.2	-	-	-	-	-
Matrix PCA 2025.08	59.9	6.65	-	-	-	-	-
Taylor 2026.02	59.23	-	-	-	-	-	-
SVD-LLM 2025.08	55.4	9.55	-	-	-	-	-
HGRN2 2024.09	55.4	8.8	-	-	-	-	-
GPT2-Medium + ACD 2023.05	55	15.4	-	-	-	-	-
Matrix PCA 2025.08	54.5	8.79	-	-	-	-	-
Learnable ResFormer 2024.10	53.5	21.2	-	-	-	-	-
Learnable ResFormer plus 2024.10	53.5	21.2	-	-	-	-	-
GSA 2024.09	52.7	9.8	-	-	-	-	-
LDDM-M 2025.10	52.4	-	-	-	-	-	-
Xfmr++ 2024.09	52.3	10.7	-	-	-	-	-
SpanNorm 2026.01	52.1	12	-	-	-	-	-
GHOST 2026.02	51.76	-	-	-	-	-	-
Mamba-3-MIMO-1.5B 2026.03	51.7	10.2	-	-	-	-	-
LIDAS 2025.12	51.41	-	-	-	-	-	-
PreNorm 2026.01	51.1	12.6	-	-	-	-	-
DLM + GA 2026.01	51.1	-	-	-	-	-	-
GPT2-XL 2023.05	51	10.6	-	-	-	-	-
LLM-Pruner 2025.09	51	11	-	-	-	-	-
LLM-Pruner 2026.02	51	11	-	-	-	-	-
MIDAS 2025.12	50.81	-	-	-	-	-	-
GPT-2 small 2026.03	50.42	-	-	-	-	-	-
GLA 2024.09	50.4	12.4	-	-	-	-	-
Nirvana (Ours) 2025.10	50.37	11.56	-	-	-	-	-
Transformer-1.5B 2026.03	50.3	11.1	-	-	-	-	-
GPT-2 small 2026.03	50.24	-	-	-	-	-	-
Baseline 2025.12	50.05	-	-	-	-	-	-
GPT-2 small 2026.03	49.93	-	-	-	-	-	-
RetNet 2024.09	49.6	11.9	-	-	-	-	-
Mamba-3-MIMO-880M 2026.03	49.5	11.8	-	-	-	-	-
HGRN2 2024.09	49.4	11.8	-	-	-	-	-
Mamba-3-SISO-1.5B 2026.03	49.4	10.9	-	-	-	-	-
Nirvana-noTrigger 2025.10	49.4	12.25	-	-	-	-	-
NeuTRENO 2024.10	49.2	20.7	-	-	-	-	-
GDN-1.5B 2026.03	49.2	10.9	-	-	-	-	-
LIME+1 2025.12	49	-	-	-	-	-	-
LN-Scaling 2025.12	48.94	-	-	-	-	-	-
Gated DeltaNet-H2 2025.10	48.76	12.55	-	-	-	-	-
DenseFormer 2024.10	48.2	20.1	-	-	-	-	-
SVD-LLM 2025.08	48.2	14.2	-	-	-	-	-
Identity ResFormer 2024.10	48.1	20.2	-	-	-	-	-
Mamba-2-1.5B 2026.03	47.8	12	-	-	-	-	-
Gated DeltaNet-H1 2025.10	47.73	12.12	-	-	-	-	-
Transformer 2024.10	47.7	18.8	-	-	-	-	-
GDN-880M 2026.03	47.6	12.9	-	-	-	-	-
GSA 2024.09	47.4	12.6	-	-	-	-	-
Mamba-3-SISO-880M 2026.03	47.2	12.9	-	-	-	-	-
DLM + extra token 2026.01	47.16	-	-	-	-	-	-
Xfmr++ 2024.09	47	15.3	-	-	-	-	-
PGM 2025.05	46.98	-	-	-	-	-	-
GPT-2 small 2026.03	46.89	-	-	-	-	-	-
Gated DeltaNet 2025.10	46.65	12.17	-	-	-	-	-
GLA 2024.09	46.4	15.4	-	-	-	-	-
Mamba 2024.09	46.4	13.6	-	-	-	-	-
Mamba2 2025.10	45.66	12.56	-	-	-	-	-
DLM + extra token 2026.01	45.24	-	-	-	-	-	-
DLM + GA 2026.01	45.1	-	-	-	-	-	-
BLIND-VALM-M 2024.09	45.08	37.63	-	-	-	-	-
Mamba-2-880M 2026.03	45	13.8	-	-	-	-	-
Samba 2025.10	44.94	13.29	-	-	-	-	-
COMPOT 2026.02	44.9	22	-	-	-	-	-

Showing 100 of 290 rows