Share your thoughts, 1 month free Claude Pro on usSee more

Language Modeling on LAMBADA (Acc, Ppl)

86.9Accuracy

PaLM-2 L

Updated 4d ago

Evaluation Results

Method
PaLM-2 L 2023.11	86.9	-	-	-	-	-	-
Falcon-180B 2023.11	84.4	-	-	-	-	-	-
PaLM-2 M 2023.11	83.7	-	-	-	-	-	-
PaLM 2023.11	81.8	-	-	-	-	-	-
PaLM-2 S 2023.11	80.7	-	-	-	-	-	-
Falcon 2023.11	79.8	-	-	-	-	-	-
Inflection-1 2023.11	78.5	-	-	-	-	-	-
PaLM 2023.11	77.9	-	-	-	-	-	-
Chinchilla 2023.11	77.4	-	-	-	-	-	-
LLaMA-2 2023.11	77.4	-	-	-	-	-	-
Falcon 2023.11	77.3	-	-	-	-	-	-
MT-NLG 2023.11	76.6	-	-	-	-	-	-
GPT-3 2023.11	76.2	-	-	-	-	-	-
Llama3 8B 2025.09	75.6	3.1	-	-	-	-	-
Llama3 8B 2026.02	75.6	3.1	-	-	-	-	-
Llama 3.1 8B 2025.08	75.4	3.13	-	-	-	-	-
Falcon 2023.11	74.9	-	-	-	-	-	-
LoRA-Drop (50%) 2026.01	74.6	-	-	-	-	-	-
Gopher 2023.11	74.5	-	-	-	-	-	-
Matrix PCA 2025.08	73.9	3.35	-	-	-	-	-
CoSpaDi 2025.09	73.8	4.3	-	-	-	-	-
FlexiDepth 2026.01	72.1	-	-	-	-	-	-
COMPOT 2026.02	70.5	4.6	-	-	-	-	-
Llama 3.2 3B 2025.08	70.5	3.94	-	-	-	-	-
SVD-LLM 2025.08	70.5	4.63	-	-	-	-	-
Matrix PCA 2025.08	69	4.39	-	-	-	-	-
LoRA-Drop (25%) 2026.01	68.4	-	-	-	-	-	-
DLM 2026.01	66.58	-	-	-	-	-	-
DLM + extra token 2026.01	66.41	-	-	-	-	-	-
Dense 2026.02	65.55	-	-	-	-	-	-
SVD-LLM 2025.08	65.1	5.57	-	-	-	-	-
COMPOT 2026.02	64.3	6.6	-	-	-	-	-
SpanNorm 2026.01	64.2	5.8	-	-	-	-	-
SpanNorm 2026.01	63	6.5	-	-	-	-	-
Llama 3.2 1B 2025.08	62.9	5.73	-	-	-	-	-
SparseGPT 2026.02	62.76	-	-	-	-	-	-
PreNorm 2026.01	61.7	6.7	-	-	-	-	-
PreNorm 2026.01	61.4	6.9	-	-	-	-	-
CoSpaDi 2025.09	61.3	9.2	-	-	-	-	-
Matrix PCA 2025.08	59.9	6.65	-	-	-	-	-
Taylor 2026.02	59.23	-	-	-	-	-	-
SVD-LLM 2025.08	55.4	9.55	-	-	-	-	-
HGRN2 2024.09	55.4	8.8	-	-	-	-	-
GPT2-Medium + ACD 2023.05	55	15.4	-	-	-	-	-
Matrix PCA 2025.08	54.5	8.79	-	-	-	-	-
NoPE 2025.11	53.8	8.59	-	-	-	-	-
Selective RoPE 2025.11	53.8	8.5	-	-	-	-	-
Learnable ResFormer 2024.10	53.5	21.2	-	-	-	-	-
Learnable ResFormer plus 2024.10	53.5	21.2	-	-	-	-	-
RoPE 2025.11	53	8.88	-	-	-	-	-
GSA 2024.09	52.7	9.8	-	-	-	-	-
LDDM-M 2025.10	52.4	-	-	-	-	-	-
Xfmr++ 2024.09	52.3	10.7	-	-	-	-	-
SpanNorm 2026.01	52.1	12	-	-	-	-	-
GHOST 2026.02	51.76	-	-	-	-	-	-
Mamba-3-MIMO-1.5B 2026.03	51.7	10.2	-	-	-	-	-
LIDAS 2025.12	51.41	-	-	-	-	-	-
PreNorm 2026.01	51.1	12.6	-	-	-	-	-
DLM + GA 2026.01	51.1	-	-	-	-	-	-
GPT2-XL 2023.05	51	10.6	-	-	-	-	-
LLM-Pruner 2025.09	51	11	-	-	-	-	-
LLM-Pruner 2026.02	51	11	-	-	-	-	-
Gated DeltaNet-2 2026.05	50.9	10.43	-	-	-	-	-
MIDAS 2025.12	50.81	-	-	-	-	-	-
GPT-2 small 2026.03	50.42	-	-	-	-	-	-
GLA 2024.09	50.4	12.4	-	-	-	-	-
Oryx-TM (Mamba-2) 2026.05	50.4	10.5	-	-	-	-	-
Nirvana (Ours) 2025.10	50.37	11.56	-	-	-	-	-
Transformer-1.5B 2026.03	50.3	11.1	-	-	-	-	-
Parallax 2026.05	50.26	10.8	-	-	-	-	-
GPT-2 small 2026.03	50.24	-	-	-	-	-	-
Oryx-TM (Transformer) 2026.05	50.2	11.1	-	-	-	-	-
Baseline 2025.12	50.05	-	-	-	-	-	-
Oryx-TG (Gated DeltaNet) 2026.05	50	10.6	-	-	-	-	-
GPT-2 small 2026.03	49.93	-	-	-	-	-	-
Transformer 2026.05	49.9	11.4	-	-	-	-	-
Gated DeltaNet 2026.05	49.9	10.6	-	-	-	-	-
Oryx-TG (Transformer) 2026.05	49.9	10.9	-	-	-	-	-
Mamba-3 (MIMO) 2026.05	49.82	10.92	-	-	-	-	-
Gated DeltaNet 2026.05	49.62	11.89	-	-	-	-	-
RetNet 2024.09	49.6	11.9	-	-	-	-	-
Softmax 2025.04	49.56	11.38	-	-	-	-	-
Softmax 2025.04	49.56	11.38	-	-	-	-	-
Parallax 2026.05	49.54	10.85	-	-	-	-	-
Mamba-3-MIMO-880M 2026.03	49.5	11.8	-	-	-	-	-
Softmax 2025.04	49.43	11.38	-	-	-	-	-
HGRN2 2024.09	49.4	11.8	-	-	-	-	-
Mamba-3-SISO-1.5B 2026.03	49.4	10.9	-	-	-	-	-
Nirvana-noTrigger 2025.10	49.4	12.25	-	-	-	-	-
KDA 2026.05	49.21	10.66	-	-	-	-	-
NeuTRENO 2024.10	49.2	20.7	-	-	-	-	-
GDN-1.5B 2026.03	49.2	10.9	-	-	-	-	-
Mamba-3 (SISO) 2026.05	49.19	10.65	-	-	-	-	-
LIME+1 2025.12	49	-	-	-	-	-	-
LN-Scaling 2025.12	48.94	-	-	-	-	-	-
Gated DeltaNet-H2 2025.10	48.76	12.55	-	-	-	-	-
Gated DeltaNet 2026.05	48.71	10.82	-	-	-	-	-
Softmax 2025.04	48.65	11.74	-	-	-	-	-
Mamba-2 2026.05	48.6	11.2	-	-	-	-	-
Transformer 2026.05	48.32	13.72	-	-	-	-	-

Showing 100 of 434 rows