Share your thoughts, 1 month free Claude Pro on usSee more

Natural Language Understanding on GLUE (val) - Subset (CoLA, MNLI, MRPC, QNLI, QQP, RTE, SST-2, STS-B)

82.4CoLA Score

ProbSurgery

Updated 22d ago

Evaluation Results

Method
ProbSurgery 2026.05	82.4	69.1	78.3	80.6	89.8	83.4	81.2	92.5	82.2
Fine-tuned (STL) 2026.05	80.2	88.5	89.2	94.4	87.2	91.7	95.2	90.9	89.6
Individual 2023.12	80.2	88.5	89.2	94.4	87.2	91.7	95.2	90.9	89.6
FEATCAL 2026.05	79.2	87.8	89	93.9	86.9	88.4	95.3	90.8	88.9
ProbSurgery 2026.05	77.2	87.6	85	93.8	86	88.4	95.2	87.8	87.6
Ties-Merging 2023.12	77.1	85.1	86.3	93.9	86	87.7	95.1	88	87.4
Task Arithmetic 2023.12	76.9	85.4	85.3	93.9	85.8	88.1	95.2	87.8	87.3
Task Arithmetic 2026.05	76.8	85.4	85.3	94	85.8	88.1	95.2	87.7	87.3
LW AM 2023.12	76.7	87.6	84.8	93.8	85.9	88.1	95.2	88.6	87.6
Concrete TA 2023.12	76.6	86.4	86	93.9	85.9	88.4	95.2	87.9	87.5
LW Concrete AM 2023.12	76.1	87.7	85.5	93.8	85.9	88.1	95.4	87.1	87.5
Surgery 2026.05	76	87.9	86	93.9	86.2	89.9	95.2	89	88
Fine-tuned (STL) 2026.05	75	83.4	87.5	91.5	85.4	85.9	93.6	88.7	86.4
Weight Averaging 2023.12	74.6	84.3	84.1	92.8	86.3	87.4	94.8	88	86.5
Pre-trained 2026.05	73.7	56.6	82.4	91.1	85.5	85.6	94.3	87.5	82.1
Pre-trained 2023.12	73.7	56.6	82.4	91.1	85.5	85.6	94.3	87.5	82.1
FEATCAL 2026.05	72.2	82.6	85.1	91.1	84.7	85.2	93	87.7	85.2
Surgery 2026.05	70.8	82.4	82.4	89.8	84.2	83	92.1	85.2	83.7
Task Arithmetic 2026.05	70.5	57.8	78.4	90.2	83.6	80.5	92.3	77.8	78.9
Pre-trained 2026.05	69.1	56.5	76.2	88.4	82.1	80.1	91.2	62.2	75.7
BOFT 2026.05	68.2	90.27	87.07	94.07	89.73	81.43	96	91.13	87.24
FT 2025.06	68	90.2	91	94.7	92.2	87	96.4	92.4	89
LoRA 2025.06	68	90.6	84	94.8	91.6	87	95.7	91.9	88
OFT 2026.05	67.63	90.27	87.93	94.3	89.77	84.27	95.83	91	87.63
LoCO 2026.05	65.56	90.2	88.28	94.36	90.08	86.1	96.14	91.28	87.75
VeRA 2025.06	64	88.8	89.4	93.1	87.62	83	95.1	91.5	86.6
MetaTT-4D 2025.06	64	90	90.1	94.4	76	84.8	95.3	92.2	86
MetaTT-4D 2025.06	62.8	89.6	88.6	93.8	88.5	84.2	95.2	91.8	86.8
LoTR 2025.06	61.3	84.7	88	92	86.9	67	93.3	91	83
LoTR 2025.06	61.3	90.3	89	94.8	89.2	84	95.9	91.6	87
LoRA 2025.06	61.1	87.3	88	91.3	90.1	73	94.2	90.7	84.5
FT 2025.06	61	87.6	89.3	92.6	91.9	79	94.1	90.4	85.7
LoTR 2025.06	61	84.6	89	92.1	86.8	71	93.4	90.9	83.6
MetaTT-4D 2025.06	61	85.9	88.9	77	88.5	77.5	72	90.1	80
LoRETTAadp 2025.06	61	89.69	88.1	94.08	89.6	72	95.5	-	84.3
MetaTT-4D 2025.06	59.7	85.5	88.6	91	87.8	74.2	92.3	89.9	83.6
LoRTA 2025.06	58.9	88.4	87.3	94	88.1	66.6	95.3	91.1	83.7
AdamW 2025.10	58.8	84.11	85.78	91.89	90.63	68.23	92.89	89.32	82.71
MetaTT-4D 2025.06	58.8	84.2	87.6	90.4	86.9	72.9	92	89.1	82.7
LoRTA 2025.06	58.6	86.1	88	92.2	89	75	93.6	89.3	84
VeRA 2025.06	58	81	87.2	89.6	85.87	73.4	92.2	88.7	82
LoTR 2025.06	58	85.2	88	92.5	87.6	53	93.8	89.8	81
LoRETTAadp 2025.06	57.9	84.6	86.4	92	88	70.3	93.3	-	81.8
Muon 2025.10	57.53	80.68	84.56	90.55	89.58	68.95	91.97	89.03	81.61
LoRTA 2025.06	55.9	84.1	86.9	91.1	86.7	70.2	93	86.6	81.8
RMSSpectral 2025.10	53.27	77.76	85.05	86.82	85.85	65.34	89.68	83.44	78.4
AdaMuon 2025.10	51.74	78.36	84.8	87.39	85.84	66.79	90.37	84.27	78.69
HRA 2026.05	51.67	90.2	88.03	93.57	90.2	84.5	95.7	91.25	85.64
Muon 2025.10	49.92	72.37	85.78	85.01	83.22	66.06	88.65	83.92	76.87
RMSSpectral-SANIA 2025.10	49.9	77.8	84.8	87.81	85.71	62.09	90.6	84.22	77.87
RMSSpectral 2025.10	48	81.16	78.19	87.94	88.01	-	-	-	76.66
AdaMuon 2025.10	47.7	81.09	82.6	87.57	87.82	-	-	-	77.36
AdamW 2025.10	47.59	73.52	83.09	85.67	84.98	63.9	89.56	84.73	76.63
RMSSpectral-SANIA 2025.10	45.87	81.22	81.37	87.88	87.82	-	-	-	76.83