Share your thoughts, 1 month free Claude Pro on usSee more

Forgetting-aware Instruction Tuning on Magicoder Stability and Plasticity suites (test)

54.27ARC-C

Pre-trained

Updated 2mo ago

Evaluation Results

Method
Pre-trained 2026.05	54.27	80.09	72.62	86.2	73.3	60.98	67.14
DualSFT 2026.05	54.01	79.64	70.33	84.61	72.15	73.75	72.95
DualSFT-Data 2026.05	53.66	80.03	70.75	85.04	72.37	74.21	73.29
DualSFT-Param 2026.05	53.52	78.72	70.26	84.22	71.68	75.13	73.41
TALR 2026.05	53.45	79.7	70.72	85.73	72.4	75.36	73.88
FLOW 2026.05	53.2	79.56	70.62	85.28	72.17	73.1	72.64
NanoAdam 2026.05	52.87	79.03	70.22	84.22	71.59	73.28	72.43
FAPM 2026.05	52.86	79.48	70.18	83.88	71.6	70.85	71.23
STM 2026.05	52.8	79.18	70.38	84.55	71.73	73.12	72.43
DavIR 2026.05	52.4	78.42	70.02	83.6	71.11	73.4	72.26
LoRA 2026.05	51.51	76.62	69.47	81.94	69.89	70.29	70.09
Pre-trained 2026.05	51.45	79.16	59.61	37	56.81	35.37	46.09
DualSFT 2026.05	51.14	79.01	59.16	35.62	56.23	56.76	56.5
DualSFT-Param 2026.05	51.13	78.54	58.14	36.34	56.04	57.52	56.78
DualSFT-Data 2026.05	51.01	78.52	58.33	36.55	56.1	56.94	56.52
TALR 2026.05	50.96	78.84	58.94	36.7	56.36	57.44	56.9
FLOW 2026.05	50.92	78.82	58.92	36.82	56.37	55.7	56.04
NanoAdam 2026.05	50.78	78.2	58.33	33.12	55.11	56.38	55.74
STM 2026.05	50.62	78.48	58.58	36.1	55.95	55.92	55.94
FAPM 2026.05	50.6	78.5	58.6	35.9	55.9	52.2	54.05
DavIR 2026.05	50.25	78.04	58.01	33.7	55	56.24	55.62
Standard SFT 2026.05	49.22	74.31	66.14	78.64	67.08	75.56	71.32
LoRA 2026.05	49.15	75.57	56.71	31.63	53.27	54.66	53.96
Standard SFT 2026.05	47.17	76.39	53.87	24.03	50.37	58.1	54.23
Pre-trained 2026.05	43	76.61	56.52	27.45	50.9	28.66	39.78
FLOW 2026.05	42.94	76.23	56.91	27.28	50.84	42.28	46.56
DualSFT 2026.05	42.94	76.23	56.09	26.88	50.54	43.67	47.1
DavIR 2026.05	42.71	74.92	53.09	24.65	48.84	43.28	46.06
NanoAdam 2026.05	42.66	74.68	52.71	24.39	48.61	43.54	46.08
DualSFT-Data 2026.05	42.66	76.44	55.28	26.54	50.23	44.05	47.14
LoRA 2026.05	42.61	74.43	51.87	23.29	48.05	41.46	44.76
FAPM 2026.05	42.55	76.01	53.86	26.01	49.61	40.99	45.3
DualSFT-Param 2026.05	42.54	76.19	54.01	26.27	49.75	44.52	47.14
TALR 2026.05	42.49	76.34	56.41	27.54	50.7	44.18	47.44
STM 2026.05	42.38	76.12	56.04	27.08	50.41	42.65	46.53
Standard SFT 2026.05	39.99	72.09	48.18	20.69	45.24	44.68	44.96