Our new X account is live! Follow @wizwand_team for updates

Natural Language Understanding on GLUE (Efficiency Metrics)

3.06SST-2 Speedup

DeeBERT-base

Updated 4d ago

Evaluation Results

Method
DeeBERT-base 2023.05	3.06	0.8	1.25	25.98	1.98	6.86	1.71	5.39	3.19	1.4	1.21	59.93	1.54	18	1.54	18
A2T 2023.05	3.06	3.1	1.25	10.16	1.32	18.5	1.63	27	3.35	1.8	1.66	21.66	1.44	26.6	1.43	25.8
HotFlip 2023.05	2.77	0.69	1.2	18.31	1.69	1.23	1.65	18.87	7.78	0.7	1.4	0	1.32	3.9	1.37	0.4
DeeRoBERTa-base 2023.05	2.74	0	1.3	3.26	1.86	0.49	1.61	2.3	2.35	1.4	1.42	0	1.38	0.72	1.36	1.5
A2T 2023.05	2.7	0.34	1.25	10.16	1.89	2.21	1.63	1.8	2.52	3.4	1.37	0.4	1.35	1.9	1.31	4.2
PWWS 2023.05	2.44	4.01	1.22	10.93	1.4	5.85	1.71	20.32	3.35	1.5	1.71	16.61	1.34	37.1	1.36	37.5
DeeBERT-base 2023.05	2.4	2.87	1.36	8.72	1.51	20.34	1.69	1.23	2.68	3.6	1.19	61.01	1.42	25.6	1.42	25.6
PWWS 2023.05	2.37	0.34	1.22	10.93	1.71	5.39	1.56	2.9	2.51	1.3	1.38	0.72	1.3	4.7	1.37	2.3
A2T 2023.05	2.35	6.77	1.24	13.23	1.27	51.96	1.58	26.72	2.7	5.2	1.18	65.34	1.3	43.8	1.3	39.3
TextFooler 2023.05	2.35	0.92	1.22	11.22	1.63	8.09	1.51	3.7	2.44	3.3	1.34	0.7	1.3	4.7	1.33	3.2
A2T 2023.05	2.34	0.8	1.24	13.23	1.52	6.62	1.46	5.7	2.15	6.5	1.29	10.47	1.33	3.4	1.32	5.2
DeeRoBERTa-base 2023.05	2.31	0.34	1.29	4.6	1.63	0.74	1.61	0.4	2.09	2.3	1.32	3.97	1.4	0	1.35	1.6
TextFooler 2023.05	2.3	5.62	1.22	11.22	1.31	12.94	1.64	29.6	3.19	2.1	1.56	28.52	1.36	37.5	1.32	38.9
TextBugger 2023.05	2.29	0.57	1.2	15.92	1.61	8.09	1.5	4	2.78	1.6	1.41	0	1.19	34.2	1.38	1
HotFlip 2023.05	2.28	1.49	1.19	22.15	1.58	2.21	1.58	26.72	6.48	1.3	1.29	9.39	1.31	5.2	1.34	0.7
TextBugger 2023.05	2.25	5.85	1.2	15.92	1.31	15.72	1.63	21.32	3.24	2.4	1.54	31.05	1.33	39.1	1.32	41.8
HotFlip 2023.05	2.21	7.11	1.2	18.31	1.31	12.18	1.63	8.09	3.13	2.6	1.71	14.08	1.42	25.6	1.42	25.6
PWWS 2023.05	2.13	0.92	1.21	14.29	1.5	8.33	1.48	1.3	2.15	2.8	1.28	12.27	1.31	6.3	1.37	1.3
TextFooler 2023.05	2.09	1.26	1.21	12.08	1.59	3.43	1.4	7.5	2.08	5	1.26	17.69	1.31	6.3	1.32	2.8
TextBugger 2023.05	2.07	1.95	1.19	19.27	1.52	6.37	1.45	7.9	2.32	3.8	1.26	17.69	1.33	3.4	1.36	2
PWWS 2023.05	1.94	10.32	1.21	14.29	1.31	12.18	1.46	39.4	2.77	3.7	1.18	66.79	1.25	48.5	1.25	48.5
TextBugger 2023.05	1.83	14.11	1.19	19.27	1.43	31.62	1.41	44.1	2.66	5.4	1.18	67.15	1.37	34.2	1.33	39.1
TextFooler 2023.05	1.83	15.48	1.21	12.08	1.31	43.38	1.41	42.9	2.57	6.3	1.18	67.15	1.24	53.9	1.24	53.9
SAME-Word 2023.05	1.79	10.09	1	98.75	1.39	21.57	1.11	82	1.43	47.5	1.28	15.88	1.12	56.1	1.17	42.1
HotFlip 2023.05	1.78	17.09	1.19	22.15	1.27	22.72	1.61	8.09	2.58	7	1.18	67.15	1.34	37.1	1.34	37.1
SAME-Word 2023.05	1.63	16.74	1.01	95.11	1.52	14.95	1.12	81.9	1.33	52.9	1.14	49.82	1.17	39.8	1.21	28.8
SAME-Char 2023.05	1.61	12.5	1	98.18	1.21	49.02	1.24	35	1.47	47.3	1.22	26.35	1.33	2.9	1.17	42.1
SAME-Char 2023.05	1.53	18.12	1.01	94.53	1.26	43.87	1.2	39.2	1.34	55.5	1.07	73.29	1.13	52.3	1.13	52.3
SAME-Char 2023.05	1.21	59.17	1.01	95.69	1.08	81.37	1.11	82	1.53	45.5	1.07	87	1.02	94.4	1.02	94.8
SAME-Word 2023.05	1.19	62.96	1.02	92.91	1.09	77.94	1.18	74.4	1.36	50.6	1.08	84.48	1.01	96	1.02	95.4
SAME-Char 2023.05	1.13	70.87	1.01	96.84	1.02	93.14	1.08	88.4	1.37	55.7	1.02	96.03	1.01	96.9	1.01	95.7
SAME-Word 2023.05	1.1	76.72	1.02	94.15	1.02	94.61	1.12	81.9	1.25	63.1	1.02	97.11	1.01	96.5	1.01	97.7