Our new X account is live! Follow @wizwand_team for updates

NLU and Reasoning on BIG-bench (unseen)

86.96Known Unknowns Score

FLIPPED

Updated 4d ago

Evaluation Results

Method	Links
FLIPPED 2022.10		86.96	42.5	53.23	52.57	48.49	60	64.93	26.87	65.57	50.39	54.34	64.56	55.08	55.17	46.88
FLIPPED 2022.10		71.74	41.7	53.19	47.43	47.93	45	61.64	19.01	57.07	50.56	46.58	59.82	53.33	48.57	25
PALM 2022.10		67.39	42.2	69	94.86	57.2	61.67	80.02	37.3	70.4	52.2	77.63	34.42	47.5	60.8	59.38
T0 2023.02		65.22	33.67	54.67	42.86	57.33	51.67	71.72	18.33	57.33	48.33	52.97	54.67	55	51.06	-
DIRECT 2022.10		63.04	35.9	53.28	50.29	47.15	33.33	58.14	22.01	63.83	49.85	50.23	38.06	44.33	43.75	3.13
GPT-3 2023.02		60.87	31.2	52.3	22.57	51.4	31.67	26.78	15.9	12.3	50.5	47.95	23.42	51.5	37.57	-
GPT-3 2022.10		60.87	31.2	52.3	32.57	51.4	31.67	26.78	15.9	12.3	50.5	47.95	23.42	51.5	38.23	46.88
Cos PE 2023.02		58.7	30.7	42.36	51.43	46.72	66.67	72.92	25.95	46.55	50	47.03	42.4	44.33	48.13	-
T0 2022.10		58.7	38.3	52.75	29.71	53.69	43.33	69.29	20.2	64.73	51.81	50	54.86	52.11	46.79	15.63
PaLM 2023.02		56.52	32.1	64	56	49.1	25	59.26	20.1	14.1	49.9	47.47	24.22	45.3	41.77	-
PALM 2022.10		56.52	32.1	64	56	49.1	25	59.26	20.1	14.1	49.9	47.49	24.22	45.3	42.14	46.88
CHANNEL 2022.10		52.17	30.9	53.01	16.57	51.06	71.67	35.67	11.55	15.73	50.43	47.79	25.73	55.36	38.07	15.63
GPT-3 2022.10		50	31.1	57.1	58.29	49.4	31.67	35.75	10.9	52.7	52.8	60.27	33.93	56.5	45.48	56.25
T0 2023.02		47.83	32.1	53.23	34.86	53.22	53.33	67.25	14.94	58.18	52.27	52.05	45.33	44.29	46.84	-
T0 2022.10		47.83	41.1	52.79	25.71	52.85	46.67	45.52	14.84	58.89	52.94	50.23	46.64	44.29	42.56	15.63