ChemBench

Benchmarks

Task Name	Dataset Name	SOTA Result
General and STEM reasoning	ChemBench	Pass@167.86	20
Scientific Reasoning	ChemBench (test)	Initial Score66	13
Chemical knowledge and reasoning	ChemBench	Overall Score86.1	11
Technical chemistry	ChemBench Technical chemistry	Spearman Correlation0.86	8
Physical chemistry	ChemBench Physical chemistry	Spearman Correlation0.74	8
Organic chemistry	ChemBench Organic chemistry	Spearman Correlation0.89	8
Material science	ChemBench Material science	Spearman Correlation0.42	8
Inorganic chemistry	ChemBench Inorganic chemistry	Spearman Correlation0.67	8
Analytical chemistry	ChemBench Analytical chemistry	Spearman Correlation0.86	8
Chemical reasoning and prediction	ChemBench4K	Product93.66	7
In-Context Learning	ChemBench	Accuracy58.4	6
Science	ChemBench	Score83.3	3
Chemistry Question Answering	ChemBench Overall	Success90.5	1
Chemistry Question Answering	ChemBench Toxicity and Safety	Success Rate90.1	1
Chemistry Question Answering	ChemBench Technical Chemistry	Success Rate0.825	1
Chemistry Question Answering	ChemBench Physical Chemistry	Success8,420	1
Chemistry Question Answering	ChemBench Organic Chemistry	Success92.1	1
Chemistry Question Answering	ChemBench Materials Science	Success Rate90.5	1
Chemistry Question Answering	ChemBench Inorganic Chemistry	Success Rate91.3	1
Chemistry Question Answering	ChemBench General Chemistry	Success Rate0.872	1
Chemistry Question Answering	ChemBench Chemical Preference	Success Rate93.1	1
Chemistry Question Answering	ChemBench Analytical Chemistry	Success Rate82.2	1

Showing 22 of 22 rows