Our new X account is live! Follow @wizwand_team for updates

Multimodal Question Answering on ScienceQA v1.3 (test)

0.9019NAT Score

Full Precision (Baseline)

Updated 3d ago

Evaluation Results

Method	Links
Full Precision (Baseline) 2024.10		0.9019	0.9314	0.8709	0.8939	0.8706	0.8983	0.9
Q-VLM 2024.10		0.8954	0.9318	0.865	0.8812	0.8701	0.8885	0.897
Q-VLM 2024.10		0.8943	0.9573	0.84	0.8871	0.8751	0.8725	0.8934
Full Precision (Baseline) 2024.10		0.8939	0.9606	0.8564	0.8871	0.8765	0.885	0.8981
QLoRA 2024.10		0.8887	0.9289	0.8564	0.8759	0.8656	0.8753	0.8887
QLoRA 2024.10		0.8845	0.9471	0.8445	0.8763	0.8607	0.8787	0.8843
AWQ 2024.10		0.8803	0.926	0.84	0.8602	0.8518	0.8641	0.8757
AWQ 2024.10		0.8539	0.9201	0.8327	0.848	0.8354	0.8599	0.8623
Q-VLM 2024.10		0.8255	0.7332	0.8318	0.8103	0.7082	0.8674	0.8078
Q-VLM 2024.10		0.8086	0.7593	0.8073	0.8001	0.7248	0.839	0.7979
AWQ 2024.10		0.8071	0.7061	0.7849	0.7946	0.7076	0.8182	0.7791
QLoRA 2024.10		0.7962	0.7143	0.8245	0.7825	0.6842	0.853	0.7864
QLoRA 2024.10		0.7753	0.7548	0.7918	0.7664	0.707	0.8195	0.7753
AWQ 2024.10		0.7433	0.7222	0.7482	0.7341	0.6713	0.7798	0.7402
Q-VLM 2024.10		0.6414	0.5868	0.6285	0.628	0.5523	0.6669	0.6246
Full Precision (Baseline) 2024.10		0.6401	0.5857	0.633	0.628	0.5478	0.6697	0.6268
QLoRA 2024.10		0.6298	0.5778	0.6185	0.6187	0.5399	0.6537	0.616
AWQ 2024.10		0.5983	0.5778	0.6148	0.5894	0.5295	0.6425	0.5983
Q-VLM 2024.10		0.5506	0.5194	0.5627	0.5457	0.4803	0.5834	0.5472
QLoRA 2024.10		0.5448	0.4969	0.5555	0.532	0.473	0.5758	0.5324
AWQ 2024.10		0.5369	0.4958	0.5427	0.5252	0.4781	0.5765	0.5298