Share your thoughts, 1 month free Claude Pro on usSee more

Predicting code correctness on LiveSQLBench SQLite

0.129Brier Score

Func. Neg.

Updated 1mo ago

Evaluation Results

Method	Links
Func. Neg. 2026.05		0.129
Func. Sets 2026.05		0.135
Func. Equiv. Rate 2026.05		0.147
Func. Neg. 2026.05		0.168
Func. Equiv. Rate 2026.05		0.169
Func. Sets 2026.05		0.174
Min. Prob. 2026.05		0.175
Func. Sets 2026.05		0.176
Func. Neg. 2026.05		0.183
Min. Tok. Neg. 2026.05		0.184
Min. Prob. 2026.05		0.199
Min. Tok. Neg. 2026.05		0.2
Func. Equiv. Rate 2026.05		0.2
Func. Sets 2026.05		0.207
Func. Neg. 2026.05		0.215
Min. Tok. Neg. 2026.05		0.225
Min. Tok. Neg. 2026.05		0.248
Func. Sets 2026.05		0.248
Func. Neg. 2026.05		0.261
Func. Equiv. Rate 2026.05		0.263
P(True) 2026.05		0.264
Min. Prob. 2026.05		0.266
Min. Tok. Neg. 2026.05		0.266
Min. Prob. 2026.05		0.302
Func. Equiv. Rate 2026.05		0.302
Min. Prob. 2026.05		0.308
Verb. Conf. 2026.05		0.368
P(True) 2026.05		0.381
Seq. Prob. 2026.05		0.398
P(True) 2026.05		0.422
Verb. Conf. 2026.05		0.43
Verb. Conf. 2026.05		0.431
Verb. Conf. 2026.05		0.46
P(True) 2026.05		0.477
P(True) 2026.05		0.485
Cos. Sim. 2026.05		0.503
Cos. Sim. 2026.05		0.51
Prob. Margin 2026.05		0.518
Seq. Prob. 2026.05		0.519
Seq. Prob. 2026.05		0.52
Prob. Margin 2026.05		0.521
Avg. Tok. Neg. 2026.05		0.532
Avg. Tok. Neg. 2026.05		0.532
Verb. Conf. 2026.05		0.548
Prob. Margin 2026.05		0.553
Avg. Tok. Neg. 2026.05		0.604
Cos. Sim. 2026.05		0.62
Seq. Prob. 2026.05		0.627
Prob. Margin 2026.05		0.686
Seq. Prob. 2026.05		0.687
Prob. Margin 2026.05		0.692
Cos. Sim. 2026.05		0.705
Avg. Tok. Neg. 2026.05		0.724
Avg. Tok. Neg. 2026.05		0.729
Cos. Sim. 2026.05		0.734