Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Aggregating incoherent agents who disagree

About

In this paper, we explore how we should aggregate the degrees of belief of of a group of agents to give a single coherent set of degrees of belief, when at least some of those agents might be probabilistically incoherent. There are a number of way of aggregating degrees of belief, and there are a number of ways of fixing incoherent degrees of belief. When we have picked one of each, should we aggregate first and then fix, or fix first and then aggregate? Or should we try to do both at once? And when do these different procedures agree with one another? In this paper, we focus particularly on the final question.

Richard Pettigrew• 2017

Related benchmarks

TaskDatasetResultRank
Medical Question AnsweringMedMCQA
Accuracy69.33
521
Multiple-choice Question AnsweringHellaSwag
Accuracy78.33
196
Question AnsweringMedMCQA
Accuracy63.21
98
Question AnsweringMMLU Pro.Med.
Accuracy93.38
42
Question AnsweringCSQA
Accuracy86.67
36
Question AnsweringMMLU Formal Logic (test)
Accuracy72.01
22
Question AnsweringHH-RLHF
Accuracy53.67
22
Showing 7 of 7 rows

Other info

Follow for update