Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Personalizing Dialogue Agents: I have a dog, do you have pets too?

About

Chit-chat models are known to have several problems: they lack specificity, do not display a consistent personality and are often not very captivating. In this work we present the task of making chit-chat more engaging by conditioning on profile information. We collect data and train models to (i) condition on their given profile information; and (ii) information about the person they are talking to, resulting in improved dialogues, as measured by next utterance prediction. Since (ii) is initially unknown our model is trained to engage its partner with personal topics, and we show the resulting dialogue can be used to predict profile information about the interlocutors.

Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, Jason Weston• 2018

Related benchmarks

TaskDatasetResultRank
Dialogue GenerationPersonaChat (test)
Persona Consistency0.15
27
Personalized Dialogue GenerationPersonaChat (Human Evaluation)
Fluency3.04
16
Dialogue GenerationPERSONA-CHAT original (dev)
Hits@154.8
13
Persona Adherence AlignmentPersonaChat 1.0 (test)
Similarity100
11
Persona SimulationPersonaChat
Adherence Score100
11
Dialogue GenerationPERSONA-CHAT Revised (dev)
Hits@138.1
11
Authorship VerificationPersonaChat original (test)
F1 Score66.2
11
Persona Simulation Naturalness EvaluationPersonaChat (test)
CS (Coherence Score)0.555
11
Persona Opinion AlignmentATP Wave 34
EMD0.35
10
Persona Opinion AlignmentATP Wave 99
EMD0.58
10
Showing 10 of 19 rows

Other info

Code

Follow for update