Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Personalizing Dialogue Agents: I have a dog, do you have pets too?

About

Chit-chat models are known to have several problems: they lack specificity, do not display a consistent personality and are often not very captivating. In this work we present the task of making chit-chat more engaging by conditioning on profile information. We collect data and train models to (i) condition on their given profile information; and (ii) information about the person they are talking to, resulting in improved dialogues, as measured by next utterance prediction. Since (ii) is initially unknown our model is trained to engage its partner with personal topics, and we show the resulting dialogue can be used to predict profile information about the interlocutors.

Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, Jason Weston• 2018

Related benchmarks

TaskDatasetResultRank
Dialogue GenerationPersonaChat (test)
Persona Consistency0.15
27
Personalized Dialogue GenerationPersonaChat (Human Evaluation)
Fluency3.04
16
Dialogue GenerationPERSONA-CHAT original (dev)
Hits@154.8
13
Persona Adherence AlignmentPersonaChat 1.0 (test)
Similarity100
11
Persona SimulationPersonaChat
Adherence Score100
11
Dialogue GenerationPERSONA-CHAT Revised (dev)
Hits@138.1
11
Authorship VerificationPersonaChat original (test)
F1 Score66.2
11
Persona Simulation Naturalness EvaluationPersonaChat (test)
CS (Coherence Score)0.555
11
Human Evaluation of DialoguePersona-Chat 1.0 (test)
Fluency3.97
9
Knowledge-Grounded ConversationPersonalized KGC dataset (test)
BLEU-13.87
9
Showing 10 of 17 rows

Other info

Code

Follow for update