Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction

About

Distractors-incorrect yet plausible answer choices in multiple-choice questions (MCQs)-are vital in educational assessments, as they help identify student misconceptions by presenting potential reasoning errors. Current distractor generation methods typically produce shared distractors for all students, ignoring the individual variations in reasoning, which limits their diagnostic effectiveness. To tackle this challenge, we introduce the task of Personalized Distractor Generation, which tailors distractors to each student's specific cognitive flaws, inferred from their past question-answering (QA) history. While promising, this task is particularly demanding due to the limited number of QA records available for each student, which are insufficient for training, as well as the absence of their underlying reasoning process. To overcome this, we propose a novel, training-free two-stage framework. In the first stage, Monte Carlo Tree Search (MCTS) is used to reconstruct the student's reasoning process from past errors, creating a student-specific misconception prototype. In the second stage, this prototype guides the simulation of the student's reasoning on new questions, generating personalized distractors that resonate with their individual misconceptions. Our experiments, conducted on 1,361 students across 6 subjects, demonstrate that this approach outperforms existing methods in generating plausible, personalized distractors, and also effectively adapts to group-level settings, highlighting its robustness and versatility.

Tao Wu, Jingyuan Chen, Wang Lin, Jian Zhan, Mengze Li, Fangzhou Jin, Min Zhang, Kun Kuang, Fei Wu• 2025

Related benchmarks

TaskDatasetResultRank
Personalized distractor generationEedi 100 Student 1361
Accuracy32.1
45
Personalized distractor generationDiscrete_40 Student_1361
Accuracy40.4
45
Distractor GenerationDiscrete_40
Plausibility3.25
10
Group-level distractor generationMMLU Elementary Math
Recall34.95
8
Group-level distractor generationDiscrete Math 40
Recall36.43
8
Group-level distractor generationCEval Discrete Math
Recall45.56
8
Group-level distractor generationEedi Elementary Math 100
Recall31.1
8
Student Misconception AnalysisStudent_1361 Eedi_700 (subset)
Accuracy21
2
Student Misconception AnalysisStudent_1361 Discrete_106
Accuracy26.5
2
Student Misconception AnalysisStudent_1361 Python_192
Accuracy20.3
2
Showing 10 of 15 rows

Other info

Follow for update