Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

A Spectral Approach to Item Response Theory

About

The Rasch model is one of the most fundamental models in \emph{item response theory} and has wide-ranging applications from education testing to recommendation systems. In a universe with $n$ users and $m$ items, the Rasch model assumes that the binary response $X_{li} \in \{0,1\}$ of a user $l$ with parameter $\theta^*_l$ to an item $i$ with parameter $\beta^*_i$ (e.g., a user likes a movie, a student correctly solves a problem) is distributed as $\Pr(X_{li}=1) = 1/(1 + \exp{-(\theta^*_l - \beta^*_i)})$. In this paper, we propose a \emph{new item estimation} algorithm for this celebrated model (i.e., to estimate $\beta^*$). The core of our algorithm is the computation of the stationary distribution of a Markov chain defined on an item-item graph. We complement our algorithmic contributions with finite-sample error guarantees, the first of their kind in the literature, showing that our algorithm is consistent and enjoys favorable optimality properties. We discuss practical modifications to accelerate and robustify the algorithm that practitioners can adopt. Experiments on synthetic and real-life datasets, ranging from small education testing datasets to large recommendation systems datasets show that our algorithm is scalable, accurate, and competitive with the most commonly used methods in the literature.

Duc Nguyen, Anderson Zhang• 2022

Related benchmarks

TaskDatasetResultRank
Item Response Theory AssessmentLSAT
AUC70.7
9
Item Response Theory Assessment3 GRADES
AUC0.532
9
Item Response Theory AssessmentHETREC
AUC72.9
9
Item Response Theory AssessmentML 1M
AUC0.698
9
Item Response Theory AssessmentUCI
AUC56.5
5
Item Response Theory AssessmentML-100K
AUC0.662
5
Item Response Theory AssessmentEACH MOVIE
AUC71.6
5
Item Response Theory AssessmentBX
AUC54.6
5
Rasch model parameter estimationUCI
AUC0.565
4
Item Response Theory AssessmentBOOK-GENOME
AUC65.8
4
Showing 10 of 23 rows

Other info

Follow for update