Automated essay scoring with string kernels and word embeddings
About
In this work, we present an approach based on combining string kernels and word embeddings for automatic essay scoring. String kernels capture the similarity among strings based on counting common character n-grams, which are a low-level yet powerful type of feature, demonstrating state-of-the-art results in various text classification tasks such as Arabic dialect identification or native language identification. To our best knowledge, we are the first to apply string kernels to automatically score essays. We are also the first to combine them with a high-level semantic feature representation, namely the bag-of-super-word-embeddings. We report the best performance on the Automated Student Assessment Prize data set, in both in-domain and cross-domain settings, surpassing recent state-of-the-art deep learning approaches.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Automated essay scoring | ASAP 1.0 (test) | Prompt 1 QWK0.804 | 51 | |
| Automatic Essay Scoring | ASAP (Automated Student Assessment Prize) prompt 1 to 2 | QWK0.661 | 24 | |
| Automatic Essay Scoring | ASAP (Automated Student Assessment Prize) prompt 3 to 4 | QWK0.779 | 24 | |
| Automatic Essay Scoring | ASAP (Automated Student Assessment Prize) prompt 5 to 6 | QWK0.788 | 24 | |
| Automatic Essay Scoring | ASAP (Automated Student Assessment Prize) prompt 7 to 8 | QWK0.649 | 24 | |
| Automated essay scoring | ASAP++ full-data setting | Score P10.674 | 10 | |
| Multi-trait automated essay scoring | ASAP++ (full-data) | Overall Score0.718 | 10 | |
| Automatic Essay Scoring | ASAP In-domain (5-fold cross-validation) | Overall QWK0.785 | 8 |