Compositional Sequence Labeling Models for Error Detection in Learner Writing
About
In this paper, we present the first experiments using neural network models for the task of error detection in learner writing. We perform a systematic comparison of alternative compositional architectures and propose a framework for error detection based on bidirectional LSTMs. Experiments on the CoNLL-14 shared task dataset show the model is able to outperform other participants on detecting errors in learner writing. Finally, the model is integrated with a publicly deployed self-assessment system, leading to performance comparable to human annotators.
Marek Rei, Helen Yannakoudakis• 2016
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Error detection | FCE (test) | F0.5 Score64.3 | 16 | |
| Error detection | CoNLL-14 (test1) | P15.4 | 9 | |
| Error detection | CoNLL-14 (test2) | Precision23.6 | 9 | |
| Automated essay scoring | FCE public (test) | Pearson's r78 | 4 | |
| Error detection | FCE (dev) | F0.5 Score60.7 | 2 | |
| Error detection | CoNLL (test1) | F0.5 Score34.3 | 2 | |
| Error detection | CoNLL (test2) | F0.544 | 2 |
Showing 7 of 7 rows