Enriching the Korean learner corpus for grammatical error correction and writing assessment

KoLLA v2.0: the first Korean L2 learner corpus with multi-reference grammatical error correction (GEC) annotations and rubric-based essay scores

Language Resources and Evaluation 2026

Jayoung Song1,*, KyungTae Lim2,*, Jungyeul Park3,†

1Department of Asian Studies, The Pennsylvania State University 2Graduate School of Culture Technology, KAIST 3Culture Technology Research Institute, KAIST

*These authors contributed equally   †Corresponding author

Paper (Language Resources and Evaluation) PDF arXiv Dataset
M2-format example from KoLLA v2.0. Source sentence: 비행기 음식이 안 막였습니다 (word positions 0 to 4). Annotator 0: edit at tokens 1-2 R:NOUN+ADP to NOUN+ADP, 음식이 corrected to 음식을, and edit at tokens 3-4 R:SPELL corrected to 먹었습니다 ('I didn't eat the airplane food'). Annotator 1: edit at tokens 3-4 R:SPELL corrected to 맞았습니다 ('The airplane food didn't agree with me').
Figure 1. Example from the Korean M2 file. For the learner sentence “비행기 음식이 안 막였습니다.”, Annotator 0 corrects it to I didn’t eat the airplane food (particle fix 음식이→음식을 plus spelling fix →먹었습니다), while Annotator 1 corrects it to The airplane food didn’t agree with me (spelling fix →맞았습니다). Both are valid, which is why KoLLA v2.0 keeps two references per sentence. Index numbers mark word positions and are added for illustration only.

Abstract

Despite growing global interest in Korean language education, learner corpora tailored to Korean L2 writing remain scarce. This paper introduces KoLLA v2.0, the first Korean learner corpus to incorporate both multi-reference GEC annotations and rubric-based essay scoring. We extend the original KoLLA dataset by adding a second human correction for each sentence, creating the first multi-reference GEC resource for Korean. This design captures the variability of valid corrections in an agglutinative and morphologically complex language, enabling fairer and more realistic evaluation of GEC systems. In parallel, we enrich the corpus with rubric-based scores based on criteria from the Korean National Language Institute, providing standardized, multi-dimensional assessments of grammatical accuracy, coherence, and lexical diversity. These enhancements position KoLLA v2.0 as a standardized resource for research in Korean L2 learning and instruction, and as a benchmark for evaluating automated error correction and essay scoring systems.

Keywords Korean learner corpus (KoLLA) Grammatical error correction (GEC) Multi-reference GEC Automated essay scoring (AES) Rubric-based scoring Korean as a second language (L2) Error annotation / M2 format Postposition (particle) errors Heritage learners

Corpus Construction

Algorithm 1, error annotation procedure. Given a source and target sentence: if the associated content word is correct, then an unnecessary ADP or PART is labeled R:POS_i+ADP to POS_j, a missing ADP or PART is labeled R:POS_i to POS_j+ADP, and an incorrect ADP or PART is labeled R:POS+ADP_i to POS+ADP_j; if the content word is incorrect, the label is R:SPELL. If a word boundary error occurs, words that should be separate get M:WB and words that should be merged get U:WB. If two words appear in swapped order, the label is R:WO.
Algorithm 1. Error annotation procedure of KoLLA v2.0: the ERRANT-style Missing / Replacement / Unnecessary scheme extended for Korean with postposition and verbal-ending errors (ADP, PART), spelling (R:SPELL), word boundary (M:WB, U:WB), and word order (R:WO) labels.

KoLLA v2.0 reuses the 100 learner essays of the original KoLLA corpus (Lee et al., 2012), written by American university students in four subgroups of 25 essays each: foreign beginners (FB), foreign intermediates (FI), heritage beginners (HB), and heritage intermediates (HI). The learner texts are unchanged; two new annotation layers are added.

  1. Sentence segmentation. The essays are split into 1,418 sentences with two references each (2,836 references in total; the paper elsewhere gives 1,419 sentences for the original KoLLA segmentation), since GEC works at the sentence level while essay scoring works at the essay level.
  2. Multi-reference GEC. Each sentence keeps its original KoLLA correction and receives a second correction written by a trained linguist (M.A. in Korean Language Education) who preserves the learner’s meaning and avoids unnecessary rewriting, giving two references per sentence.
  3. Korean error typology. Errors are labeled at the word (eojeol) level in the M2 format, following ERRANT’s Missing / Replacement / Unnecessary framework extended for Korean (Algorithm 1): postposition and verbal-ending errors, spelling, word boundary, and word order. Preprocessing normalized frequent particle and compound-verb variants, lowering segmentation misalignment errors to under 5%.
  4. Rubric-based essay scoring. Two raters with M.A. degrees in Korean Language Education scored every essay with a Korean L1 writing rubric based on Korean National Language Institute criteria: 10 criteria in three categories (lexis and grammar; organization; content), each scored 0–3, for a maximum of 30 points. Raters were calibrated with two training sessions on 15 sample essays and 20% of the data before scoring all essays independently.

The released dataset contains one M2 file with both references for every sentence and four CSV files (FB, FI, HB, HI) with both raters’ scores on the 10 rubric dimensions for 25 essays each.

Main Results

  • Reliable essay scores. Inter-rater agreement (Cohen’s κ) is 0.8401 overall: 0.8253 (FB), 0.8752 (FI), 0.7971 (HB), and 0.7721 (HI).
  • Valid corrections diverge. Treating the original corrections as reference and the new ones as hypothesis gives an M2 F1 of only 0.3881 (708 true positives, 1,173 false positives, 1,060 false negatives), showing how much single-reference evaluation can miss.
  • Multi-reference evaluation is fairer. For a KoBART GEC system, F0.5 rises from 0.2709 with the original reference only to 0.3946 with both references (precision 0.2974 → 0.4170, recall 0.1997 → 0.3248).
  • Spelling dominates learner errors. R:SPELL is the most frequent error type in every subgroup (0.3729 FB, 0.3975 FI, 0.4620 HB, 0.4818 HI), followed by postposition errors such as particle omission (R:NOUN → NOUN+ADP).
Table 4. KoBART-based GEC system evaluated with the M2 scorer against the original single reference and against both references (sentence-level best match). (Table 4 in the paper, Appendix B.)
Evaluation settingTPFPFNPrecisionRecallF0.5
Single reference (original only)35383414150.29740.19970.2709
Multi-reference (original + proposed)49569210290.41700.32480.3946

BibTeX

@article{Song_2026,
  title={Enriching the Korean learner corpus for grammatical error correction and writing assessment},
  volume={60},
  ISSN={1574-0218},
  url={http://dx.doi.org/10.1007/s10579-025-09882-9},
  DOI={10.1007/s10579-025-09882-9},
  number={1},
  journal={Language Resources and Evaluation},
  publisher={Springer Science and Business Media LLC},
  author={Song, Jayoung and Lim, KyungTae and Park, Jungyeul},
  year={2026},
  month=Jan
}
More projects from KAIST MLP Lab →