A Layered Taxonomy for Chinese Learner Grammatical Error Annotation

A layered error annotation scheme linking Chinese grammatical error correction (CGEC) with pedagogical error analysis

arXiv 2026

Mengyang Qiu1, Jungyeul Park2

1Saint Elizabeth University, USA 2KAIST, South Korea

Paper (arXiv) PDF
Flowchart: an edit defined relative to a selected target hypothesis is first tested for ORTH. If yes, it receives an orthographic label OP:ORTH:SUBTYPE with subtypes phon, shape, complex, order, punc. If no, the annotator determines the operation (M, R, U, WO), assigns POS (VERB, NOUN, PART, ADV, ..., X), applies the domain rule (every M, U, or WO is STRUCT; R with a content item is LEX-CONT; R with a closed-class item is LEX-FUNC), yielding the core label OP:DOM:POS, optionally followed by a pedagogical extension.
Figure 1. Annotation flow. Orthographic edits receive op:orth:subtype (subtypes phon, shape, complex, order, punc). All other edits receive the core label op:dom:pos: operation M (missing), R (replacement), U (unnecessary) or WO (word order); a part-of-speech tag; and a domain set by a deterministic rule (every non-orthographic M, U, or WO → STRUCT; R with a content item → LEX-CONT; R with a closed-class item → LEX-FUNC), optionally followed by a pedagogical extension.

Abstract

Grammatical error annotation in Chinese learner writing requires labels that are both consistent and linguistically meaningful. This paper proposes a layered scheme linking computational Chinese grammatical error correction (CGEC) with pedagogical error analysis. The scheme first identifies character- and punctuation-level orthographic errors, labeling them by edit operation and subtype. Other errors receive a three-layer core label combining edit operation, linguistic domain, and part of speech, with optional Chinese-specific extensions for aspect, modality, comparison, argument structure, and complements. Drawing on CGEC resources, learner-error taxonomies, and Mandarin grammar, the taxonomy is evaluated through a coverage analysis of automatically extracted MuCGEC edits and a preliminary consistency study in which five large language models apply it to a sample. The results support the layered approach while identifying category boundaries requiring further refinement.

Keywords Chinese GEC (CGEC) Error annotation Error taxonomy Learner corpus L2 Chinese / CSL ERRANT / ChERRANT MuCGEC Pedagogical feedback LLM annotators

The Taxonomy

Every edit is defined relative to a selected correction (target hypothesis) and follows one of two routes.

  1. Edit operation. Missing (M), Replacement (R), Unnecessary (U), and Word Order (WO) as a separate top-level operation, plus an orthographic-only Character Order (CO) operation.
  2. Orthographic route (op:orth:subtype): wrong-character spellings, internal character reversals, and punctuation errors, e.g. *做业 → 作业 zuòyè ‘homework’ (phon), *自已 → 自己 zìjǐ ‘oneself’ (shape), *亚州 → 亚洲 Yàzhōu ‘Asia’ (complex), *么什 → 什么 shénme ‘what’ (order). The homophonous particles 的/地/得 de are not orthographic, because their correction concerns grammatical function.
  3. Non-orthographic core (op:dom:pos): a linguistic domain (LEX-CONT for content-word choice, LEX-FUNC for choosing the wrong item from a closed functional inventory, STRUCT for phrase and clause structure such as 把 bǎ, 被 bèi, 比 bǐ constructions and complements) and a POS tag (VERB, NOUN, ADJ, ADV, PART, AUX, PREP, CLF, CONJ, or X).
  4. Optional pedagogical extensions: a family code and item, e.g. M:STRUCT:PART + ASP:le for a missing aspect marker 了 le; families cover aspect, modality, structural particles, comparison, argument structure, complements, and more.
Table 1. Inventory of orthographic labels.
LabelUse
R:ORTH:phonPhonological identity or close similarity without salient visual similarity
R:ORTH:shapeVisual similarity with different pronunciation
R:ORTH:complexBoth phonological and visual similarity are salient
CO:ORTH:orderCorrect characters in the wrong internal order within one word
R:ORTH:puncWrong punctuation form or choice
M:ORTH:puncMissing punctuation mark
U:ORTH:puncRedundant punctuation mark

Main Results

  • Coverage on MuCGEC. Of 4,426 first-reference MuCGEC development edits (extracted with a linguistically informed ChERRANT extension), Replacement accounts for 55.9%, Missing for 27.4%, and Unnecessary for 16.7%; orthographic screening identifies 24.0% (11.9% similarity-typed character substitutions, 12.1% punctuation).
  • Extensions are relevant. A cautious count of single-token function-word edits places another 17.6% in the extension inventory (e.g. 的/地/得 triggers 5.1%, modal auxiliaries 2.2%); of the 185 edits labeled as de-particle errors, 168 are insertions or deletions and only 17 replacements.
  • Consistency with five LLMs. On 391 sampled edits, pairwise agreement is 97.1% for the orthographic decision (κ = 0.883) and 91.6% for the non-orthographic operation, but falls to 84.5% for extension codes (κ = 0.736) and 69.9% for the exact composite label, rising to 86.1% with partial credit.
  • Within-model variation. Repeated runs of the same model match on approximately four fifths of composite labels; disagreements cluster around R vs. M, LEX-CONT vs. LEX-FUNC for some adverbs, use of POS X, and whether an extension applies.
Table 5. Agreement among five LLMs applying the taxonomy to 391 MuCGEC edits, per layer. The last column compares the most common model label with an expert reference on a deliberately varied 115-item subset (difficult cases overrepresented).
LayerPairwise (%)καPanel label vs. expert (%)
Orthographic screening decision97.10.8830.88399.1
Orthographic label (operation and subtype)92.70.8980.898100.0
Non-orthographic edit operation91.60.8780.87885.6
Linguistic domain89.80.8230.82382.7
Part of speech84.60.8200.82076.1
Extension code84.50.7360.73775.7
Composite label (exact, code level)69.90.6880.68953.8
Composite (partial credit)86.1–––

BibTeX

@misc{qiu2026layeredtaxonomychineselearner,
      title={A Layered Taxonomy for Chinese Learner Grammatical Error Annotation}, 
      author={Mengyang Qiu and Jungyeul Park},
      year={2026},
      eprint={2609.02153},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2609.02153}, 
}
More projects from KAIST MLP Lab →