Unified Automated Essay Scoring and Grammatical Error Correction

Multitask learning of essay scoring and grammatical error correction on a new Korean learner dataset (K-UEED)

Findings of NAACL 2025

SeungWoo Song1,*, Junghun Yuk2,*, ChangSu Choi1, HanGyeol Yoo1, HyeonSeok Lim1, KyungTae Lim3,†, Jungyeul Park4,†

1Seoul National University of Science and Technology, South Korea 2Hanbat National University, South Korea 3Korea Advanced Institute of Science and Technology (KAIST), South Korea 4The University of British Columbia, Canada

*Equal contribution   †Corresponding authors

Paper (ACL Anthology) PDF
Horizontal stacked bar chart of error-type percentages (noop, WS, UNCLASSIFIED, PARTICLE, INSERTION, CONJUGATION, SPELL, NOUN, ENDING, DELETION, VERB, PUNCT, MODIFIER, ADJECTIVE, SHORTEN, WO) for K-UEED, KAGAS L1, KAGAS L2, and predictions of Bllossom_GA, Bllossom_AG, Bllossom_GEC and BllossomV_GEC. L1 is dominated by whitespace (WS) errors, while L2 and K-UEED are dominated by particle errors and look alike.
Figure 1. Distributions of GEC error types in K-UEED and KAGAS (L1 native, L2 learner), and error predictions by the Bllossom models. Apart from the noop type that only K-UEED has, the K-UEED and L2 learner distributions are strikingly similar, while L1 is dominated by spacing (WS) errors.

Abstract

This study explores the integration of automated writing evaluation (AWE) and grammatical error correction (GEC) through multitask learning, demonstrating how combining these distinct tasks can enhance performance in both areas. By leveraging a shared learning framework, we show that models trained jointly on AWE and GEC outperform those trained on each task individually. To support this effort, we introduce a dataset specifically designed for multitask learning using AWE and GEC. Our experiments reveal significant synergies between tasks, leading to improvements in both writing assessment accuracy and error correction precision. This research represents a novel approach for optimizing language learning tools by unifying writing evaluation and correction tasks, offering insights into the potential of multitask learning in educational applications.

Keywords Automated essay scoring (AES) Automated writing evaluation (AWE) Grammatical error correction (GEC) Multitask learning K-UEED Korean learner corpus L2 Korean writing KAGAS LLM fine-tuning

Dataset and Method

Spreadsheet excerpt of K-UEED: for each essay, the segmented original learner sentence, corrections by annotators A and B, and the combined result; two rows where the annotators disagreed are highlighted.
Figure 2. A part of the K-UEED data. Original learner sentences, corrections by two annotators (A, B), and the final combined result agreed through discussion.

Essay scoring (AWE) works on whole essays, while GEC works sentence by sentence. The paper bridges the two with a single dataset, K-UEED, and instruction-tunes one generative model (Bllossom, a Korean-English Llama-3-8B) for both tasks.

  1. Start from a Korean AWE corpus. The Korean learner essays of Lim et al. (2023) carry proficiency levels 1–6 (CEFR A1–C2), teacher-assigned scores (0–100), writing prompts, and learners' native languages.
  2. Add GEC annotation. A native speaker manually split the essays into 48,937 sentences, which were corrected with minimal edits and verified by two native Korean linguists (82.6% agreement; conflicts resolved by discussion).
  3. Split by prompt. The prompt “The most memorable day of my life”, shared by Levels 3 (B1) and 5 (C1), forms the test set (207 essays, 4,072 sentences); the remaining 3,804 essays and 44,865 sentences are for training.
  4. Train with different task combinations. Single-task models (AWE only, GEC only) are compared with multitask variants: AG (predict the score, then correct the essay), GA (correct the essay, then predict the score), and MUL (sentence-level GEC combined with essay-level GA data).

AWE is measured by the paper's AWE metric and quadratic weighted kappa (QWK); GEC by GLEU and M2 precision, recall, and F0.5. The models are also evaluated on the KAGAS Korean GEC benchmark (Yoon et al., 2023) for native (L1) and learner (L2) text.

Main Results

  • Correct first, then score. Bllossom-GA reaches 60.39 AWE and 64.16 QWK, versus 36.72 and 46.18 for the AWE-only model and 41.55 / 39.25 for GPT-4o (5-shot). Multitask training raises QWK by 0.68 to 17.98 points.
  • GEC on K-UEED. GPT-4o (5-shot) has the highest GLEU (56.19); among fine-tuned Bllossom models the GEC-only model is best (50.76), so adding AWE data did not help GEC on K-UEED.
  • Helps existing GEC benchmarks. Adding K-UEED GEC data to KAGAS training raises L2 (learner) GLEU from 53.64 to 54.65 and F0.5 from 53.22 to 55.01; Bllossom-GA+KAGAS gives the best Union (L1+L2) GLEU (64.28) and F0.5 (67.09).
  • Korean pre-training matters; vocabulary expansion does not. Bllossom+KAGAS beats Llama-3-8B-Inst+KAGAS (e.g. L2 F0.5 53.22 vs. 50.63), while the vocabulary-expanded BllossomV-GEC+KAGAS drops to 48.05 L2 F0.5.
  • Errors track essay scores. In K-UEED, sentences needing no correction (noop) rise from 3.59% at score 0 to 28.83% at score 100, while INSERTION errors fall from 20.96% to 7.01%.
Table 1. Results on the K-UEED test set. AWE and QWK are essay-scoring metrics; GEC is scored with GLEU. Bllossom models are fine-tuned on K-UEED: AWE only, GEC only, or multitask (MUL, GA, AG). Bold marks the best score per column.
ModelAWEQWKGEC (GLEU)
Llama3-8B-MUL47.3436.4649.06
Llama3-8B (5-shot)16.424.8637.01
GPT-4o (5-shot)41.5539.2556.19
Bllossom-AWE36.7246.18-
Bllossom-GEC--50.76
Bllossom-MUL46.3846.8649.92
Bllossom-GA60.3964.1650.13
Bllossom-AG41.0658.6050.01

BibTeX

@inproceedings{song-etal-2025-unified,
    title = "Unified Automated Essay Scoring and Grammatical Error Correction",
    author = "Song, SeungWoo  and
      Yuk, Junghun  and
      Choi, ChangSu  and
      Yoo, HanGyeol  and
      Lim, HyeonSeok  and
      Lim, KyungTae  and
      Park, Jungyeul",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {NAACL} 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.findings-naacl.250/",
    doi = "10.18653/v1/2025.findings-naacl.250",
    pages = "4412--4426",
    ISBN = "979-8-89176-195-7"
}
More projects from KAIST MLP Lab →