A Linguistically-Informed Annotation Strategy for Korean Semantic Role Labeling

Chen, Yige; Lim, KyungTae; Park, Jungyeul

Korean semantic role labeling data converted from the Sejong verb dictionary into CoNLL-style format

LREC-COLING 2024

Yige Chen¹, KyungTae Lim^2,‡, Jungyeul Park³

¹The Chinese University of Hong Kong, Hong Kong ²SeoulTech, South Korea ³The University of British Columbia, Canada

^‡Corresponding author

Abstract

Semantic role labeling is an essential component of semantic and syntactic processing of natural languages, which reveals the predicate-argument structure of the language. Despite its importance, semantic role labeling for the Korean language has not been studied extensively. One notable issue is the lack of uniformity among data annotation strategies across different datasets, which often lack thorough rationales. In this study, we suggest an annotation strategy for Korean semantic role labeling that is in line with the previously proposed linguistic theories as well as the distinct properties of the Korean language. We further propose a simple yet viable conversion strategy from the Sejong verb dictionary to a CoNLL-style dataset for Korean semantic role labeling. Experiment results using a transformer-based sequence labeling model demonstrate the reliability and trainability of the converted dataset.

Keywords Korean semantic role labeling (SRL) Predicate-argument structure Arguments vs. modifiers Sejong verb dictionary CoNLL-style dataset Korean PropBank Case-marking postpositions Annotation scheme Korean NLP

Annotation Strategy and Conversion

XML entry from the Sejong verb dictionary for the verb 부치다 (buchida) in the sense "be beyond (one's capacity)", showing its subcategorization frame X=N0-이 Y=N1-에|에게 V, the semantic roles of X (THM) and Y (CRT), and the example sentences 그 일은 네 힘에 부친다 and 철수는 나에게 부친다.

Figure 1 (paper). Entry for the lexeme 부치다 (buchida, “be beyond (one's capacity)”) in the Sejong verb dictionary. Each sense lists its subcategorization frame (frame), the semantic roles of its arguments (sel_rst) and example sentences (eg), which are the input of the conversion.

Korean is an agglutinative, subject-object-verb (SOV) language in which postpositions attached to a word usually mark its case (e.g. 이/가 i/ga nominative, 을/를 (l-)eul accusative, 에서 eseo locative/ablative). Following Categorial Grammar, the paper defines arguments as syntactically mandatory and semantically necessary constituents that bear case and appear in the predicate's subcategorization frame; every other constituent is a modifier. Two consequences for annotation:

An argument may span several words (eojeol), which the word-level Korean PropBank SRL data (about 186,300 tokens) cannot represent.
The case-marking postposition belongs to the argument, whereas the NIKL SRL data (about 2,000,000 tokens) label only the lexical morphemes.

The Sejong verb dictionary is then converted into a CoNLL-style SRL dataset:

Morphological analysis. Example sentences are POS-tagged with a morpheme-level tagger and merged back into word-level tokens.
Dependency parsing. Stanza adds heads and dependency relations to each word.
Chunking. The segment before the target verb is split into candidate argument chunks, closing a chunk whenever a token ends with a postposition (according to its XPOS tag).
Chunk-frame alignment. Each argument in the subcategorization frame is assigned, left to right, to the first unlabeled chunk carrying the same postposition. Labels use the BIO scheme with argument indices 0 to 3 inherited from the frame.

Sentences with null (phonologically covert) postpositions or with the auxiliary postposition 도 (do, ‘as well’), which replaces the case marker, cannot be aligned and are removed from the final dataset. The converted instance of the example sentence from Figure 1 (“The task is beyond your strength.”) looks like this (Figure 3 in the paper):

# text = 그 일은 네 힘에 부친다.
# target = 부치다
# frame = X=N0-이 Y=N1-에|에게 V
# arg="X" tht="THM", (일)|인간
# arg="Y" tht="CRT", 인간|(힘|능력)
1  그      그        DET    MM        2  det         B-ARG0
2  일은    일+은     NOUN   NNG+JX    5  dislocated  I-ARG0
3  네      네        DET    MM        4  nummod      B-ARG1
4  힘에    힘+에     NOUN   NNG+JKB   5  obl         I-ARG1
5  부친다  부치+는다 VERB   VV+EF     0  root        TARGET
6  .       .         PUNCT  SF        5  punct       O

Main Results

Trainable converted data. A KoELECTRA-Base-v3 sequence labeler trained on 20,437 converted sentences (2-fold cross-validation) reaches exact-match precision 0.946 ± 0.003, recall 0.971 ± 0.002 and F1 0.954 ± 0.003.
Multi-word arguments. Arguments are annotated as BIO spans over chunks, so an argument can cover several eojeol, unlike the word-level Korean PropBank SRL dataset.
Case-bearing arguments. The postposition that carries the case is kept inside the argument span, unlike the NIKL SRL annotation that labels only the lexical part.

**Table 1.** Korean SRL on the converted Sejong dataset with KoELECTRA-Base-v3: cross-validation results (mean ± standard deviation) of exact matches on the test set (Table 2 in the paper).
Precision	Recall	F1
0.946 ± 0.003	0.971 ± 0.002	0.954 ± 0.003

@inproceedings{chen-etal-2024-linguistically, title = "A Linguistically-Informed Annotation Strategy for {K}orean Semantic Role Labeling", author = "Chen, Yige and Lim, KyungTae and Park, Jungyeul", editor = "Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen", booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation ({LREC}-{COLING} 2024)", month = may, year = "2024", address = "Torino, Italia", publisher = "ELRA and ICCL", url = "https://aclanthology.org/2024.lrec-main.65/", pages = "733--738" }

A Linguistically-Informed Annotation Strategy for Korean Semantic Role Labeling

Abstract

Annotation Strategy and Conversion

Main Results

BibTeX