Towards Standardized Annotation and Parsing for Korean FrameNet

Chen, Yige; Ihn, Jae; Lim, KyungTae; Park, Jungyeul

A morpheme-based CoNLL-U annotation scheme for Korean FrameNet data and frame-semantic parsing

LREC-COLING 2024

Yige Chen¹, Jae Ihn^2,∗∗, KyungTae Lim^3,∗, Jungyeul Park²

¹The Chinese University of Hong Kong, Hong Kong ²The University of British Columbia, Canada ³SeoulTech, South Korea

^∗Corresponding author ^∗∗Currently at Rocketbrew Inc., Canada

Abstract

Previous research on Korean FrameNet has produced several datasets that serve as resources for FrameNet parsing in Korean. However, these datasets suffer from the problem that annotations are assigned on the word level, which is not optimally designed based on the agglutinative feature of Korean. To address this issue, we introduce a morphologically enhanced annotation strategy for Korean FrameNet datasets and parsing by leveraging the CoNLL-U format. We present the results of the FrameNet parsers trained on the Korean FrameNet data in the original format and our proposed format, respectively, and further elaborate on the linguistic rationales of our proposed scheme. We suggest the morpheme-based scheme to be the standard of Korean FrameNet data annotation.

Keywords Korean FrameNet Frame-semantic parsing Morpheme-based annotation CoNLL-U format Eojeol vs. morpheme Agglutinative languages Frame element extraction Data standardization Korean NLP

Data and Annotation Scheme

The Korean PropBank fragment 북한이 국무부의테러지원국명단에 빠지게 되면 (if North Korea were excluded from the Department of State's list of state sponsors of terrorism) annotated in PropBank as (thing-excluded) ARG1, (excluded-from) ARG2 and (exclude) PRED, and in Korean FrameNet as Theme, Source and the target of the frame Removing.

Figure 1 (paper). The same instance annotated in the Korean PropBank and in Korean FrameNet (pkfn): FrameNet inherits the predicate-argument relations (Arg1 → Theme, Arg2 → Source) and re-analyzes the predicate as the target of the frame Removing.

The study uses three parts of the Korean FrameNet released by KAIST: pkfn (from the Korean PropBank; 1,767 sentences, 2,350 instances, mostly verbal targets), jkfn (projected from the Japanese FrameNet; 1,357 sentences, 2,919 instances, including 755 nominal lexical units) and skfn (from Sejong dictionary example sentences; 5,703 sentences, one frame each, verbal targets only). In the original data, frames are annotated on eojeol, the space-separated units that combine a lexical morpheme with functional morphemes such as case particles.

Morpheme-based tokens. Each eojeol is decomposed into morphemes, which become the basic tokens; the original eojeol is kept as a CoNLL-U multiword line above them.
Labels on morphemes. Targets and frame elements are annotated on morphemes. Functional morphemes stay inside the target/argument span (they are not labeled O) and no extra labels are introduced.
Lossless conversion. A script aligns eojeol and morphemes to convert existing Korean FrameNet data into the morpheme-based format and back.
Parsing setup. Given the frame target and frame, parsers extract arguments as sequence labeling over five classes (O, B-/I-FrameTarget, B-/I-Argument), since the data contain more than 2,000 unique frame elements. KoELECTRA-Base-v3 and KR-BERT-char16424 are fine-tuned with 2-fold cross-validation, and morpheme-based outputs are converted back to eojeol for a fair comparison.

Main Results

Large gains on jkfn. With KoELECTRA-Base, exact F1 rises from 0.3968 (eojeol) to 0.6528 (morpheme) and partial F1 from 0.4438 to 0.7152; with KR-BERT, exact F1 rises from 0.3493 to 0.6256.
Gains on pkfn. Exact F1 improves from 0.2523 to 0.3319 (KoELECTRA) and from 0.2964 to 0.3070 (KR-BERT); partial F1 from 0.3051 to 0.4091 and from 0.3475 to 0.4094.
skfn is the exception. On skfn, morpheme-based parsing scores lower (KoELECTRA exact F1 0.8091 → 0.6054), which the authors attribute to argument boundaries that were detected automatically rather than inherited from the Sejong source.
Recommendation. The authors propose the morpheme-based scheme as the standard way to annotate Korean FrameNet data for parsing.

**Table 1.** Cross-validation mean ± standard deviation of exact and partial F1 for frame argument extraction on eojeol-based (original) and morpheme-based (proposed) variants of the three Korean FrameNet subsets (Table 4 in the paper).
F1	Format	KoELECTRA-Base			KR-BERT-char16424
F1	Format	pkfn	jkfn	skfn	pkfn	jkfn	skfn
exact	eojeol	0.2523 ± 0.0215	0.3968 ± 0.0445	0.8091 ± 0.0003	0.2964 ± 0.0229	0.3493 ± 0.0281	0.8041 ± 0.0009
exact	morph	0.3319 ± 0.0807	0.6528 ± 0.0135	0.6054 ± 0.0056	0.3070 ± 0.0868	0.6256 ± 0.0127	0.5343 ± 0.0042
partial	eojeol	0.3051 ± 0.0224	0.4438 ± 0.0444	0.8279 ± 0.0003	0.3475 ± 0.0226	0.4010 ± 0.0267	0.8241 ± 0.0008
partial	morph	0.4091 ± 0.0694	0.7152 ± 0.0096	0.7373 ± 0.0047	0.4094 ± 0.0677	0.6929 ± 0.0083	0.6627 ± 0.0036

@inproceedings{chen-etal-2024-towards, title = "Towards Standardized Annotation and Parsing for {K}orean {F}rame{N}et", author = "Chen, Yige and Ihn, Jae and Lim, KyungTae and Park, Jungyeul", editor = "Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen", booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation ({LREC}-{COLING} 2024)", month = may, year = "2024", address = "Torino, Italia", publisher = "ELRA and ICCL", url = "https://aclanthology.org/2024.lrec-main.1447/", pages = "16653--16658" }

Towards Standardized Annotation and Parsing for Korean FrameNet

Abstract

Data and Annotation Scheme

Main Results

BibTeX