WEKO3
アイテム
Domain Adaptation with Medical Vocabulary-Aware Tokenizer for Radiology Report Analysis in RadNLP at KAIYO03
https://doi.org/10.20736/0002002066
https://doi.org/10.20736/000200206626d1b4fa-cad6-4e1f-b7eb-bfc938a02f4f
| 名前 / ファイル | ライセンス | アクション |
|---|---|---|
06-NTCIR18-RADNLP-ShirafujiD.pdf (1.0 MB)
|
|
| アイテムタイプ | デフォルトアイテムタイプ（フル）(1) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 公開日 | 2025-06-06 | |||||||||
| タイトル | ||||||||||
| タイトル | Domain Adaptation with Medical Vocabulary-Aware Tokenizer for Radiology Report Analysis in RadNLP at KAIYO03 | |||||||||
| 言語 | en | |||||||||
| 作成者 |
Daiki Shirafuji
× Daiki Shirafuji
× Takafumi Niwa
|
|||||||||
| 内容記述 | ||||||||||
| 内容記述タイプ | Abstract | |||||||||
| 内容記述 | Recent advances in language models (LMs) have significantly improved the handling of complex medical narratives compared to classical methods. However, one major obstacle to the practical usage of these LMs in the medical domain is that the models lack training on medical knowledge. In particular, standard tokenizers trained on open-domain corpora fail to accurately capture domain-specific terminologies, abbreviations, and writing styles in radiology reports or clinical notes. To address this issue, we propose a two-step domain-transfer method that updates both the tokenizer vocabulary and the LM representations. First, we replace low-frequency tokens in the original general-domain vocabulary with high-frequency bi- and tri-grams extracted from medical text, ensuring that domain-relevant tokens are learned. Second, we continually pre-train the LM on the medical corpus using the masked language modeling to more closely align the model parameters to the domain-specific language parameters. We evaluated the effectiveness of this approach in the RadNLP 2024 shared task on lung cancer staging from radiology reports, covering both English and Japanese. Experimental results indicate that our method improves performance on this specialized task, suggesting that customizing tokenizers and re-training language models can substantially mitigate the domain gap. In the future, we address standardizing radiology report formats to facilitate more robust and accurate automated analysis. | |||||||||
| 言語 | en | |||||||||
| 出版者 | ||||||||||
| 出版者 | NII Institutional Repository | |||||||||
| 言語 | en | |||||||||
| 日付 | ||||||||||
| 日付 | 2025-06-06 | |||||||||
| 日付タイプ | Issued | |||||||||
| 言語 | ||||||||||
| 言語 | eng | |||||||||
| 資源タイプ | ||||||||||
| 資源タイプ識別子 | http://purl.org/coar/resource_type/c_5794 | |||||||||
| 資源タイプ | conference paper | |||||||||
| ID登録 | ||||||||||
| ID登録 | 10.20736/0002002066 | |||||||||
| ID登録タイプ | JaLC | |||||||||
| 関連情報 | ||||||||||
| 関連タイプ | isReferencedBy | |||||||||
| 識別子タイプ | URI | |||||||||
| 関連識別子 | https://research.nii.ac.jp/ntcir/ntcir-18/index.html | |||||||||
| 言語 | en | |||||||||
| 関連名称 | NTCIR-18 Conference | |||||||||
| 開始ページ | ||||||||||
| 開始ページ | none | |||||||||
| 会議記述 | ||||||||||
| 会議名 | NTCIR-18 Conference | |||||||||
| 言語 | en | |||||||||
| 回次 | 18 | |||||||||
| 主催機関 | National Institute of Informatics | |||||||||
| 言語 | en | |||||||||
| 開始年 | 2025 | |||||||||
| 開始月 | 6 | |||||||||
| 開始日 | 10 | |||||||||
| 終了年 | 2025 | |||||||||
| 終了月 | 6 | |||||||||
| 終了日 | 13 | |||||||||
| 開催期間 | June 10-13, 2025 | |||||||||
| 言語 | en | |||||||||
| 開催会場 | National Institute of Informatics | |||||||||
| 言語 | en | |||||||||
| 開催国 | JPN | |||||||||
