Measuring Semantic Similarity in Japanese Key Audit Matters
Nobushige Doi, Yusuke Nobuta, Takeshi Mizuno · 2024
In this study, some methods based on natural language processing for automatically evaluating the semantic similarity of key audit matters (KAMs) in audit reports of publicly listed companies in Japan were proposed. KAMs are issues that auditors deem particularly significant in a financial statement audit, and their reporting enhances transparency and facilitates audit quality assessment. However, the issue of “boilerplate” (i.e., the standardization of KAM content) was identified. To address this issue, the similarity between KAMs was precisely measured using indicators based on word frequency, word matching rates, and context-embedded vectors. The results were compared with those obtained using manual annotations, which indicated that the evaluation based on word match rate effectively measured the degree of boilerplate. Moreover, the evaluation based on context embedding vectors effectively captured the semantic similarity of KAMs.