A Study of the Classification of IT Jobs Using LSTM and LIME
In Hyeok Choi, Yang Sok Kim, Choong Kwon Lee · 2020
This study aims to suggest a new approach that finds important job skill terms using deep learning and eXplainable Artificial Intelligence (XAI) algorithms. A total of 52,190 job advertisements were collected from a job posting website using web crawling technique. The job advertisements were classified into specific job roles using Deep Learning-based Bidirectional LSTM(Long Short Term Memory) and Bidirectional LSTM Attention. Finally, the best performing Bidirectional LSTM Attention model was used to extract important terms from the selected job advertisements by using Local Interpretable Model-agnostic Explanations (LIME), one of the XAI techniques, and compared them with those selected by term frequency. The results show that these two sets are significantly different in some cases, even when one set is more reasonable compared to other set and vice versa. Although this research cannot conclude the LIME is better than the frequency-based approach for identifying important skills, at least we found that LIME could guide researchers to a new path for this task.