Explainability for NLP
Pradeepta Mishra · Apress eBooks · 2021
This chapter explains the use of ELI5 and SHAP-explainable AI-based Python libraries with natural language processing (NLP) based tasks such as text classification models. The prediction decisions made by machine learning models for supervised learning tasks are of unstructured data. Text classification is a task where you need to consider text sentences or phrases as input and classify them into discrete categories. An example is news classification, where the content is used as the input and the output is classified into politics, business, sports, technology, and so on. A similar use case is spam detection in email classification, where the email content is used as the input and classified into spam or not spam. In this scenario, it is important to know if an email is classified into spam, then why? Which tokens present in the content actually lead to the prediction? This is of interest to the end user. When I say NLP tasks here, there any many tasks, but I’ll limit it to text classification use cases and similar use cases like entity recognition, parts of speech tagging, and sentiment analysis.