Markov Chain Based Explainable Pattern Forecasting
Debdeep Paul, Chandra Wijaya, Sahim Yamaura, Koji Miura, Yosuke Tajika · 2023
The explosive penetration of artificial intelligence (AI) and machine learning (ML) based technologies are dramatically transforming the traditional decision support systems. We consider the pattern recognition and forecasting for demand timeseries in a business-to-business supply chain where demand exhibits high volatilities, non-stationarities, and skewness. We develop a pattern forecasting system by developing a data driven, feature dependent Markov chain-based framework. This may include any arbitrary user defined pattern qualitative in nature, such as plummet or recovery from plummet. To increase adoption of AI based techniques among the various stakeholders (e.g., sales, marketing, procurement, production planning) the inherent modeling and forecasting of different patterns needs to be explained in terms of domain knowledge the user is more familiar with. We therefore define two metrices to evaluate explainability to enable cross scenario comparison as the notion of explainability lacks mathematical precision. These are, namely, relevance and informativeness. Relevance is measured by direct scoring from the user whereas informativeness is inspired by the fundamental concept of measuring differences or discrepancies between distributions and for the sake of simplicity in our case measured by the variance in the main attribute of explainability. Moreover, our dataset is high dimensional where number of columns are much higher than number of rows and therefore the method for selecting features for fitting the Markov chain is extremely critical. To provide guidelines on selecting different attributes of our pipeline, we compare between feature selection methods from two families, one advanced and one traditional. We perform extensive evaluation with real dataset obtained from a business division belonging to the Panasonic Industry Co. Ltd and observe a sparsity promoting feature selection method performs better in terms of accuracy and explainability.