LLM4cast: Repurposed LLM for Viral Disease Forecasting
Farah Saeed, Mohammed Aldosari, Ismailcem Budak Arpinar, John Arthur Miller · 2024
Viral diseases have had a significant impact on millions of people worldwide. Utilizing time series forecasting methods allows for the estimation of cases, facilitating the control of disease spread and the allocation of necessary resources in medical facilities. Traditional estimation methods use model per dataset methodology where a model is trained only on a single dataset for a disease. However, foundation models including large language models have shown improved results by training on multiple datasets before being applied to a target dataset. Following this strategy, we aim to train a time series model using multiple datasets from diverse domains and viral diseases. We utilize a pretrained large language model and adapt it for estimating Influenza-Like Illness. We propose a novel network architecture called LLM4cast that encodes the input patches through a bidirectional encoder for rich embedding extraction and passes the encoded patches to a pretrained TinyLlama for fine-tuning. The output from TinyLlama is then flattened and projected to estimate the probable cases. The framework is trained in two stages. In the first stage, we train using time series data from diverse domains with 2.56M timesteps. The second stage involves training using domain specific time series data about viral diseases. The trained network is used in the evaluation of disease cases. The results demonstrate significant improvement in the accuracy of forecasts for foundation models compared to state-of-the-art models trained from scratch.