Multimodal Foundation Models: a Taxonomy and Reflections
Won Kim · International Journal of Web and Grid Services · 2024
During the past decade, thousands of technical blogs and research papers on foundation machine learning models, in particular large language models and multimodal models have inundated the technical literature. Tech companies vying for dominant positions or market shares in the AI market have been releasing new or more advanced models in rapid succession. There have even been speculations that artificial general intelligence is imminent. In this paper, first I try to organise the many multimodal foundation models in the form of a reasonable taxonomy. I hope such a taxonomy will help understand the relationships among different types of models. Then I try to provide a reminder that despite the amazing images and videos generated by the multimodal models, and amazing multimodal reasoning capabilities top multimodal models have shown, there are glaring limitations. Further, I emphasise that there is the inevitable 'dark side' to the advancements of technology. There are many potential negative impacts of the LLM and multimodal models; and manufacturing, running, cooling, and disposing of the computers contribute to the accelerated depletion of the natural sources and contamination of the environment.