Evaluating Benchmark Cheating and the Superiority of MAMBA over Transformers in Bayesian Neural Networks: An in-depth Analysis of AI Performance
Idoko Peter Idoko, Omolola Eniodunmo, Mary Ofosua Danso, Olubunmi Bashiru, Onuh Matthew Ijiga, Helena Nbéu Nkula Manuel · World Journal of Advanced Engineering Technology and Sciences · 2024
Artificial Intelligence (AI) models have seen unprecedented advancements with the rise of architectures like Transformers and Bayesian Neural Networks (BNNs). However, these innovations have also given rise to concerns over benchmark cheating, potentially skewing results that influence model selection in practical applications. This review paper provides an in-depth analysis of benchmark cheating and explores the relative performance of the Multi-resolution Aggregated Memory and Boundary-Aware Architecture (MAMBA) compared to Transformers within the context of Bayesian Neural Networks. The paper begins with an exploration of benchmark cheating, outlining its manifestations in different AI research settings and its impact on evaluating model performance. It investigates how overfitting, data leakage, and selective benchmark reporting can distort comparative analyses. The subsequent section delves into the architecture and advantages of MAMBA over Transformers, highlighting its memory aggregation and boundary-awareness strategies that potentially make it superior in certain contexts.