Addressing the Synchronization Challenge in Cypress End-to-End Tests
Thiago Santos de Moura, Everton L. G. Alves, Regina Letícia Santos Felipe, Cláudio de Souza Baptista, Ismael Raimundo da Silva Neto, Hugo Feitosa de Figueirêdo · 2024
Automated end-to-end testing plays a crucial role in modern web software projects, helping testers identify faults within complex applications and shorten development cycles. Frameworks such as Cypress are essential to provide a comprehensive testing environment with features that facilitate better access and validation of page elements. However, time-related challenges (synchronization issues) remain a significant concern in such suites. Testers need to be aware of these challenges and employ appropriate waiting mechanisms to ensure test reliability. This paper presents a catalog of waiting mechanisms for Cypress tests and a set of empirical studies that investigate the potential impact of synchronization issues and waiting mechanisms on test suites. Our studies examine the suites of an open-source and industrial project. Our findings reveal that up to 32% of a suite can break due to synchronization issues, exposing flaky tests. Subsequently, we revised the suites by applying four waiting mechanisms (Static Wait, Stable DOM Wait, Network Wait, and Explicit Wait). Network Wait and Explicit Wait emerged as the most promising strategies leading to no breakages.