Reliability Assessment of Mass-Market Software: Insights from Windows Vista®
Paul Luo Li, Mingtian Ni, Song Xue, Joseph P. Mullally, Mario Garzia, Mujtaba Khambatti · 2008
Assessing the reliability of mass-market software (MMS), such as the Windowsregoperating system, presents many challenges. In this paper, we share insights gained from the Windows Vistaregand Windows VistaregSP1 operating systems. First, we find that the automated reliability monitoring approach, which periodically reports reliability status, provides higher quality data and requires less effort compared to other approaches available today. We describe one instance in detail: the Windows reliability analysis component, and illustrate its advantages using data from Windows Vista. Second, we show the need to account for usage scenarios during reliability assessments. For pre-release versions of Windows Vista and Vista SP1, usage scenarios differ by 2-4X for Microsoft internal and external samples; corresponding reliability assessments differ by 2-3X. Our results help motivate and guide further research in reliability assessment.