Evaluating Dynamic Software Update Safety Using Systematic Testing
Christopher M. Hayden, Edward K. Smith, Eric A. Hardisty, Michael Hicks, Jeffrey S. Foster · IEEE Transactions on Software Engineering · 2011
Dynamic software updating (DSU) systems patch programs on the fly without incurring downtime. To avoid failures due to the updating process itself, many DSU systems employ timing restrictions. However, timing restrictions are theoretically imperfect, and their practical effectiveness is an open question. This paper presents the first significant empirical evaluation of three popular timing restrictions: activeness safety (AS), which prevents updates to active functions; con-freeness safety (CFS), which only allows modifications to active functions when doing so is provably type-safe; and manual selection, which permits updates at developer chosen program points. We evaluated these timing restrictions using a series of DSU patches to three programs: OpenSSH, vsftpd, and ngIRCd. We systematically applied updates at each distinct update point reached during execution of a suite of system tests for these programs to determine which updates pass and which fail. We found that all three timing restrictions prevented most failures, but only manual selection allowed none. Further, although CFS and AS allowed many more update points, manual selection still supported updates with minimal delay. Finally, we found that manual selection required the least developer effort. Overall, we conclude that manual selection is most effective.