Mobiilisovellusten suunnitteluratkaisujen vahvistaminen datankeruulla ja metriikoilla
Kiiski, Kimmo · Aaltodoc (Aalto University) · 2013
This thesis discusses how software developers can measure the use of a mobile application and validate whether or not their design decisions added value to the software. Our subject is a comic reading application Comicked, currently available for the iPad and developed by a start-up called Witchcraft Labs. The company aims to create a new way of reading and publishing comics on tablets. In this thesis, we present an empirical approach for validating design decisions by monitoring metrics, suitable for the start-up with scarce resources. Application usage is measured from the actual end-users from the real environment. This kind of evaluation is done remotely, and therefore we need to instrument the application in a way that data can be collected on the client-side and then sent over the Internet to evaluators for analysis. By analysing remotely collected user events, we can estimate values for different kinds of metrics that describe the quality of the application. In case of Comicked, we found that the most interesting metrics describe user adoption, user loyalty, and usage rates of application features. However, we also need methods that allow us to compare the metrics trustworthy. In this thesis, we discuss A/B testing and statistical analysis, which evaluators can use to gain confidence in their measurements. We found that the developers can reliably validate their design decisions by using a combination of the metrics, remote instrumentation, A/B testing, statistical analysis and proper monitoring tools. However, they must understand the limits of these methods and not use them as the only methods of collecting information about their customers. The approach is meant for validating design decisions and not for discovering them.