Automated Quality Assessment of Firmware Releases
Dragan Adzaip, Sven Andersen · Lund University Publications Student Papers (Lund University) · 2020
Some companies strive for the best quality in their releases.For some companies the rmware quality can be "good enough" to be released.And others just want to get new functionality out quick.No matter the case, with the high tempo in the industry, with frequent releases and constantly having to be up to date, it is hard to stop and evaluate the quality.If customers or users give feedback, the companies have some idea of how the product is doing.However, this can only be seen after the release, given that there is any feedback.Is it possible to gather data from previous releases and predict the quality of the next release before it is released?Is it possible to determine or identify issues earlier to avoid problems when the customer is using the product?This thesis investigates how to create a model that is able to predict the quality of a release.The help from this kind of assessment would allow the company to measure how their rmware is doing before a release.The end product of the work are three models that are able to indicate the quality of a rmware release.Based on the data available before release, it is able to give one or several metrics that can be used to predict how well the release would fare if shipped out.We made a program for applying this model, so that it can be used at Axis.The way we did it was by rst doing a literature study as well as an in-house research at the company, to nd relevant metrics and data to use.We then proceeded to extract the data we decided to continue with.That data was based on said literature study and primarily on discussions and interviews with relevant personnel.After that, by using linear regression, we created several models where each model used di erent metrics.The models are in fact just functions where the metrics are unknown variables and their values in-parameters.Based on the values of the metrics, the model will predict a value corresponding to the quality.Lastly, the models were evaluated by training them on all but one data point and then predicting on that missing point.The three best were presented to the case company.They were satis ed in the end.They realised that there was too little data to make a super model of this kind, but will continue to use and further develop them to improve their accuracy.