Accuracy Evaluation of SBOM Tools for Web Applications and System-Level Software
Andreas Halbritter, Dominik Merli · 2024
Recent vulnerabilities in software like Log4j raise the question whether the software supply chain is secured sufficiently. Governmental initiatives in the United States (US) and the European Union (EU) demand a Software Bill of Materials (SBOM) for solving this issue. An SBOM has to be produced by using creation tools and it has to be accurate and complete. In the past, there had been investigations in this field of research. However, no detailed investigation of several tools producing SBOMs has been conducted regarding accuracy and reliability. For this reason, we present a selection of four popular programming languages: Python, C, Rust and Typescript. For web application software we consider Python and Typescript while for system-level software C and Rust are investigated. They build the base for four sample software projects and their package manager. For manual checking, the software projects are kept small with a small amount of packages and a single dependency. The open-source analysis tools are categorized as programming language dependent and general tools, and run in the standard execution mode on the software projects. The results were checked against completeness and the National Telecommunications and Information Administration (NTIA) minimum and recommended elements. There is no recommendation for a specific tool as no tool fulfills every requirement, only two tools can be recommended in a limited way. Many tools do not provide a complete SBOM, as they do not depict every test package and dependency. Governmental initiatives should define further specifications on SBOM for example regarding their accuracy and depth. Further research in this field, for example for proprietary tools or other programming languages is desirable.