Document Type
Article
Publication Date
11-2022
Abstract
Background Research software plays an important role in solving real-life problems, empowering scientific innovations, and handling emergency situations. Therefore, the correctness and trustworthiness of research software are of absolute importance. Software testing is an important activity for identifying problematic code and helping to produce high-quality software. However, testing of research software is difficult due to the complexity of the underlying science, relatively unknown results from scientific algorithms, and the culture of the research software community.
Aims The goal of this paper is to better understand current testing practices, identify challenges, and provide recommendations on how to improve the testing process for research software development.
Method We surveyed members of the research software developer community to collect information regarding their knowledge about and use of software testing in their projects.
Results We analysed 120 responses and identified that even though research software developers report they have an average level of knowledge about software testing, they still find it difficult due to the numerous challenges involved. However, there are a number of ways, such as proper training, that can improve the testing process for research software.
Conclusions Testing can be challenging for any type of software. This difficulty is especially present in the development of research software, where software engineering activities are typically given less attention. To produce trustworthy results from research software, there is a need for a culture change so that testing is valued and teams devote appropriate effort to writing and executing tests.
Copyright Statement
This version of the article has been accepted for publication and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/s10664-022-10184-9
Publication Information
Eisty, Nasir U. and Carver, Jeffrey C.. (2022). "Testing Research Software: A Survey". Empirical Software Engineering, 27(6), 138. https://doi.org/10.1007/s10664-022-10184-9