Unit and regression tests of scientific software: A study on SWMM

Zedong Peng; Xuanyi Lin; Michelle Simon; Nan Niu

doi:10.1016/j.jocs.2021.101347

Unit and regression tests of scientific software: A study on SWMM

Zedong Peng
, Xuanyi Lin
, Michelle Simon
, Nan Niu

Computer Science

Research output: Contribution to journal › Article › peer-review

11 Scopus citations

Abstract

Testing helps assure software quality by executing a program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM) [32], we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 2953 unit tests of SWMM have a 39.7% statement-level code coverage and a 82.4% user manual coverage. Meanwhile, an examination of 58 regression tests of SWMM shows a 44.9% statement-level code coverage and a near 100% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests, and suggest a diversified way of executing regression tests.

Original language	English
Article number	101347
Journal	Journal of Computational Science
Volume	53
DOIs	https://doi.org/10.1016/j.jocs.2021.101347
State	Published - Jul 2021

Keywords

Regression testing
Scientific software
Storm Water Management Model (SWMM)
Test coverage
Unit testing
User manual

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.jocs.2021.101347

Cite this

@article{1ebba992764a410bb7ac3ee6d0882156,

title = "Unit and regression tests of scientific software: A study on SWMM",

abstract = "Testing helps assure software quality by executing a program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM) [32], we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 2953 unit tests of SWMM have a 39.7\% statement-level code coverage and a 82.4\% user manual coverage. Meanwhile, an examination of 58 regression tests of SWMM shows a 44.9\% statement-level code coverage and a near 100\% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests, and suggest a diversified way of executing regression tests.",

keywords = "Regression testing, Scientific software, Storm Water Management Model (SWMM), Test coverage, Unit testing, User manual",

author = "Zedong Peng and Xuanyi Lin and Michelle Simon and Nan Niu",

note = "Publisher Copyright: {\textcopyright} 2021 Elsevier B.V.",

year = "2021",

month = jul,

doi = "10.1016/j.jocs.2021.101347",

language = "English",

volume = "53",

journal = "Journal of Computational Science",

issn = "1877-7503",

publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - Unit and regression tests of scientific software

T2 - A study on SWMM

AU - Peng, Zedong

AU - Lin, Xuanyi

AU - Simon, Michelle

AU - Niu, Nan

PY - 2021/7

Y1 - 2021/7

N2 - Testing helps assure software quality by executing a program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM) [32], we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 2953 unit tests of SWMM have a 39.7% statement-level code coverage and a 82.4% user manual coverage. Meanwhile, an examination of 58 regression tests of SWMM shows a 44.9% statement-level code coverage and a near 100% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests, and suggest a diversified way of executing regression tests.

AB - Testing helps assure software quality by executing a program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating-point computations. Extending the recent work on analyzing the unit tests written by the developers of the Storm Water Management Model (SWMM) [32], we report in this paper the investigation of both unit and regression tests of SWMM. The results show that the 2953 unit tests of SWMM have a 39.7% statement-level code coverage and a 82.4% user manual coverage. Meanwhile, an examination of 58 regression tests of SWMM shows a 44.9% statement-level code coverage and a near 100% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests, and suggest a diversified way of executing regression tests.

KW - Regression testing

KW - Scientific software

KW - Storm Water Management Model (SWMM)

KW - Test coverage

KW - Unit testing

KW - User manual

UR - https://www.scopus.com/pages/publications/85104071997

U2 - 10.1016/j.jocs.2021.101347

DO - 10.1016/j.jocs.2021.101347

M3 - Article

AN - SCOPUS:85104071997

SN - 1877-7503

VL - 53

JO - Journal of Computational Science

JF - Journal of Computational Science

M1 - 101347

ER -

Unit and regression tests of scientific software: A study on SWMM

Abstract

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this