Unit tests of scientific software: A study on swmm

Zedong Peng; Xuanyi Lin; Nan Niu

doi:10.1007/978-3-030-50436-6_30

Unit tests of scientific software: A study on swmm

Zedong Peng
, Xuanyi Lin
, Nan Niu

Computer Science

University of Cincinnati

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

Testing helps assure software quality by executing program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating point computations. We report in this paper a manual analysis of the unit tests written by the developers of the Storm Water Management Model (SWMM). The results show that the 1,458 SWMM tests have a 54.0% code coverage and a 82.4% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests. Based on these results, we offer insights to improve test development and coverage.

Original language	English
Title of host publication	Computational Science – ICCS 2020 - 20th International Conference, Proceedings
Editors	Valeria V. Krzhizhanovskaya, Gábor Závodszky, Michael H. Lees, Peter M.A. Sloot, Peter M.A. Sloot, Peter M.A. Sloot, Jack J. Dongarra, Sérgio Brissos, João Teixeira
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	413-427
Number of pages	15
ISBN (Print)	9783030504359
DOIs	https://doi.org/10.1007/978-3-030-50436-6_30
State	Published - 2020
Event	20th International Conference on Computational Science, ICCS 2020 - Amsterdam, Netherlands Duration: Jun 3 2020 → Jun 5 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12143 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	20th International Conference on Computational Science, ICCS 2020
Country/Territory	Netherlands
City	Amsterdam
Period	06/3/20 → 06/5/20

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 6 Clean Water and Sanitation

Keywords

Scientific software
Storm Water Management Model (SWMM)
Test coverage
Test oracle
Unit testing
User manual

Access to Document

10.1007/978-3-030-50436-6_30

Cite this

Peng, Z., Lin, X., & Niu, N. (2020). Unit tests of scientific software: A study on swmm. In V. V. Krzhizhanovskaya, G. Závodszky, M. H. Lees, P. M. A. Sloot, P. M. A. Sloot, P. M. A. Sloot, J. J. Dongarra, S. Brissos, & J. Teixeira (Eds.), Computational Science – ICCS 2020 - 20th International Conference, Proceedings (pp. 413-427). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12143 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-50436-6_30

Peng, Zedong ; Lin, Xuanyi ; Niu, Nan. / Unit tests of scientific software : A study on swmm. Computational Science – ICCS 2020 - 20th International Conference, Proceedings. editor / Valeria V. Krzhizhanovskaya ; Gábor Závodszky ; Michael H. Lees ; Peter M.A. Sloot ; Peter M.A. Sloot ; Peter M.A. Sloot ; Jack J. Dongarra ; Sérgio Brissos ; João Teixeira. Springer Science and Business Media Deutschland GmbH, 2020. pp. 413-427 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b8624566c74c43e5ac284aa257efe781,

title = "Unit tests of scientific software: A study on swmm",

abstract = "Testing helps assure software quality by executing program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating point computations. We report in this paper a manual analysis of the unit tests written by the developers of the Storm Water Management Model (SWMM). The results show that the 1,458 SWMM tests have a 54.0\% code coverage and a 82.4\% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests. Based on these results, we offer insights to improve test development and coverage.",

keywords = "Scientific software, Storm Water Management Model (SWMM), Test coverage, Test oracle, Unit testing, User manual",

author = "Zedong Peng and Xuanyi Lin and Nan Niu",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2020.; 20th International Conference on Computational Science, ICCS 2020 ; Conference date: 03-06-2020 Through 05-06-2020",

year = "2020",

doi = "10.1007/978-3-030-50436-6\_30",

language = "English",

isbn = "9783030504359",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "413--427",

editor = "Krzhizhanovskaya, \{Valeria V.\} and G{\'a}bor Z{\'a}vodszky and Lees, \{Michael H.\} and Sloot, \{Peter M.A.\} and Sloot, \{Peter M.A.\} and Sloot, \{Peter M.A.\} and Dongarra, \{Jack J.\} and S{\'e}rgio Brissos and Jo{\~a}o Teixeira",

booktitle = "Computational Science – ICCS 2020 - 20th International Conference, Proceedings",

}

Peng, Z, Lin, X & Niu, N 2020, Unit tests of scientific software: A study on swmm. in VV Krzhizhanovskaya, G Závodszky, MH Lees, PMA Sloot, PMA Sloot, PMA Sloot, JJ Dongarra, S Brissos & J Teixeira (eds), Computational Science – ICCS 2020 - 20th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12143 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 413-427, 20th International Conference on Computational Science, ICCS 2020, Amsterdam, Netherlands, 06/3/20. https://doi.org/10.1007/978-3-030-50436-6_30

Unit tests of scientific software: A study on swmm. / Peng, Zedong; Lin, Xuanyi; Niu, Nan.
Computational Science – ICCS 2020 - 20th International Conference, Proceedings. ed. / Valeria V. Krzhizhanovskaya; Gábor Závodszky; Michael H. Lees; Peter M.A. Sloot; Peter M.A. Sloot; Peter M.A. Sloot; Jack J. Dongarra; Sérgio Brissos; João Teixeira. Springer Science and Business Media Deutschland GmbH, 2020. p. 413-427 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12143 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Unit tests of scientific software

T2 - 20th International Conference on Computational Science, ICCS 2020

AU - Peng, Zedong

AU - Lin, Xuanyi

AU - Niu, Nan

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2020.

PY - 2020

Y1 - 2020

N2 - Testing helps assure software quality by executing program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating point computations. We report in this paper a manual analysis of the unit tests written by the developers of the Storm Water Management Model (SWMM). The results show that the 1,458 SWMM tests have a 54.0% code coverage and a 82.4% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests. Based on these results, we offer insights to improve test development and coverage.

AB - Testing helps assure software quality by executing program and uncovering bugs. Scientific software developers often find it challenging to carry out systematic and automated testing due to reasons like inherent model uncertainties and complex floating point computations. We report in this paper a manual analysis of the unit tests written by the developers of the Storm Water Management Model (SWMM). The results show that the 1,458 SWMM tests have a 54.0% code coverage and a 82.4% user manual coverage. We also observe a “getter-setter-getter” testing pattern from the SWMM unit tests. Based on these results, we offer insights to improve test development and coverage.

KW - Scientific software

KW - Storm Water Management Model (SWMM)

KW - Test coverage

KW - Test oracle

KW - Unit testing

KW - User manual

UR - https://www.scopus.com/pages/publications/85087274192

U2 - 10.1007/978-3-030-50436-6_30

DO - 10.1007/978-3-030-50436-6_30

M3 - Conference contribution

AN - SCOPUS:85087274192

SN - 9783030504359

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 413

EP - 427

BT - Computational Science – ICCS 2020 - 20th International Conference, Proceedings

A2 - Krzhizhanovskaya, Valeria V.

A2 - Závodszky, Gábor

A2 - Lees, Michael H.

A2 - Sloot, Peter M.A.

A2 - Dongarra, Jack J.

A2 - Brissos, Sérgio

A2 - Teixeira, João

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 3 June 2020 through 5 June 2020

ER -

Peng Z, Lin X, Niu N. Unit tests of scientific software: A study on swmm. In Krzhizhanovskaya VV, Závodszky G, Lees MH, Sloot PMA, Sloot PMA, Sloot PMA, Dongarra JJ, Brissos S, Teixeira J, editors, Computational Science – ICCS 2020 - 20th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 413-427. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-50436-6_30

Unit tests of scientific software: A study on swmm

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this