Software Mutational Robustness: Bridging The Gap Between Mutation Testing and Evolutionary Biology

Created by W.Langdon from gp-bibliography.bib Revision:1.4202

  author =       "Eric Schulte and Zachary P. Fry and Ethan Fast and 
                 Stephanie Forrest and Westley Weimer",
  title =        "Software Mutational Robustness: Bridging The Gap
                 Between Mutation Testing and Evolutionary Biology",
  howpublished = "arXiv",
  year =         "2012",
  month =        "18 " # apr,
  keywords =     "genetic algorithms, genetic programming, SBSE,
                 mutational robustness, proactive diversity, mutation
                 testing, equivalent mutant, reliability, CIL, intel
                 64bit x86 assembler",
  URL =          "",
  size =         "11 pages",
  abstract =     "In the mutation testing paradigm, test suite quality
                 is measured by its ability to detect (or kill) variant
                 programs generated through application of random
                 changes to an original program. The ideal test suite is
                 sensitive to any semantic change. In evolutionary
                 biology, however, neutral mutations that leave fitness
                 unchanged are considered to be beneficial improving the
                 system's robustness and its ability to discover
                 evolutionary improvements. In this paper, we generate a
                 population of variant programs, created from an
                 original program by applying lightweight random
                 mutations. We adopt biological terminology and refer to
                 unkilled variants as neutral, and the percentage of all
                 variants that are neutral is defined as the program's
                 mutational robustness. Although they are related to
                 equivalent mutants in mutation testing, which are
                 viewed as problematic, we demonstrate positive
                 properties of neutral variants which easily generated
                 and can be leveraged to protect software against
                 unknown defects.

                 Even when the mutations are restricted to statements
                 executed by the test cases, we find that mutational
                 robustness is surprisingly high: 36.75percent on a
                 corpus of 22 programs including production software
                 projects, the Siemens benchmark suite, and a few
                 specially constructed programs. The results hold for
                 mutations at both the source code and assembly
                 instruction levels, across various programming
                 languages, and are not fully explained by test suite
                 quality. We conclude that mutational robustness is an
                 inherent property of software, existing even when a
                 program is correct according to its specification.
                 Rather than an overhead cost or indicator of test suite
                 inadequacy, it is an opportunity to create useful
                 proactive diversity because neutral mutations often
                 cause nontrivial algorithmic changes. To demonstrate
                 this, we generate and select diverse populations of
                 neutral program variants. For a program with seven or
                 more held-out latent bugs, we can, on average,
                 construct and select seven neutral variants such that
                 at least one fixes a latent bug.",
  notes =        "

                 Early draft of \cite{Schulte:2014:GPEM}",
  bibsource =    "DBLP,",

Genetic Programming entries for Eric Schulte Zachary P Fry Ethan Joseph Fast Stephanie Forrest Westley Weimer