Higher Order Mutant Generation to Decrease the Cost of Mutation Testing Sudhir Kumar Mohapatra, Manoranjan Pradhan

1. International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access Online
Journal) Volume1, Issue2, Sept-Oct, 2014.ISSN:2349-7173(Online)
Higher Order Mutant Generation to Decrease the
Cost of Mutation Testing
Sudhir Kumar Mohapatra1, Manoranjan Pradhan2
______________________________________________
Abstract:
Although powerful, mutation could be a computationally
terribly high-priced testing technique. In fact, its 3 main
stages (mutant generation, mutant execution and result
analysis) need several resources to be with success
accomplished. Thus, researchers have created necessary
efforts to scale back its prices. This paper represents a
further effort during this sense. It describes the results of
2 experiments within which, by suggests that of mixing the
initial set of mutants and so getting a replacement set of
mutants—each one with 2 faults—the range of mutants
used is reduced to 0.5. Results cause believes that mutant
combination doesn't decrease the standard of the test
suite, whereas it supposes necessary savings in mutant
execution and result analysis.
_______________________________________________
Keyword: Mutant, First order mutant, higher order
mutant, Mutant generation, Mutant cost reduction
________________________________________________
Introduction:
Mutation is a testing technique that usually involves the
following three stages [2], [3], [4]:
(1) Mutant generation, whose goal is the generation of
mutants of the program under test.
(2) Mutant execution, whose goal is the execution of
test cases against the original program and the
mutants.
(3) Result analysis, whose goal is to check the
mutation score obtained by the test suite.
A mutant M of a program beneath test P may be a copy of
P, however it contains a little modification in its code, that
is taken as a fault. the thought of mutation testing is to jot
down test cases detection those faults: simplifying, the a lot
of the faults found by the test cases, the higher the standard
of the test suite.
_______________________________________________
First Author’s Name: Sudhir Kumar Mohapatra, Dept of Computer
science & engg. GITA, Bhubaneswar, Odisha.
Second Author’s Name: Manoranjan Pradhan,Dept of Computer
science & engg. GITA, Bhubaneswar, Odisha
____________________________________________________________
Mutant generation is typically meted out with machine-controlled
tools that apply a group of mutation operators to
the sentences of the first program, therefore manufacturing
a high range of mutants[9], [10], [11]. for instance, an easy
program with simply a sentence like come a+b (where a, b
ar integers) is also a minimum of mutated into twenty other
ways (a−b,a +b,a/b,a+b++,−a+b,a+−b,0+b,a+0, |a|+b,a+|b|,
etc.). this instance is illustrated in Figure one. Figure one
shows the ASCII text file of the first program and of some
mutants; Figure one presents the results obtained from
execution some test cases on the various program versions.
The test suit resembling the test information (1,1)produces
completely different outputs on the first program (whose
output is correct) and on Mutant 1;thus, this test suit has
found the fault introduced within the mutant, and it's
aforesaid that the mutant is killed. On the opposite hand, as
all test cases supply a similar output on the first program
and on Mutant four, it's aforesaid that Mutant four is alive.
Moreover, this mutant can ne'er be killed by any test suit, as
variable b is incremented when returning the result.
Mutants like this ar referred to as ‘functionally equivalent
mutants’ and should be thought-about as noise within the
result analysis step, as they represent obstacles in knowing
the standard of the test suite: they need a fault (actually, a
syntactical change) with regard to the first ASCII text file
that can't be found tho'. the standard of a take a look at suite
is measured in terms of the mutation score, that is that the
share of non-equivalent mutants killed. A take a look at
suite is mutation-adequate if its mutation score is 100
percent. As shown higher than, so as to visualize whether or
not a test suite is mutation-adequate, the detection of
equivalent mutants is needed [13], [14], [15]. However, this
can be a really expensive task, as a result of equivalent
mutants should be searched (usually by hand) among the
large range of mutants engineered up by the generation tool
applied. for instance, in a very additional study from 1996,
Offutt et al. [12] reduced the quantity of purposeful
operators of the Mothra mutation tool from twenty two to
five , therefore considerably reducing the quantity of the
test suite has additionally a powerful dependence on the
mutation operators applied [4]. The 3 same steps of
mutation testing ar sometimes terribly expensive , and
researchers have created vital efforts to cut back its prices.
this text describes a way to decrease the prices of mutation
testing. the thought relies on the reduction within the range
of mutants by combining mutant pairs generated by a tool
into new mutants, every with the 2 faults introduced within
the mutants they are available from.
All Rights Reserved © 2014 IJARTES Visit: www.ijartes.org Page 30

2. International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access
Online Journal) Volume1, Issue2, Sept
Fig 1. Source code and their mutants
Proposed methods
1. Decreasing the cost of mutation by mutant
combination:
Sept-Oct, 2014.ISSN:2349-7173(Online)
A substantive a part of the benefits of mutation testing
resides within the coupling impact, by that a take a look at
information set that detects all straightforward faults during
a program is thus sensitive that it conjointly detects a lot of
advanced faults. In fact, mutant generation tools introduce
one straightforward fault in every mutant. Given a group of
mutants containing one straightforward fa
to use identical mutant generation tool to every of those
mutants, so obtaining a replacement set of mutants,
wherever each has 2 faults (the initial continuing from the
primary mutant, and also the second, introduced within the
second generation). In testing literature, mutants containing
one straightforward fault area unit referred to as first
mutants and mutants containing 2 straightforward faults
area unit referred to as second-order mutants. As the
expansion of mutants range with every new generation is
exponential: ifn first-order mutants (M1 . . .Mn) area unit
obtained from P, then n second-order mutants are obtained
from every Mi .
As the range of equivalent mutants generated by mutation
tools is comparatively, our plan cons
range by combining pairs of first-order mutants to get a
group of second-order mutants, each with 2 faults. At a
primary look, the result's terribly completely different from
that shown in Figure three, because the range of mutants
obtained is near 0.5 the dimensions of the first suite .As
proportion of equivalent mutants is not up to the percentage
of non-equivalent mutants, every equivalent mutant are in
all probability combined with a non
mutant.When a first-order mutant Mi is combined with
another first-order mutant Mj to supply a second
mutant Mi, j , the ensuing mutant is also equivalent or non
All Rights Reserved © 2014 IJARTES Visit: www.ijartes.
org
methods:
fault, it's potential
first-order
ith consists in reducing its
ained the
non-equivalent
second-order
non-equivalent,
reckoning on the equivalence of Mi and Mj .
The potential cases area unit summarized in
fourth case of Table I could be a special case: normally, the
mix of 2 first-order non-equivalent mutants can turn out one
second-order non-equivalent mutant; but, it's potential that
the ensuing mutant is equivalent, as within the example of
Table II, wherever mutations area unit highlighted with .
However, in our experiment, this terribly inconceivable
case has not appeared. With these assumptions, and setting
aside the same inconceivable case of Table II, if a take a
look at suite T is mutation
order mutants M, T also will be mutation
wherever M’ is that the set of second
continuing from the mix of mutants in M.
Fig 2 No of mutant Generated
Fig 3 Higher order mutant generation
order mutant
2. Decreasing the cost of mutation by higher
order mutant
Mutants is classified into 2 types: 1st Order Mutants
(FOMs) higher Order Mutants (HOMs). FOMs are
generated by applying mutation operators one time. HOMs
are generated by applying mutation operators over once.
This paper introduces the thought of subsuming HOMs. A
subsuming HOMs is more durable to kill than the FOMs
from that it's made. As such, it
interchange the FOMs with the onl
paper introduces the thought of a powerfully subsuming
HOMs. A subsuming HOMs is simply killed by a set of the
Page 31
equivalent, Table I.The
ion-adequate for a group of first-order
mutation-adequate for M’
second-order mutants
by combining 1st
'it's going to be preferred to
only HOM. especially, the

3. International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access
Online Journal) Volume1, Issue2, Sept-Oct, 2014.ISSN:2349-7173(Online)
intersection of test cases that kill every FOM from that it's
made.
Consider a subsuming, h, made from the FOMs f1... fn. The
set of test cases that kill h additionally kill every and each
FOM f1... fn. Therefore, h will replace all of the mutants
f1... fn while not loss of take a look at effectiveness. The
converse doesn't hold; there exist take a look at sets that
kill all FOMs f1... fn however that fail to kill h. The FOMs
cannot, even taken together, replace the HOM while not
attainable loss of test effort. {this is|this is often|this is} the
sense during which h can be same to ‘strongly subsume’
f1... fn.
HOMs is classified in terms of the manner that they're
‘coupled’ and ‘subsuming’, as shown in Figure one. In
Figure one, the region space within the central diagram
represents the domain of all HOMs. The sub-diagrams close
the central region illustrate every class. For sake of
simplicity of exposition
these examples illustrate the second order mutant case; one
that assumes that there ar 2 FOMs f1 and f2, and h denotes
the HOM made from the FOMs f1 and f2. the 2 regions
represented by every sub diagram represent the test sets
containing all the take a look at cases that kill FOMs f1 and
f2. The shaded space represents the test set that contains all
take a look at cases that kill HOM h. The areas of the
regions indicate the proportion of the domain of HOMs for
every class. Following the coupling result hypothesis, if a
test set that kills the FOMs additionally contains cases that
kill the HOM, we have a tendency to shall say that the
HOM may be a ‘coupled HOM’, otherwise we have a
tendency to shall say it's a ‘de-coupled HOM’
Subsuming HOMs, by definition, is more durable to kill
than their constituentFOMs. Therefore, in Figure one, the
subsuming HOMs is described as those wherever the
shaded space is smaller than the world of the union of the 2
unshaded regions, like sub-diagrams (a), (b) and (c). against
this, (d), (e) and (f) ar non-subsuming. what is more, the
subsuming HOMs is classified into powerfully subsuming
HOMs and weak subsuming HOMs. By definition, if a
legal action kills a powerfully subsuming HOM, it
guarantees that its constituent FOMs ar killed furthermore.
2.1. Reasons to support hom
1) Cost: Work on Mutant Sampling and Selective Mutation
has shown however the amount of mutants are often
reduced with solely a little impact on check effectiveness
[1], [8], [7].
2) Uncertainty: Work on reducing the impact of equivalent
mutants has reduced, tho' not eradicated, this drawback [6],
[5], [4], [1].
3) Realism: Empirical proof has been on condition that the
faults denoted by mutants do, indeed, overlap with a
category of real faults [12], [5].
2.2. Proposed work
In order to explain Higher Order Mutation Testing we take
the following example.
Original Program
{
if ( (a>b) && (a>c))
………
}
We create a first order mutant of the Original Program by
adding a single fault i.e. we change a>b to a<b and name it
FOM1 as below:
FOM1
{
if ( (a<b) && (a>c))
………
}
We create another first order mutant of the Original
Program by adding a single fault i.e. we change a>c to a<c
and name it FOM2 as below:
FOM2
{
if ( (a>b) && (a<c))
………
}
FOM1 and FOM2 are first order mutants as they vary by a
single fault from the Original program. Now we create a
HOM from FOM1 and FOM2 by having more than one
fault from the Original program .Our HOM differs from
original program by 2 faults. We change a>b to a<b and
also a>c to a<c.
HOM
{
if ((a<b) && (a<c))
………
}
Now we prove that if we are able to find a subsuming HOM
in particular a strongly subsuming HOM, it will kill all the
FOM’s from which it is constructed thereby reducing the
number of test cases without loss of test case effectiveness
.We take simple example to find largest of 3 numbers a, b
and c. Our program takes as input 3 numbers and as output
it gives the largest of these numbers.
All Rights Reserved © 2014 IJARTES Visit: www.ijartes.org Page 32

4. International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access
Online Journal) Volume1, Issue2, Sept-Oct, 2014.ISSN:2349-7173(Online)
Original program
#include<stdio.h>
#include<conio.h>
void main ()
{
int a,b,c;
clrscr();
printf(“ENTER THE 3 NUMBERS”);
scanf(“%d%d%d”,&a,&b,&c);
if((a>b) &&(a>c))
printf(“A IS GREATEST”);
else if ((b>a) &&(b>c))
printf(“B IS GREATEST”);
else if ((c>a) && (c>b))
printf(“C IS GREATEST”);
else printf (“ WRONG RESULT”);
}
We create First Order Mutant FOM1 from the
Original_program by changing a> b to a<b FOM1
if((a<b) &&(a>c))
printf(“A IS GREATEST”);
else if ((b>a) &&(b>c))
printf(“B IS GREATEST”);
else if ((c>a) && (c>b))
printf(“C IS GREATEST”);
else
printf(“ WRONG RESULT”);
We create another First Order Mutant FOM2 from the
Original_program by changing a> c to a<c FOM2
if((a>b) &&(a<c))
printf(“A IS GREATEST”);
else if ((b>a) &&(b>c))
printf(“B IS GREATEST”);
else if ((c>a) && (c>b))
printf(“C IS GREATEST”);
else
printf(“ WRONG RESULT”);
We create Higher order mutant HOM from First order
mutant FOM1 and FOM2 by changing a> b to a<b and a>c
to a<c. This differs from Original_program by 2 faults.
HOM
if((a<b) &&(a<c))
printf(“A IS GREATEST”);
else if ((b>a) &&(b>c))
printf(“B IS GREATEST”);
else if ((c>a) && (c>b))
printf(“C IS GREATEST”);
else
printf(“ WRONG RESULT”);
TABLE 1: TEST CASE WHICH KILL FOM1, FOM2 & HOM
From the Table-1 we find that there are 2 test cases which
kill FOM1 and also 2 test cases which kill FOM2. There is
one test case which kills HOM and is found in the
intersection of FOM1 and FOM2. This test case kills both
FOM1 and FOM2. The converse is not true. The test case
((a<b) && (a>c)) which kills FOM1 does not kill FOM2
and HOM. Similarly the test case ((a>b) && (a<c)) which
kills FOM2 does not kill FOM1 and HOM. From the Table-
2 its clear that the test case a<b && a>c which kills FOM1
and the Original_program does not kill FOM2 and HOM,
similarly the test case a>b && a<c which kills FOM2 and
the Original_program does not kill FOM1 and HOM but the
test case a>b &&a>c found in the intersection of FOM1 and
FOM2 kills our HOM and also FOM1 and FOM2. So if use
this test case automatically both FOM1 and FOM2 will get
killed thereby reducing the number of test cases without
leading to loss of effectiveness.
All Rights Reserved © 2014 IJARTES Visit: www.ijartes.org Page 33

5. International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access
Online Journal) Volume1, Issue2, Sept
TABLE 2:TEST DATA WITH FOM1, FOM2 & HOM
Conclusion:
Sept-Oct, 2014.ISSN:2349-7173(Online)
In this paper, we have a tendency to examine the utilities of
upper Order Mutation Testing by making mutants first
initial and higher Order. From the higher than work we
have a tendency to conclude that although HOM’s area unit
tough to make however if we have a tendency to area unit
able to notice a HOM than the amount of test cases scale
backs that ultimately reduce the cost of testing.
References
[1] R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J.
Offutt. An Extended Overview of the Mothra Software Testing
Environment. In Proceedings of the 2nd Workshop on Software Testing,
Verification, and Analysis (TVA’88), pages 142
Canada, July 1988. IEEE Computer society.
[2] T. A. Budd. Mutation Analysis of Program Test Data. Phd thesis, Yale
University, New Haven, Connecticut, 1980.
[3] W. E.Wong. On Mutation and Data Flow. Phd thesis, Purdue
University, West Lafayette, Indiana, 1993.
[4] A. P. Mathur and W. E. Wong. An Empirical Comparison of Mutation
and Data Flow Based Test Adequacy Criteria. Technique repo
University, West Lafayette, Indiana, 1993.
[5] A. S. Namin and J. H. Andrews. On Sufficiency of Mutants. In
Proceedings of the 29th International Conference on Software Engineering
(ICSE COMPANION’07), pages 73–74, Minneapolis, Minnesota, 20
May 2007.
[6] A. P. Mathur. Performance, Effectiveness, and Reliability Issues in
Software Testing. In Proceedings of the 5th International Computer
Software and Applications Conference (COMPSAC’79), pages 604
Tokyo, Japan, 11-13 September 1991.
All Rights Reserved © 2014 IJARTES Visit: www.ijartes.
org
142–151, Banff Alberta,
report, Purdue
20-26
604–605,
[7] M. Sahinoglu and E. H. Spafford. A Bayes Sequential Statistical
Procedure for Approving Software Products. In Proceedings of the IFIP
Conference on Approving Software Products (ASP’90), pages 43
Garmis Partenkirchen, Germany, September 1990. Elsevier
[8] R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J.
Offutt. An Extended Overview of the Mothra Software Testing
Environment. In Proceedings of the 2nd Workshop on Software Testing,
Verification, and Analysis (TVA’88), pages 142
Canada, July 1988. IEEE Computer society.
[9] A. J. Offutt, G. Rothermel, and C. Zapf. An Experimental Evaluation
of Selective Mutation. InProceedings of the 15th International Conference
on Software Engineering (ICSE’93), pages 100
May 1993. IEEE Computer Society Press.
[10] W. E. Wong and A. P. Mathur. Reducing the Cost of Mutation
Testing: An Empirical Study. Journal of Systems and Software, 31(3):185
196, December 1995.
[11] K. N. King and A. J. Offutt. A Fortran Language System for
Mutation- Based Software Testing Software: Practice and Experience,
21(7):685–718, October 1991
[12] E. S. Mresa and L. Bottaci. Efficiency of Mutation Operators and
Selective Mutation Strategies: An Empirical Study. Software Testing,
Verification and Reliability, 9(4):205
[13] A. S. Namin and J. H. Andrews. Finding Sufficient Mutation
Operators via Variable Reduction. In Proceedings of the 2nd Workshop on
Mutation Analysis (MUTATION’06), page 5, Raleigh, North Carolina,
November 2006. IEEE Computer Society.
[14] A. S. Namin and J. H. Andrews. On Sufficiency of Mutants. In
Proceedings of the 29th International Conference on Software Engineering
(ICSE COMPANION’07), pages 7
May 2007.
[15] A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf. An
Experimental Determination of Sufficient Mutant Operators. ACM
Transactions on Software Engineering and Methodology, 5(2):99
April 1996.
Author Profile
Sudhir kumar Mohapatra is an Assistant Professor in the
Department of Computer Science& Engineering, Gandhi
Institute for Technological Advancement (GITA),
Bhubaneswar, Odisha ,India.
national seminars and international journals. His
includes software engineering and soft computing
Manoranjan Pradhan holds a Ph. D Degree in
Science. He is presently working as a
the Department of Computer
Gandhi Institute for
Bhubaneswar, Odisha, India.
experience. He has published many
journals. His research interests include Computer Se
Detection, Soft Computing & Cloud computing
Page 34
] 43–56
Science.
142–151, Banff Alberta,
100–107, Baltimore, Maryland,
185–
gies: 205–232, December 1999.
s 73–74, Minneapolis, Minnesota, 20-26
99–118,
udhir He published many papers in
research interest
computing.
Computer
professor and Head of
Science & Engineering,
Technological Advancement (GITA),
He has 15 years of teaching
papers in national and international
Security, Intrusion
computing.