Data Flow Testing as Model Checking - Research Article | CS 576, Papers of Computer Science

Download Data Flow Testing as Model Checking - Research Article | CS 576 and more Papers Computer Science in PDF only on Docsity! Data Flow Testing as Model Checking∗ Hyoung Seok Hong, Sung Deok Cha Department of Electrical Engineering and Computer Science and AITrc Korea Advanced Institute of Science and Technology {hshong,cha}@salmosa.kaist.ac.kr Insup Lee, Oleg Sokolsky Department of Computer and Information Science University of Pennsylvania {lee,sokolsky}@saul.cis.upenn.edu Hasan Ural School of Information Technology and Engineering University of Ottawa ural@site.uottawa.ca Abstract This paper presents a model checking-based approach to data flow testing. We characterize data flow oriented cover- age criteria in temporal logic such that the problem of test generation is reduced to the problem of finding witnesses for a set of temporal logic formulas. The capability of model checkers to construct witnesses and counterexamples allows test generation to be fully automatic. We discuss complexity issues in minimal cost test generation and describe heurstic test generation algorithms. We illustrate our approach us- ing CTL as temporal logic and SMV as model checker. 1 Introduction During the last two decades, there have been a num- ber of data flow testing methods. Included are those pro- posed by Rapps and Weyuker[26], Ntafos[24], Ural[30], and Laski and Korel[22], which are originally devel- oped for modules in procedural languages. These meth- ods have been extended for interprocedural programs in procedural languages[13], object-oriented programmming languages[14], and requirements specification languages such as SDL[29, 31] and statecharts[17]. In data flow test- ∗This research was supported in part by Advanced Information Tech- nology Research Center at KAIST, NSF CCR-9988409, NSF CCR- 0086147, NSF CCR-0209024, ARO DAAD19-01-1-0473, DARPA ITO MOBIES F33615-00-C-1707, and the Natural Sciences and Engineering Research Council of Canada under grant OGP00000976. ing, we usually model a software as a flow graph which identifies the information of control flow and data flow in the software. We then establish certain associations be- tween definitions and uses of variables required to be cov- ered in a given coverage criterion by applying conventional data flow analysis upon the flow graph. Finally we select a finite number of paths which cover the associations as a test suite. Model checking is a formal verification technique for de- termining whether a system model satisfies a property writ- ten in temporal logic and model checkers such as SMV[23] and SPIN[16] are already used on a regular basis for the verification of real-world applications. In addition to being automatic, an important feature of model checking is the ability to explain the success or failure of a temporal logic formula[5, 6, 15]. If a system model satisfies a formula, model checkers are capable of supplying an execution of the model as a witness demonstrating the success of the for- mula. Conversely, a counterexample is supplied when the model fails to satisfy the formula. This paper presents a model checking-based approach to data flow testing. In our approach, the problems of data flow analysis and path selection in data flow testing are for- mulated in terms of model checking. We investigate four groups of coverage criteria in [26, 24, 30, 22] and character- ize each coverage criterion by specifying the requirements of the coverage criterion using a set of temporal logic for- mulas such that the problem of test generation is reduced to the problem of finding witnesses for the set of formu- las. The capability of model checkers to construct witnesses
                 !" #   and counterexamples allows test generation to be fully au- tomatic. As a by-product, the characterization enables us to discuss complexity issues in minimal cost test genera- tion. This paper illustrates our approach using CTL[4] as temporal logic and SMV[23] as model checker. The main advantages of our approach may be summarized as follows: First, the approach enables test generation from large flow graphs whose size is limited by the capabilities of current model checkers. Second, the approach allows focusing on only high-level specifications of coverage criteria written in temporal logic. All the details about test generation algo- ritms and their implementations are hidden in model check- ers. Third, the approach is language independent in that the temporal logic formulas employed in the approach are applicable with minor modifications to flow graphs con- structed from various kinds of programming languages and requirements specification languages. Connections between data flow analysis and model checking were made in [27, 28] which show that model checking can be used to solve various data flow analysis problems including the standard bit-vector problems. Our approach extends the work of [27, 28] in that data flow testing combines data flow analysis with the path selec- tion problem. Recently, connections between test genera- tion and model checking have been considered especially in specification-based testing. In [20], local and on-the-fly model checking algoritms are applied to test generation. In [32], SPIN is used for on-the-fly test generation. Test gen- eration using the capability of model checker to construct counterexamples has been applied in several contexts. In [1], the application of model checking to mutation analysis is described. In [3, 9], tests are generated by constructing counterexamples for user-supplied temporal logic formulas. In [12], the capability of SMV and SPIN to construct coun- terexamples is applied to test generation for control flow oriented coverage criteria. No consideration is given to data flow testing in the above work. In [18, 19], the authors discuss the application of model checking to test generation from requirements specifica- tions for both control flow and data flow oriented coverage criteria. The approach in [18, 19] is based on the fact that the state space of a specification is often finite and hence one can use reachability graphs instead of flow graphs for test generation. On one hand, this paper extends [18, 19] by considering more comprehensive groups of data flow ori- ented coverage criteria. On the other hand, the flow-graph approach we advocate here can be seen as complementary to the reachability-graph approach in [18, 19]. In the flow- graph approach one can generate tests from programs or specifications with infinite state space because the values of variables are not expanded in flow graphs. It, however, requires posterior analysis such as symbolic execution or constraint solving to determine the executability of tests and for the selection of variable values which make tests exe- cutable. The reachability-graph approach can handle only finite state space but has the advantage that only executable tests are generated which obviates the necessity of posterior analysis. Section 2 briefly reviews the basics of flow graph and CTL which are the model and logic employed in our ap- proach, respectively. Section 3 characterizes the coverage criteria in [26, 24, 30, 22] by associating a CTL formula, parameterized with the propositions of a given flow graph, with each entity required to be covered in a given criterion. Each formula is defined in such a way that a flow graph sat- isfies the formula if and only if the flow graph has an execu- tion covering the entity described by the formula. By find- ing witnesses for every formula in a given criterion, we gen- erate a test suite satisfying the criterion. Section 4 discusses complexity issues in minimal cost test generation. Typically a CTL formula can have several executions as its witness. By selecting the right witness for each formula, one can minimize the size of the test suite. We show that two opti- mization problems of minimal cost test generation are NP- hard and describe heuristic test generation algorithms em- ploying the capability of model checkers to construct coun- terexamples. We report the experimental results obtained by applying the heuristics to a moderate flow graph. In our experience with SMV, we were able to generate test suites from flow graphs containing dozens of variable definitions and uses in seconds. Finally, Section 5 concludes the paper with a discussion of future work. 2 Flow Graph and CTL A flow graph G = (V ,vs,vf ,A) is a directed graph where V is a finite set of vertices; vs ∈ V is the start vertex; vf ∈ V is the final vertex; and A is a finite set of arcs. A vertex represents a statement and an arc represents possible flow of control between statements. We adopt the following convention to decorate each vertex with data flow informa- tion. Let x be a variable and v be a vertex. We say that x is defined at v, denoted by dxv , if v represents a statement assigning a value to x. We say that x is used at v, denoted by uxv , if v represents a statement referencing x. We use DEF(v) and USE(v) to denote the sets of definitions and uses at v, respectively. A sequence v1...vn of vertices is a path if (vi,vi+1) ∈ A for 1 ≤ i ≤ n− 1. A path is complete if it starts from the start vertex vs and ends at the final vertex vf . A test sequence is a complete path and a test suite is a finite set of test sequences. Figure 1 shows a program and its flow graph. We view a flow graph as a Kripke structure M = (Q,qinit ,L,R) where Q is a finite set of states; qinit ∈ Q is the initial state; L: Q → 2AP is the function labelling each state with a subset of the set AP of atomic proposi-
                 !" #   path with respect to x from v to v′ is covered by a test se- quence in Π. A test suite Π satisfies all-uses coverage crite- rion if and only if it is a witness-set for {wctl(dxv , uxv′) | dxv ∈ DEF (G), uxv′ ∈ USE (G)}. In the worst case, the number of formulas can be quadrac- tic in the size of a flow graph since the number of pairs (dxv , u x v′) can be O(n 2) in a flow graph of size n. For example, for all-uses coverage criterion in Fig- ure 1 we associate 11 formulas with the pairs (dxv1 ,u x v2), (dxv1 ,u x v3), (d y v1 ,u y v2), (d y v1 ,u y v4), (d z v1 ,u z v6), (d max v3 ,u max v6 ), (dmaxv3 ,u max v7 ), (d max v4 ,u max v6 ), (d max v4 ,u max v7 ), (d max v6 ,u max v6 ), and (dmaxv6 ,u max v7 ). Among them, the formulas for (dmaxv3 ,u max v7 ), (d max v4 ,u max v7 ), and (d max v6 ,u max v6 ) are not satis- fied in Figure 1, which means that the pairs are not du-pairs. A test suite Π satisfies all-du-paths coverage criterion if, for every definition dxv and every use u x v′ , every cycle-free definition-clear path with respect to x from v to v′ is cov- ered by a test sequence in Π. Unlike other coverage crite- ria, all-du-paths coverage criterion cannot be characterized in terms of witness-sets. To generate test suites satisfying this criterion properly in our approach, we should be able to construct all cycle-free witnesses instead of only one for a given formula, which is beyond the capability of existing model checkers. In general, extending model checkers to construct all witnesses for a given formula or a subset of witnesses satisfying certain constraints is an open problem. 3.2 Ntafos’ Criteria Ntafos’ criteria emphasize interactions between differ- ent variables[24]. Such interactions are captured in terms of sequences of alternating definitions and uses, called k-dr interactions. A sequence [dx1v1 u x1 v2 d x2 v2 u x2 v3 ... d xn vn u xn n+1] is a data flow chain (df-chain) if, for every 1 ≤ i ≤ n, (dxivi , u xi vi+1 ) is a du-pair[30]. Note that the use u xi vi+1 and definition dxi+1vi+1 occur at the same vertex for every 1 ≤ i ≤ n. A path v1π1v2π2...vn+1 is an interaction subpath of a df- chain if, for every 1 ≤ i ≤ n, viπivi+1 is a definition-clear path from vi to vi+1 with respect to xi. A df-chain consist- ing of k − 1 du-pairs, k ≥ 2, is a k-definition/reference in- teraction (k-dr interaction) in the terminology of [24]1. For example, in Figure 1 we observe that [dxv1 u x v3 d max v3 u max v6 ] is a 3-dr interaction which has v1v2v3v5v6 as its interaction subpath. 3.2.1 Characterization For a sequence κ = [dx1v1 u x1 v2 d x2 v2 u x2 v3 ... d xk−1 vk−1 u xk−1 k ], k ≥ 2, define wctl(κ) as follows. 1We do not require the variables x1, ..., xn and the vertices v1, ..., vn+1 be distinct. This definition is consistent with that of Clarke et al.[7] and Ntafos[25] and is different from the original one[24] which requires the vertices to be distinct. • if κ is empty, then wctl(κ) = EFfinal, • if κ is [dxivi uxivi+1] · κ′, then wctl(κ) = dxivi∧ EXE[¬def(xi) U (uxivi+1 ∧ wctl(κ′))], • wctl(κ) = EFwctl(κ). By induction on the number of pairs (dxivi , u xi vi+1) in κ, it can be shown that κ is a k-dr interaction if and only if the Kripke structureM (G) of a flow graph G satisfies wctl(κ). Moreover, a test sequence covers κ if and only if it is a witness for wctl(κ). For example, a test sequence covering the 3-dr interaction [dxv1 u x v3 d max v3 u max v6 ] is shown in Fig- ure 4, which is also a witness for EF(dxv1∧ EXE[¬def(x) U (uxv3 ∧ dmaxv3 ∧ EXE[¬def(max) U (umaxv6 ∧ EFfinal)])]). ✒✑ ✏ vs start ✲ ✒✑ ✏ v1 dxv1 ✲ ✒✑ ✏ v2 ¬def(x) ✲ ✒✑ ✏ v3 dmaxv3 uxv3 ✲ ✒✑ ✏ v5 ¬def(max) ✲ ✒✑ ✏ v6 umaxv6 ✲ ✒✑ ✏ v7 ✲✒✑ ✏ vf final Figure 4. A test sequence covering 3-dr inter- action [dxv1 u x v3 d max v3 u max v6 ] A test suite Π satisfies required k-tuples coverage crite- rion if, for every k-dr interaction κ, some interaction sub- path of κ is covered by a test sequence in Π. A test suite Π satisfies required k-tuples coverage criterion if and only if it is a witness-set for {wctl([dx1v1ux1v2dx2v2ux2v3 ...dxk−1vk−1u xk−1 k ]) | dxivi ∈ DEF (G), uxivi+1 ∈ USE (G), 1 ≤ i ≤ k − 1}. 3.3 Ural’ Criteria Ural’s criteria also emphasize interactions between dif- ferent variables[30]. While Ntafos’ criteria consider df- chains consisting of fixed number of du-pairs, Ural’s crite- ria consider df-chains consisting of an arbitrary (but finite) number of du-pairs which start with inputs and end with outputs. The rationale here is to identify the functionality of a module in terms of the interactions with its environ- ment by identifying the effects of inputs accepted from the environment on outputs offered to the environment. We say that a definition dxv affects a use u x′ v′ if • either x = x′ and (dxv , ux ′ v′ ) is a du-pair or • there is a use uxv′′ such that (dxv , uxv′′) is a du-pair and there is a definition dx ′′ v′′ , given in terms of u x v′′ , that affects ux ′ v′ .
                 !" #   A pair (dxv , u x′ v′ ) is an affect-pair if d x v affects u x′ v′ . Among the particular affect-pairs of interest to Ural’s criteria are those starting with inputs and ending with outputs, which we call io-pairs. We define an input as a definition at an input statement and an output as a use at an output state- ment. For example, dxv1 , d y v1 , d z v1 are inputs and u max v7 is an output in Figure 1. We observe that the input dxv1 affects the output umaxv7 through the df-chain [d x v1 u x v3 d max v3 u max v6 dmaxv6 u max v7 ]. 3.3.1 Simple Characterization For an affect-pair (dxv , u x′ v′ ), define CHAIN(d x v , u x′ v′ ) as the set of sequences κ = [dx1v1 u x1 v2 d x2 v2 u x2 v3 ... d xn vn u xn n+1] such that dx1v1 = d x v and u xn vn+1 = u x′ v′ . In general, there may be multiple occurrences of the same pair (dxivi , u xi vi+1) in κ thereby causing the possibility of an infinite number of ele- ments in CHAIN(dxv , u x′ v′ ). To ensure that CHAIN(d x v , u x′ v′ ) be finite, we consider its subset SCHAIN(dxv , u x′ v′ ) consist- ing of simple sequences in which at most one occurrence of each pair (dxivi , u xi vi+1) is allowed. A test suite Π satisfies all-inputs coverage criterion if, for every input i and some output o, an iteraction subpath of some simple df-chain in SCHAIN(dxv , u x′ v′ ) is covered by a test sequence in Π. Let IN(G) and OUT(G) be the sets of inputs and outputs inG, respectively. A test suite Π satisfies all-inputs coverage criterion if and only if it is a witness-set for { ∨ o∈OUT(G) ∨ κ∈SCHAIN (i,o) wctl(κ) | i ∈ IN (G)}. A test suite Π satisfies all-outputs coverage criterion if, for every input i and every output o, an iteraction subpath of some simple df-chain in SCHAIN(dxv , u x′ v′ ) is covered by a test sequence in Π. A test suite Π satisfies all-outputs coverage criterion if and only if it is a witness-set for { ∨ κ∈SCHAIN (i,o) wctl(κ) | i ∈ IN (G), o ∈ OUT (G)}. A test suite Π satisfies all-IO-df-chains coverage crite- rion if, for every input i and every output o, an iteraction subpath of every simple df-chain in SCHAIN(dxv , u x′ v′ ) is covered by a test sequence in Π. A test suite Π satisfies all- IO-df-chains coverage criterion if and only if it is a witness- set for {wctl(κ) | i ∈ IN (G), o ∈ OUT (G), κ ∈ SCHAIN (i, o)}. 3.3.2 Fixpoint Characterization The above characterization of all-inputs and all-outputs coverage criteria is naive in that we need to identify all sim- ple sequences in SCHAIN(i, o) for a given io-pair in order to generate a test sequence covering just one simple df-chain for the io-pair. A more faithful characterization should al- low the generation of a test sequence without identifying all simple sequences in SCHAIN(i, o) prior to test genera- tion. Put another way, we like to model-check a new for- mula Q(dxv , u x′ v′ ) whose semantics is defined below without model-checking all formulas wctl(κ). q |= Q(dxv , ux ′ v′ ) if and only if q |= wctl(κ) for some κ in CHAIN(dxv , u x′ v′ ). We note that Q(dxv , u x′ v′ ) is not directly expressible in CTL because there is in general an infinite number of κ in CHAIN(dxv , u x′ v′ ) and thus an infinite number of wctl(κ). The formula Q(dxv , u x′ v′ ) leads to a natural characteriza- tion of all-inputs and all-outputs coverage criteria as fol- lows: A test suite Π satisfies all-inputs+ coverage criterion (resp. all-outputs+ coverage criterion) if, for every input i and some output o (resp. every output o), an interaction subpath of some df-chain2 in CHAIN(dxv , u x′ v′ ) is covered by a test sequence in Π. A test suite Π satisfies all-inputs+ coverage criterion if and only if it is a witness-set for { ∨ o∈OUT(G) Q(i, o) | i ∈ IN (G)}. A test suite Π satisfies all-outputs+ coverage criterion if and only if it is a witness-set for {Q(i, o) | i ∈ IN (G), o ∈ OUT (G)}. Finally we make a sketch of how to model-check Q(dxv , ux ′ v′ ). Although the formula is not in CTL, it has a symbolic model checking algorithm similar to that of CTL because it can be characterized as a fixpoint of a predicate transformer. In fact, the formula is directly expressible in alternation- free mu-calculus which has a linear-time model-checking algorithm[8]. By the definition of affect-pairs, we have the following equivalence. Q(dxv ,u x′ v′ ) = EFQ(d x v ,u x′ v′ ) Q(dxv ,u x′ v′ ) = (d x v∧EXE[¬def(v)U(ux ′ v′∧EFfinal)])∨ (dxv ∧ EXE[¬def (v)U ∨ ux v′′∈USE(G) (uxv′′ ∧Q(dx ′′ v′′ , u x′ v′ ))]) where dx ′′ v′′ is the definition of x ′′ occurring at v′′ for some x′′. Let τ : 2Q → 2Q be a predicate transformer defined by τ (Z) = (dxv∧EXE[¬def(v)U(ux ′ v′∧EFfinal)])∨ (dxv∧EXE[¬def (v)U ∨ ux v′′∈USE(G) (uxv′′∧Z[x′′/x, v′′/v])) where Z[x′′/x, v′′/v] is the formula obtained by replacing each occurrence of x and v in Z by x′′ and v′′, respectively. 2We do not require a df-chain be simple here.
                 !" #   Theorem 1 Q(dxv , u x′ v′ ) is a least fixpoint of τ . PROOF Assume that Z1 ⊆ Z2. Then τ(Z1) ⊆ τ(Z2) be- causeZ1[x′′/x, v′′/v] ⊆ Z2[x′′/x, v′′/v] and the modal op- erator U is monotonic. Hence τ is monotonic. Let Zf be Q(dxv , u x′ v′ ). It is easy to see that Zf = τ(Zf ) and hence Zf is a fixpoint of τ . To prove that Zf is a least fixpoint of τ , it is sufficient to show that Zf = ∪iτ i where τ0(Z) = Z and τ i+1(Z) = τ(τ i(Z)). We first prove that τ i(false) ⊆ Zf for every i. Clearly, τ0(false) ⊆ Zf . Assume that τ i(false) ⊆ Zf . Be- cause τ is monotonic, τ i+1(false) ⊆ τ(Zf ). Because Zf is a fixpoint of τ , τ i+1(false) ⊆ Zf . Hence we have the first direction ∪iτ i(false) ⊆ Zf . The other direction, Zf ⊆ ∪iτ i(false), is proved by induction on the number of du- pairs. Suppose that q0 |= Zf , then there is a path q0q1... covering a df-chain for (dxv , u x′ v′ ). Let j ≥ 1 be the number of du-pairs of the df-chain. We show that q0 ∈ τ j(false) for every j. For the base case j = 1, we have that x = x′ and q0 |= dxv∧ EXE[¬def(v) U (ux ′ v′∧ EFfinal)]. Hence q0 ∈ τ1(false). For the inductive step, suppose q0 ∈ τ j(false) for j = n. Let j = n + 1 and qk be the state in q0q1... at which the first du-pair in the df-chain ends. Then there are n du-pairs from qk and qk ∈ τn(false) by the induction hy- pothesis. Hence q0 |= (dxv ∧ EXE[¬def(v)U ∨ ux v′′∈USE(G) (uxv′′∧ τn(false)[x′′/x, v′′/v])]) and q0 ∈ τn+1(false). 3.4 Laski and Korel’s Criteria Laski and Korel’s criteria emphasize that a vertex may contain uses of several different variables in which each use may be reached by several different definitions[22]. Such definitions constitute the definition context of the vertex. Let v be a vertex and {ux1v , ..., uxnv } be a subset of USE(v). An ordered definition context of v with respect to {ux1v , ..., uxnv } is a sequence [dx1v1 ... dxnvn ] of definitions such that there is a subpath v1π1v2π2...πnv, called orderded con- text subpath, satisfying the following property: for every 1 ≤ i ≤ n, viπivi+1...πnv is a definition-clear path from vi to v with respect to xi. A definition context of v is a set of definitions, some permutation of which is an ordered defi- nition context of v. For example, consider the vertex v6 in Figure 1. [dzv1 d max v6 ] is an ordered definition context of v6 with respect to {uzv6 umaxv6 } whose ordered context subpath is v1v2v3v5v6. 3.4.1 Characterization Let v be a vertex and {ux1v , ..., uxnv } be a subset of USE(v). For a sequence λ = [dx1v1 ... d xn vn ] of definitions, define wctl(λ) as follows. • if λ is empty, then wctl(λ,nodef) = ux1v ∧ ... ∧ uxnv ∧EFfinal , • if λ is [dxivi ] · λ′, then wctl(λ,nodef) = nodef ∧ dxivi∧ EXE[nodef′ U wctl(λ′,nodef′)]), where nodef′ = nodef ∧ ¬def(vi), • wctl(λ) = EFwctl(λ,true). By induction on the number of definitions in λ, it can be shown that λ is an ordered definition context of v with re- spect to {ux1v , ..., uxnv } if and only if the Kripke structure M (G) of a flow graph G satisfies wctl(λ). Moreover, a test sequence covers λ if and only if it is a witness for wctl(λ). For example, a test sequence covering the or- dered definition context [dzv1 d max v3 ] with respect to {uxv6 , umaxv6 } is shown in Figure 5, which is also a witness for EF(dzv1∧ EXE[¬def(z) U (¬def(z) ∧dmaxv3 ∧ EXE[(¬def(z)∧ ¬def(max)) U (uzv6 ∧ umaxv6 ∧ EFfinal)])]). ✒✑ ✏ vs start ✲ ✒✑ ✏ v1 dzv1 ✲ ✒✑ ✏ v2 ¬def(z) ✲ ✒✑ ✏ v3 dmaxv3 ¬def(z) ✲ ✒✑ ✏ v5 ¬def(max) ¬def(z) ✲ ✒✑ ✏ v6 umaxv6 uzv6 ✲ ✒✑ ✏ v7 ✲✒✑ ✏ vf final Figure 5. A test sequence covering ordered context [dzv1 d max v3 ] with respect to {uzv6 , umaxv6 } A test suite Π satisfies context coverage criterion if, for every vertex v and every definition context dc of v, an or- dered context subpath of dc is covered by a test sequence in Π. A test suite Π satisfies context coverage criterion if and only if it is a witness-set for {wctl({dx1v1 , ..., dxnvn }) | v ∈ V, uxiv ∈ USE (v), dxivi ∈ DEF (G), 1 ≤ i ≤ n} where wctl({dx1v1 , ..., dxnvn }) is defined as wctl(λ1) ∨...∨ wctl(λn), where λ1, ..., λn are the permutations of {dx1v1 , ..., dxnvn }. A test suite Π satisfies ordered context coverage crite- rion if, for every vertex v and every ordered definition con- text odc of v, an ordered context subpath of odc is covered by a test sequence in Π. A test suite Π satisfies ordered context coverage criterion if and only if it is a witness-set for {wctl([dx1v1 , ..., dxnvn ]) | v ∈ V, uxiv ∈ USE (v), dxivi ∈ DEF (G), 1 ≤ i ≤ n}.
                 !" #   to generate test suites from flow graphs with 220 vertices and one hundred formulas in one minute and flow graphs with 270 vertices and one hundred formulas in one hour. Of course, further experiments are compulsory to demonstrate the feasibility of our approach when applied to data flow testing with huge state space. We are planning to extend our approach for interproce- dural programs and object-oriented programs. Data flow testing of such programs is more complicated due to proce- dure call/return, recursion, and reference parameters as well as global variables. Data flow testing methods for such pro- grams were proposed in [13, 14] which employ interproce- dural data flow analysis. Recently, the problem of interpro- cedural data flow analysis has been formulated as a model checking problem[2, 10]. Combining both work together may be a starting point for developing a model checking- based approach to data flow testing of interprocedural pro- grams and object-oriented programs. We showed that a subclass of CTL, which we call WCTL, is expressive enough to characterize a number of data flow oriented coverage criteria except those by Ural[30]. For Ural’s criteria, we extended WCTL with least fixpoints so that model checking of the resulting logic can be readily implemented in existing model checkers for CTL such as SMV. To characterize the criteria considered in this paper in a more uniform way, it is necessary to employ a logic more powerful than CTL. We are currently work- ing with a subclass of mu-calculus[21], more specifically alternation-free mu-calculus[8], which supports the explicit use of fixpoint operators. We cannot directly use linear time temporal logic for the characterization of data flow oriented coverage criteria, be- cause it requires existential quantification over paths. It is, however, possible to construct a witness for a WCTL for- mula using linear time model checkers by exploiting the fact that a path is a witness for a WCTL formula if and only if the path is a counterexample for its negation. For example, we can construct a witness for a WCTL formula EFEFp by finding a counterexample for ¬EFEFp = AGAG¬p, which is in turn equivalent to the LTL formula AGG¬p. This opens the possibility of applying linear time model checkers such as SPIN to data flow testing. References [1] P. Ammann, P. Black, and W. Majurski, “Using Model Checking to Generate Tests from Specifications,” in Pro- ceedings of the 2nd IEEE International Conference on For- mal Engineering Methods, pp. 46-54, 1998. [2] T. Ball and S.K. Rajamani, “Bebop: a Symolic Model Checker for Boolean Programs,” SPIN Workshop ’00, Vol. 1885 of LNCS, pp. 113-130, Springer-Verlag, 2000. [3] J. Callahan, F. Schneider, and S. Easterbrook, “Specification- based Testing Using Model Checking,” in Proceedings of 1996 SPINWorkshop, also Technical Report NASA-IVV-96- 022, West Virginia Univeristy, 1996. [4] E.M. Clarke, E.A. Emerson, and A.P. Sistla, “Automatic Ver- ification of Finite-State Concurrent Systems Using Temporal Logic Specifications,” ACM Transactions on Programming Languages and Systems, 8(2):244-263, Apr. 1986. [5] E.M. Clarke, O. Grumberg, K. McMillan, and X. Zhao, “Efficient Generation of Counterexamples and Witnesses in Symbolic Model Checking,” in Proceedings of the 32nd De- sign Automation Conference, pp. 427-432, 1995. [6] E.M. Clarke, S. Jha, Y. Lu, and H. Veith, “Tree-Like Coun- terexamples in Model Checking,” in Proceedings of the 17th Annual IEEE Symposium on Logic in Computer Science, pp. 19-29, 2002. [7] L.A. Clarke, A. Podgurski, D.J. Richardson, and S.J. Zeil, “A Formal Evaluation of Data Flow Path Selection Criteria,” IEEE Transactions on Software Engineering, 15(11):1318- 1332, Nov. 1989. [8] R. Cleaveland and B. Steffen, “A Linear-Time Model- Checking Algorithm for the Alternation-Free Modal Mu- Calculus,” Formal Methods in System Design, Vol. 2, pp. 121-147, 1993. [9] A. Engels, L. Feijs, and S. Mauw, “Test Generation for Intel- ligent Networks Using Model Checking,” TACAS ’97, Vol. 1217 of LNCS, pp. 384-398, Springer-Verlag, 1997. [10] J. Esparza and J. Knoop, “An Automata-Theoretical Ap- proach to Interprocedural Data-Flow Analysis,” FOSSACS ’99, Vol. 1578 of LNCS, pp. 14-30, Springer-Verlag, 1999. [11] P.G. Frankl and E.J. Weyuker, “An Applicable Family of Data Flow Testing Criteria,” IEEE Transactions on Software Engineering, 14(10):1483-1498, Oct. 1988. [12] A. Gargantini and C. Heitmeyer, “Using Model Checking to Generate Tests from Requirements Specifications,” in Pro- ceedings of ESEC/FSE ’99 pp. 146-162, 1999. [13] M.J. Harrold and M.L. Soffa, “Interprocedural Data Flow Testing,” in Proceedings of the 3rd Symposium on Software Testing, Analysis, and Verification, pp. 158-167, 1989. [14] M.J. Harrold and G. Rothermel, “Performing Data Flow Testing on Classes,” in Proceedings of the 2nd ACM SIG- SOFT Symposium on the Foundations of Software Engineer- ing, pp. 154-163, 1994. [15] R. Hojati, R.K. Brayon, and R.P. Kurshan, “BDD-based De- bugging of Designs Using Language Containment and Fair CTL,” CAV ’99, Vol. 697 of LNCS, pp. 41-58, Springer- Verlag, 1993. [16] G.J. Holzmann, “The Model Checker SPIN,” IEEE Transac- tions on Software Engineering, Vol. 23, No. 5, pp. 279-295, May 1997. [17] H.S. Hong, Y.G. Kim, S.D. Cha, D.H. Bae, and H. Ural, “A Test Sequence Selection Method for Statecharts,” Journal of Software Testing, Verification, and Reliability, 10(4):203- 227, Dec. 2000.
                 !" #   [18] H.S. Hong, I. Lee, O. Sokolsky, and S.D. Cha, “Automatic Test Generation from Statecharts Using Model Checking,” in Proceedings of the First Workshop on Formal Approaches to Testing of Software, pp. 15-30, 2001. [19] H.S. Hong, I. Lee, O. Sokolsky, and H. Ural, “A Tempo- ral Logic Based Theory of Test Coverage and Generation,” TACAS ’02, Vol. 2280 of LNCS, pp. 327-341, Springer- Verlag, 2002. [20] T. Jeron and P. Morel, “Test Generation Derived From Model Checking,” CAV ’99, Vol. 1633 of LNCS, pp. 108-121, Springer-Verlag, 1999. [21] D. Kozen, “Results on the Propositional Mu-Calculus,” The- oretical Computer Science, 27:333-354, 1983. [22] J.W. Laski and B. Korel, “A Data Flow Oriented Program Testing Strategy,” IEEE Transactions on Software Engineer- ing, 9(5):347-354, May 1983. [23] K.L. McMillan, Symbolic Model Checking− an Approach to the State Explosion Problem, Kluwer Academic Publishers, 1993. [24] S.C. Ntafos, “On Required Element Testing,” IEEE Transac- tions on Software Engineering, 10(11):795-803, Nov. 1984. [25] S.C. Ntafos, “A Comparison of Some Structural Testing Strategies,” IEEE Transactions on Software Engineering, 14(6):868-874, June 1988. [26] S. Rapps and E.J. Weyuker, “Selecting Software Test Data Using Data Flow Information,” IEEE Transactions on Soft- ware Engineering, 11(4):367-375, Apr. 1985. [27] D.A. Schmidt and B. Steffen, “Data-flow Analysis as Model Checking of Abstract Interpretations,” SAS ’98, Vol. 1503 of LNCS, pp. 351-380, Springr-Verlag, 1998. [28] B. Steffen, “Generating Data-Flow Analysis Algorithms for Modal Specifications,” Science of Computer Programming, 21:115-139, 1993. [29] H. Ural and B. Yang, “A Test Sequence Generation Method for Protocol Testing,” IEEE Transactions on Communica- tions, 39(4):514-523, Apr. 1991. [30] H. Ural, “IO-df-chains criterion,” ISO Working Group on Formal Methods on Conformance Testing, Draft Interna- tional Standard, Sept. 1993. [31] H. Ural, K. Saleh, and A. Williams, “Test Generation Based on Control and Data Dependencies within System Specifica- tions in SDL,” Computer Communications, 23(7):609-627, Mar. 2000. [32] R. de Vries and J. Tretmans, “On-the-Fly Conformance Test- ing Using SPIN,” International Journal on Software Tools for Technology Transfer, 2(4):382-393, 2000.
                 !" #