Model Checking | EE 244: Fundamental Algorithms for System ..., Exams of Algorithms and Programming

Download Model Checking | EE 244: Fundamental Algorithms for System ... and more Exams Algorithms and Programming in PDF only on Docsity! EE 244: Fundamental Algorithms for System Modeling, Analysis, and Optimization Fall 2016 Model Checking Stavros Tripakis University of California, Berkeley Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 1 / 68 Recall: the model-checking problems for LTL and CTL Given: the implementation: a transition system (Kripke structure) M = (AP, S, S0, L,R) the specification: a temporal logic (LTL or CTL) formula φ check where M satisfies φ: M ? |= φ If φ is LTL: every execution trace of M must satisfy φ. If φ is CTL: every initial state of M must satisfy φ. For finite-state M , the question can be answered fully automatically! Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 2 / 68 ACM Turing Award for Model-Checking Clarke, Emerson, and Sifakis won the ACM Turing Award in 2007, for their role in developing Model-Checking into a highly effective verification technology that is widely adopted in the hardware and software industries. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 3 / 68 Simplest model-checking problem: checking invariants Suppose φ is of the form Gψ or AGψ where ψ is a propositional formula (boolean expression on atomic propositions). E.g., G(p ∨ q), G(p→ q), · · · Then ψ is invariant: it must hold at all reachable states. Examples: “Whenever train is at intersection the gate must be lowered” “If the autopilot is off then the pilot must not believe it is on” Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 4 / 68 Reachability analysis: summary Generate all reachable states ... ... while at the same time checking that each of them is “OK”, i.e., I it is not a deadlock state I it does not violate an invariant I ... Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 9 / 68 Reachability Algorithms Enumerative (also called “explicit state”). I These are basically search algorithms on directed graphs. Symbolic I Bounded model-checking using SAT/SMT solvers. I Symbolic reachability. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 10 / 68 An Enumerative Algorithm: Depth-First Search Assume given: Kripke structure (P, S, S0, L,R). main: 1: V := ∅; /* V : set of visited states */ 2: for all s ∈ S0 do 3: DFS(s); 4: end for DFS(s): 1: check s; /* is s a deadlock? is given p ∈ L(s)? ... */ 2: V := V ∪ {s}; 3: for all s′ such that (s, s′) ∈ R do 4: if s′ 6∈ V then 5: DFS(s′); /* recursive call */ 6: end if 7: end for Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 11 / 68 An Enumerative Algorithm: Depth-First Search Let’s simulate the algorithm on this graph. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 12 / 68 An Enumerative Algorithm: Depth-First Search Quiz: Does the algorithm terminate? Yes, if state space is finite. Does it visit all reachable states? Yes: if s is reachable, then either s ∈ S0, or s is the immediate successor of some s′, which is itself reachable. In the first case, s is inserted into V because of the main loop. In the second case, assuming (by induction) that s′ is inserted to V , s will also be inserted to V by loop in lines 3-6. Does it visit any unreachable states? No: following the “inverse” of the argument above, if s is inserted into V , either this is done because of the main loop, or because of the loop in lines 3-6. In the first case, s must be in S0, so it’s an initial state, so it’s reachable. In the second case, s must be successor of some s′, which by induction must be itself in V , therefore reachable. What is the complexity of the algorithm? O(n+m) where n is number of nodes/states and m is number of edges/transitions in the graph. Every node and edge are visited at most once. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 13 / 68 Other enumerative algorithms Every search algorithm on finite graphs can be used for reachability analysis: DFS: depth-first search BFS: breadth-first search Best-first search: I every state is assigned a “value” (using some heuristic value function, e.g., how “close” we are likely to be to the goal – in our case a “bad” state) and then next state to explore is the one with the highest value. A*: classic search technique in artificial intelligence. ... Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 14 / 68 SYMBOLIC METHODS Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 19 / 68 Symbolic Methods: Why? Motivation: attack the state explosion problem. A seminal paper: Symbolic model checking: 1020 states and beyond. [Burch et al., 1990]. 1020 is less than 267, so still not quite enough for modern circuits. Nevertheless: a great leap forward at that time. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 20 / 68 Symbolic Representation of State Spaces Key idea: Instead of reasoning about individual states, reason about sets of states. How do we represent a set of states? Symbolic representation: Set = predicate. Set of states = predicate on state variables. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 21 / 68 Symbolic Representation of Sets of States Examples: 1 Assume 3 state variables, p, q, r, of type boolean. S1 : p ∨ q = {pqr, pqr, pqr, pqr, pqr, pqr} 2 Assume 3 state variables, x, i, b, of types real, integer, boolean. S2 : x > 0 ∧ (b→ i ≥ 0) How many states are in S2? Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 22 / 68 Symbolic Representation of Transition Relations Key idea: Use a predicate on two copies of the state variables: unprimed (current state) + primed (next state). If ~x is the vector of state variables, then the transition relation R is a predicate on ~x and ~x′: R(~x, ~x′) e.g., for three state variables, x, i, b: R(x, i, b, x′, i′, b′) Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 23 / 68 Symbolic Representation of Transition Relations Examples: 1 Assume one state variable, p, of type boolean. R1 : (p→ ¬p′) ∧ (¬p→ p′) Which transition relation does this represent? Is it a relation or a function (deterministic)? 2 Assume one state variable, n, of type integer. R2 : n′ = n+ 1 ∨ n′ = n Which transition relation does this represent? Is it a relation or a function (deterministic)? Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 24 / 68 Recall: Symbolic Representation of Kripke Structures (P, Init ,Trans) where P = {x1, x2, ..., xn}: set of boolean state variables, also taken to be the atomic propositions. Predicate Init(~x) on vector ~x = (x1, ..., xn) represents the set S0 of initial states. Predicate Trans(~x, ~x′) represents the transition relation R. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 29 / 68 Recall: Symbolic Representation Set of states = predicate φ(~x) on vector of state variables ~x. E.g.: I Init(x, y, z) : x ∧ ¬y I Bad(x1, x2) : x1 = crit ∧ x2 = crit Transition relation = predicate Trans(~x, ~x′) on state variables and next-state variables. E.g.: I Trans(x, y, x′, y′) : x′ = x+ 1 ∧ (y′ = 0 ∨ y′ = 1) How do we perform set-theoretic operations with predicates? I Union of two sets represented by φ1 and φ2: φ1 ∨ φ2. I Intersection of two sets represented by φ1 and φ2: φ1 ∧ φ2. I Complement of a set represented by φ: ¬φ. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 30 / 68 Symbolic Reachability Analysis Main idea: Start with set of initial states S0. Compute S1 := S0 ∪ {all 1-step successors of S0}. Compute S2 := S1 ∪ {all 1-step successors of S1}. ... Until Sk+1 = Sk. Sk contains all reachable states. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 31 / 68 Computing Successors Symbolically Given a set of states represented as a predicate φ(~x). We want to compute a new predicate φ′, representing the set of all 1-step successors of states in φ(~x). Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 32 / 68 Predicate Transformer Successors can be computed by a predicate transformer : succ ( φ(~x) ) := ( ∃~x : φ(~x) ∧ Trans(~x, ~x′) ) [~x′ ; ~x] I ∃~x : φ(~x) ∧ Trans(~x, ~x′): successors of states in φ I [~x′ ; ~x]: renames variables so that resulting predicate is over current state variables Example: φ = 0 ≤ x ≤ 5 Trans = x ≤ x′ ≤ x+ 1 succ(φ) = (∃x : 0 ≤ x ≤ 5 ∧ x ≤ x′ ≤ x+ 1)[x′ ; x] = (∃x : 0 ≤ x ≤ 5 ∧ 0 ≤ x′ ≤ 5 + 1)[x′ ; x] = (0 ≤ x′ ≤ 6)[x′ ; x] = 0 ≤ x ≤ 6 Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 33 / 68 Predicate Transformer succ ( φ(~x) ) := ( ∃~x : φ(~x) ∧ Trans(~x, ~x′) ) [~x′ ; ~x] How to do quantifier elimination automatically? In the case of propositional logic, quantifier elimination is simple. Suppose x is a boolean variable: ∃x : φ ⇔ φ[x ; 0] ∨ φ[x ; 1] Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 34 / 68 Symbolic Model-Checking: Implementation For finite-state systems, boolean variables can be used to encode state. All predicates then become boolean expressions. Efficient data structures for boolean expressions: I BDDs (Binary Decision Diagrams) [Bryant, 1992] (paper available in bcourses - follow link from lectures web page) Efficient algorithms for implementing logical operations (conjunction, disjunction, satisfiability check, ...) on BDDs. Note: logical operations correspond to set-theoretic operations: I Conjunction: intersection I Disjunction: union I Satisfiability check: emptiness check I ... Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 39 / 68 Example: BDD Can you guess which boolean expression this BDD represents? x4 ( x3(x2 + x2x1) + x3(x2 x1 + x2) ) + x4x2x1 Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 40 / 68 BDDs Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 41 / 68 Binary decision trees Binary decision tree: A tree representing all possible variable assignments, and corresponding truth values of a boolean expression. For n variables, the tree has 1 + 2 + 22 + · · ·+ 2n = 2n+1 − 1 nodes (including the leaves). Let’s draw the binary decision tree for (z1 ∧ z3) ∨ (z2 ∧ z3) (assuming the order of variables z1, z2, z3). Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 42 / 68 From binary decision trees to BDDs Main idea: make the representation compact (i.e., smaller) by eliminating redundant nodes. If two subtrees (including leaves) T1 and T2 are identical then keep only T1. All incoming links to T2 are redirected to T1. If both the true-branch and the false-branch of a node v lead to the same node v′, then node v is redundant: v can be removed, with its incoming links being redirected to v′. The result is a reduced ordered binary decision diagram (ROBDD). It is a DAG: directed acyclic graph. We often use BDD to mean ROBDD. Let’s try this on the following formulas: a+ b, and (z1 ∧ z3) ∨ (z2 ∧ z3) Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 43 / 68 From binary decision trees to BDDs 394 Computation Tree Logic z1 z2 z2 z3 z3 z3 z3 1 0 1 0 1 0 0 0 z1 z2 z2 z3 1 0 z1 z2 z3 1 0 Figure 6.22: Binary decision diagrams for f = (z1 ∧ z3) ∨ (z2 ∧ z3). 394 Computation Tree Logic z1 z2 z2 z3 z3 z3 z3 1 0 1 0 1 0 0 0 z1 z2 z2 z3 1 0 z1 z2 z3 1 0 Figure 6.22: Binary decision diagrams for f = (z1 ∧ z3) ∨ (z2 ∧ z3). 394 Computation Tree Logic z1 z2 z2 z3 z3 z3 z3 1 0 1 0 1 0 0 0 z1 z2 z2 z3 1 0 z1 z2 z3 1 0 Figure 6.22: Binary decision diagrams for f = (z1 ∧ z3) ∨ (z2 ∧ z3). Figure taken from [Baier and Katoen, 2008]. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 44 / 68 1: 2: 3: Shannon expansion Let f be a boolean expression and x be a boolean variable. Recall: f [x ; 0] denotes the new formula f ′ obtained by replacing any occurrence of x in f by 0. Similarly for f [x ; 1]. f [x ; 1] and f [x ; 0] are called the (positive and negative) cofactors of f , and are denoted fx and fx. Then f ⇔ x · fx + x · fx︸ ︷︷ ︸ this is called the Shannon expansion of f Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 49 / 68 Shannon expansion f ⇔ x · fx + x · fx This is the essence of binary decision trees and BDDs: if f is the root, then fx is the sub-tree rooted at the 0-branch (“false”-branch) child of f fx is the sub-tree rooted at the 1-branch (“true”-branch) child of f Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 50 / 68 Recursive application of boolean operations based on Shannon expansion Suppose is some boolean operation (e.g., conjunction or disjunction). Let f and g be two boolean expressions, and x be a boolean variable (usually f and g refer to x, but they don’t have to). Then f g ⇔ x · (fx gx) + x · (fx gx) For instance, if is conjunction: f · g ⇔ x · fx · gx + x · fx · gx This leads to the apply function. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 51 / 68 The apply function Takes as input: I A boolean operation (e.g., conjunction or disjunction). I Two BDDs Bf and Bg (with the same variable ordering) representing two boolean functions f and g. Computes as output: I A BDD B representing f g: B = apply( , Bf , Bg) such that B ⇔ Bf g Operates recursively based on Shannon expansion. Resulting BDD may not be reduced, so needs to be generally reduced afterwards. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 52 / 68 The apply function We are computing apply( , Bf , Bg). Let vf and vg be the root nodes of Bf and Bg respectively. There are the following cases to consider: 1 Both vf and vg are leaves (i.e., 0 or 1). Then, apply returns the leaf BDD with truth value vf vg. 2 Both vf and vg are internal x-nodes, i.e., corresponding to variable x. Then, let Bx f , B x g be the positive sub-BDDs (i.e., positive cofactors, i.e., BDDs rooted at the true-branch children) of vf and vg, respectively; and similarly with Bx f , B x g . Then: 1 Recursively compute BDD Bx := apply( , Bx f , B x g ). 2 Recursively compute BDD Bx := apply( , Bx f , B x g ). 3 Create and return a new BDD with root x and Bx as positive sub-BDD and Bx as negative sub-BDD. The justification for this comes directly from f g ⇔ x · (fx gx) + x · (fx gx) Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 53 / 68 The apply function (continued) 3 vf is an internal x-node, but vg is either a leaf (0 or 1) or an internal y-node, with y > x, i.e., variable y is after x in the ordering (y is lower in the tree). Then we know, since Bf and Bg must follow the same variable ordering, that Bg is independent from x at this point in the tree. So we proceed as follows: 1 Recursively compute BDD Bx := apply( , Bx f , Bg). 2 Recursively compute BDD Bx := apply( , Bx f , Bg). 3 Create and return a new BDD with root x and Bx as positive sub-BDD and Bx as negative sub-BDD. Do you see room for optimization here? E.g., when is + and vg is 0 or 1. If 0, return vf . If 1, return 1. 4 Symmetric to case 3 above, but with vg being higher in the tree than vf instead of lower. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 54 / 68 Bounded reachability Question: Can a “bad” state be reached in up to n steps (transitions)? i.e., given a transition system (P, S, S0, L,R) and a set of states Bad ⊆ S, does there exist a path s0 −→ s1 −→ · · · −→ sk in the transition system such that s0 ∈ S0 and sk ∈ Bad , and k ≤ n. Key idea: Reduce the above question to a SAT (satisfiability) problem. SAT problem NP-complete for propositional logic. In practice, today’s SAT solvers can handle formulas with thousands of variables (or more!): see [Malik and Zhang, 2009]. BMC (bounded model-checking) has emerged thanks to the advances in SAT solver technology. Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 59 / 68 Bounded reachability Suppose I have predicates Init(~x), Trans(~x, ~x′), and Bad(~x). How to use them for bounded reachability? Bad state reachable in 0 steps iff SAT ( Init(~x) ∧ Bad(~x) ) Bad state reachable in 1 step iff SAT ( Init(~x0) ∧ Trans(~x0, ~x1) ∧ Bad(~x1) ) ... Bad state reachable in n steps iff SAT ( Init(~x0)∧Trans(~x0, ~x1)∧· · ·∧Trans(~xn−1, ~xn)∧Bad(~xn) ) Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 60 / 68 Bounded reachability algorithm – outer loop 1: for all k = 0, 1, ..., n do 2: φ := Init(~x0)∧Trans(~x0, ~x1)∧· · ·∧Trans(~xk−1, ~xk)∧Bad(~xk); 3: if SAT(φ) then 4: print “Bad state reachable in k steps”; 5: output solution as counter-example; 6: end if 7: end for 8: print “Bad state unreachable up to n steps”; Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 61 / 68 Bounded reachability: soundness and completeness 1: for all k = 0, 1, ..., n do 2: φ := Init(~x0) ∧ Trans(~x0, ~x1) ∧ · · · ∧ Trans(~xk−1, ~xk) ∧ Bad(~xk); 3: if SAT(φ) then 4: print “Bad state reachable in k steps”; 5: output solution as counter-example; 6: end if 7: end for 8: print “Bad state unreachable up to n steps”; BMC algorithm is sound in the following sense: if algorithm reports “reachable” then indeed a bad state is reachable if algorithm reports “unreachable up to n steps” then there is no path of length ≤ n that reaches a bad state. Can we make BMC complete? It should report unreachable iff there are no reachable bad states (w.r.t. any bound). Is this even possible in general? For finite-state systems? Yes! Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 62 / 68 Complete BMC: “brute-force” threshold 1: for all k = 0, 1, ..., n do 2: φ := Init(~x0) ∧ Trans(~x0, ~x1) ∧ · · · ∧ Trans(~xk−1, ~xk) ∧ Bad(~xk); 3: if SAT(φ) then 4: print “Bad state reachable in k steps”; 5: output solution as counter-example; 6: end if 7: end for 8: print “Bad state unreachable up to n steps”; A finite-state transition system is essentially a finite graph. How can we turn BMC into a complete method for finite-state systems? If we know |S| (the number of all possible states) then we can set n := |S|. Because no acyclic path can have length greater than |S|, and we only care about acyclic paths. But: with 100 boolean variables, |S| = 2100, so this isn’t practical ... (formulas become too big). Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 63 / 68 Complete BMC: a better threshold Reachability diameter: number of steps that it takes to reach any reachable state. d := min{i | ∀s ∈ Reach : ∃ path s0, s1, ..., sj : j ≤ i∧s0 ∈ S0∧sj = s} where Reach is the set of reachable states. d is generally a much better threshold than |S|. Why? d ≤ |Reach| ≤ |S|. Problem: we don’t know |Reach|, therefore how to compute d? Stavros Tripakis (UC Berkeley) EE 244, Fall 2016 Model Checking 64 / 68