Introduction to Artificial Intelligence, Summaries of Material Engineering

Download Introduction to Artificial Intelligence and more Summaries Material Engineering in PDF only on Docsity! ICS-271:Notes 5: 1 Set 5: Constraint Satisfaction Problems Chapter 6 R&N ICS 271 Fall 2016 Kalev Kask ICS-271:Notes 5: 2 Outline • The constraint network model – Variables, domains, constraints, constraint graph, solutions • Examples: – graph-coloring, 8-queen, cryptarithmetic, crossword puzzles, vision problems,scheduling, design • The search space and naive backtracking, • The constraint graph • Consistency enforcing algorithms – arc-consistency, AC-1,AC-3 • Backtracking strategies – Forward-checking, dynamic variable orderings • Special case: solving tree problems • Local search for CSPs Example: Map-Coloring Northern Territory | Western | Australia Queensland South ae _ Australia [ — | New South Wales on Victoria Variables WA, NT, Q, NSW, V,SA,7 RP Domains D; = {red, green, blue} Constraints: adjacent regions must have different colors e.g., WA # NT (if the language allows this), or (WA, NT) € {(red, green), (red, blue), (green, red), (green, blue) Sections 3.7 and 4.4, Chapter Sof AIMA2e ICS-271:Notes 5: 5  Example: Map-Coloring contd. eA ee | fe, TRH Solutions are assignments satisfying all constraints, e.g., {WA=red, NT = green, Q =red, NSW =green, V =red, SA= blue, T = green} ions 3.7 and 4.4, Chapter § of AIMA2e 4  Constraint graph Binary CSP: each constraint relates at most two variables Constraint graph: nodes are variables, arcs show constraints oT, 6 © General-purpose CSP algorithms use the graph structure to speed up search. E.g., Tasmania is an independent subproblem! Sections 3.7 and 4.4, Chapter § of AIMA2e ICS-271:Notes 5: 10 Hard vs Soft Constraints • Hard constraints : must be satisfied – Satisfaction problem • Soft constraints : capture preferences – Optimization problem ICS-271:Notes 5: 11 Sudoku Each row, column and major block must be all different “Well posed” if it has unique solution: 27 constraints ICS-271:Notes 5: 12 Sudoku Each row, column and major block must be all different “Well posed” if it has unique solution: 27 constraints 2 3 4 62 Constraint propagation •Variables: 81 slots •Domains = {1,2,3,4,5,6,7,8,9} •Constraints: •27 not-equal ICS-271:Notes 5: 16 A network of constraints • Variables – • Domains – of discrete values: • Binary constraints: – Rij which represent the list of allowed pairs of values, Rij is a subset of the Cartesian product: . • Constraint graph: – A node for each variable and an edge for each constraint • Solution: – An assignment of a value from its domain to each variable such that no constraint is violated. • A network of constraints represents the relation of all solutions. n,....,XX1 n,....,DD1 ji xDD },,),(|){( 1 jjiiijjin DxDxRxx,....,xxsol  1X 5X 4X 3X 2X ICS-271:Notes 5: 17 Example : The 4-queen problem Q Q Q Q Q Q Q Q Place 4 Queens on a chess board of 4x4 such that no two queens reside in the same row, column or diagonal. Standard CSP formulation of the problem: • Variables: each row is a variable. Q Q Q Q 1X 4X 3X 2X 1 2 3 4 • Domains: }.4,3,2,1{iD • Constraints: There are = 6 constraints involved: 4 2( ) )}2,4)(1,4)(1,3)(4,2)(4,1)(3,1{(12 R )}3,4)(1,4)(4,3)(2,3)(3,2)(1,2)(4,1)(2,1{(13 R )}3,4)(2,4)(4,3)(2,3)(1,3)(4,2)(3,2)(1,2)(3,1)(2,1{(14 R )}2,4)(1,4)(1,3)(4,2)(4,1)(3,1{(23 R )}3,4)(1,4)(4,3)(2,3)(3,2)(1,2)(4,1)(2,1{(24 R )}2,4)(1,4)(1,3)(4,2)(4,1)(3,1{(34 R • Constraint Graph : 1X 2X 4X 3X Class scheduling/Timetabling e Teachers, Subjects, Classrooms, Time-slots. e Constraints: - A teacher teaches a subset of subjects, - Subjects are taught at certain classrooms, - A teacher prefers teaching in the morning. e Task: Assign a teacher and a subject to each class at each time slot, s.t. teachers’ happiness is maximized. DCT, )= Aeabes Tj - teacher at class C; at time t; $j - subject taught at class C; at time t, Dowat™ * 9 DT ICS-271:Notes 5: 19  Backtracking search Variable assignments are commutative, i.e., [WA=redthen NT = green] sameas [NT = green then WA =red] Only need to consider assignments to a single variable at each node => b=d and there are d” leaves Depth-first search for CSPs with single-variable assignments is called backtracking search Backtracking search is the basic uninformed algorithm for CSPs Can solve n-queens for n = 25 7 and dt, Chapter 5 of AIMAZe 12 ICS-271:Notes 5: 23 The search space • Definition: given an ordering of the variables – a state: • is an assignment to a subset of variables that is consistent. – Operators: • add an assignment to the next variable that does not violate any constraint. How to choose next variable? – Goal state: • a consistent assignment to all the variables. n,....,XX1 Backtracking example Sections 3.7 and 4.4, Chapter S.0f AIMAZe 14 ICS-271:Notes 5: 27 ICS-271:Notes 5: 28 Backtracking • Complexity of extending a partial solution: – Complexity of consistent: O(e log t), t bounds #tuples, e bounds #constraints – Complexity of selectvalue: O(e k log t), k bounds domain size ICS-271:Notes 5: 30 The effect of variable ordering z divides x, y and t Backtracking Search for a Solution xl xt Coote green, site) a2 reberentel ) C red.bluc > hiweeecn 43 ~ 33 red,bhue ‘ ad % x vs ICS-271:Notes 5: 33 Backtracking Search for All Solutions xl xt Coote green, site) a2 reberentel ) C red.bluc > hiweeecn 43 ~ 33 red,bhue ‘ ad % x vs ICS-271:Notes 5: 34 ICS-271:Notes 5: 35 Summary so far • Constraint Satisfaction Problems – variables, domains, constraints – solution = assignment to all variables such that all constraints are satisfied – constraint graph • Search vs Inference: – conditioning vs local consistency. • Search – Backtracking search : • In the space of partial assignments; • add consistent var=value assignment at each search space node. • If not possible, backtrack – Effect on search space size on var/val order : • Variable order – most constrained variable first • Value order – least constraining value first ICS-271:Notes 5: 38 Arc-consistency 32,1, 32,1, 32,1, 1  X, Y, Z, T  3 X  Y Y = Z T  Z X  T X Y T Z 32,1,  =   ICS-271:Notes 5: 39 1  X, Y, Z, T  3 X  Y Y = Z T  Z X  T X Y T Z  =   1 3 2 3 • Incorporated into backtracking search • Constraint programming languages powerful approach for modeling and solving combinatorial optimization problems. Arc-consistency ICS-271:Notes 5: 40 Arc-consistency algorithm domain of X domain of Y Arc is arc-consistent if for any value of there exist a matching value of Algorithm Revise makes an arc consistent Begin 1. For each a in Di if there is no value b in Dj that is consistent with a then delete a from the Di. End. Revise is , k is the number of values in each domain. )( ji ,XX iX iX )( ji ,XX )O(k 2 ICS-271:Notes 5: 43 Sudoku Each row, column and major block must be all different “Well posed” if it has unique solution Path-consistency or 3-consistency 4-consistency and i-consistency in geeral ICS-271:Notes 5: 44 The Effect of Consistency Level • After arc(2)-consistency z=5 and l=5 are removed • After path(3)-consistency – R’_zx – R’_zy – R’_zl – R’_xy – R’_xl – R’_yl Tighter networks yield smaller search spaces ICS-271:Notes 5: 45 • Before search: (reducing the search space) – Arc-consistency, path-consistency, k-consistency – Variable ordering (fixed) • During search: – Look-ahead schemes: • Variable ordering (Choose the most constraining variable) • Value ordering (Choose the least restricting value) – Look-back schemes: • Backjumping • Constraint recording • Dependency-directed backtracking Improving Backtracking O(exp(n)) Most constrained variable Most constrained variable: choose the variable with the fewest legal values Be FF & Sections 3.7 and 4.4, Chapter S.0f AIMAZe 19  Most constraining variable Tie-breaker among most constrained variables Most constraining variable: choose the variable with the most constraints on remaining variables Boy yO Sections 3.7 and 4.4, Chapter S.0f AIMAZe 20  Least constraining value Given a variable, choose the least constraining value: the one that rules out the fewest values in the remaining variables Allows 1 value for SA ( = -€ -¢€ << s en Allows 0 values for SA o e o Combining these heuristics makes 1000 queens feasible Sections 3.7 and 4.4, Chapter Sof AIMAZe 21  Forward checking Idea: Keep track of remaining legal values for unassigned variables Terminate search when any variable has no legal values CoS WA NT Q NSW v SA T  Forward checking Idea: Keep track of remaining legal values for unassigned variables Terminate search when any variable has no legal values Coy #5 WA NT Q NSW v SA T  Constraint propagation Forward checking propagates information from assigned to unassigned vari- ables, but doesn’t provide early detection for all failures: Cote #S WA NT Q NSW Vv SA T BESS ee See Peele eee ST _ 1 e e_ _ a 1 a Le ajnnn NT and SA cannot both be blue! Constraint propagation repeatedly enforces constraints locally Sections 3.7 and 4.4, Chapter 5 of AIMAZe 26 ICS-271:Notes 5: 63 Forward-checking on Graph-coloring )( 2ekO )( )( 3 2 ekO ekOFW overhead: MAC overhead: Algorithm DVO (DVFC) procedure DVFC Input: A constraint network R = (X,D,C) Output: Either a solution, or notification that the network is inconsistent. BD— Dforl<isn (copy all domains} aol (initialize variable counter) s=minjejen|Di| (find future var with smallest dornain) Tip. — #, (rearrange variables so that x, follows 2;} whilel<ica instantiate 7; — SELECTVALUE-FORWARD-CHECKING if z; is null (no value was returned) reset each J’ set to its value before x; was last instantiated teil (backtrack) else ife<an @—aitl (step forward to z,) s=minjejen|Di| (find future var with smallest dornain) Tip. — #, (rearrange variables so that x, follows 2;} te-aitl (step forward to 7,} end while ifi=0 return “inconsistent” else return instantiated values of {2,...,tn} end procedure Figure 5.12: The DVFC algorithm. It uses the SELECTWALUE-FORWARD-CHECKING sub- procedure given in Fig. 5.8. ICS-271:Notes 5: 64 ICS-271:Notes 5: 65 Propositional Satisfiability • If Alex goes, then Becky goes: • If Chris goes, then Alex goes: • Query: Is it possible that Chris goes to the party but Becky does not? Example: party problem ) (or, BA BA  ) (or, ACA C   e?satisfiabl Is  C B, A,C B,A theorynalpropositio ICS-271:Notes 5: 68 Look-ahead for SAT: DPLL Only enclosed area will be explored with unit-propagation Backtracking look-ahead with Unit propagation= Generalized arc-consistency (Davis-Putnam, Logeman and Laveland, 1962) example: (┐A V B) ˄ (┐C V A) ˄ (A V B V D) ˄ (C) ICS-271:Notes 5: 70 Look-back: Backjumping / Learning • Backjumping: – In deadends, go back to the most recent culprit. • Learning: – constraint-recording, no-good recording. – good-recording ICS-271:Notes 5: 72 Backjumping • (X1=r,x2=b,x3=b,x4=b,x5=g,x6=r,x7={r,b}) • (r,b,b,b,g,r) conflict set of x7 – Conflict set : a partial assignment that cannot be extended to the next variable • (r,-,b,b,g,-) c.s. of x7 • (r,-,b,-,-,-,-) minimal conflict-set • Leaf deadend: (r,b,b,b,g,r) – No-good : a partial assignment that cannot be extended to a solution – No-good = internal deadend • Every conflict-set is a no-good – But not other way around (x1=r is a no-good) ? Tree-structured CSPs Theorem: if the constraint graph has no loops, the CSP can be solved in O(nd?) time Compare to general CSPs, where worst-case time is O(d") This property also applies to logical and probabilistic reasoning: an important example of the relation between syntactic restrictions and the complexity of reasoning. Sections 3.7 and 4.4, Chapter 5. of AIMAZe 34 ICS-271:Notes 5: 77 4. Arc-consistency in tree-structured CSPs makes search backtrack-free Nearly tree-structured CSPs Conditioning: instantiate a variable, prune its neighbors’ domains O—@ O—~@ a ae ~~ £ © © Cutset conditioning: instantiate (in all ways) a set of variables such that the remaining constraint graph is a tree Cutset sizec = runtime O(d*- (n—c)d?), very fast for small c Sections 3.7 and 4.4, Chapter Sof AIMAZe 46 ICS-271:Notes 5: 81 Local Search for CSPs • Local Search • Works with complete variable assignments (called a state) • Has a cost function associated with each state • E.g. min-conflicts – number of constraints violated • Considers only immediate neighborhood for next state • E.g. change the value of one variable • Operators • Greedy heuristic : pick a value for a variable that leads to greatest reduction in total cost – can get stuck in local optima • Random moves : occasionally make a random move • Sideways moves, re-starts, constraint re-weighting, … • Incomplete • Often fast • orders of magnitude faster than systematic e.g. DFS Example: 4-Queens States: 4 queens in 4 columns (44 = 256 states) Operators: move queen in column Goal test: no attacks Evaluation: h(n) = number of attacks Sections 3.7 and 4.4, Chapter 5. of AIMAZe 38 ICS-271:Notes 5: 84 GSAT – local search for SAT (Selman, Levesque and Mitchell, 1992) 1. For i=1 to MaxTries 2. Select a random assignment A 3. For j=1 to MaxFlips 4. if A satisfies all constraints, return A 5. else flip a variable to maximize the score 6. (number of satisfied constraints; if no variable 7. assignment increases the score, flip at random) 8. end 9. end Greatly improves hill-climbing by adding restarts and sideway moves Problem Hardness Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000) The same appears to be true for any randomly-generated CSP except in a narrow range of the ratio R number of constraints number of variables CPU} time cab critical ratio Sections 3.7 and 4.4, Chapter 5 of AIMA2e 39 ICS-271:Notes 5: 87  Summary CSPs are a special kind of problem: states defined by values of a fixed set of variables goal test defined by constraints on variable values Backtracking = depth-first search with one variable assigned per node Variable ordering and value selection heuristics help significantly Forward checking prevents assignments that guarantee later failure Constraint propagation (e.g., arc consistency) does additional work to constrain values and detect inconsistencies The CSP representation allows analysis of problem structure Tree-structured CSPs can be solved in linear time Iterative min-conflicts is usually effective in practice Sections 3.7 and 4.4, Chapter $.of AIMAZe 40 Game of Mastermind Sample Mastermind game: Gamel Red Green Yellow Hidden Combination Cc t Position: 1 Guess 2 | Red Blue Green Connect Coloue 1 Correct Position: 1 Guess 1 | Red Red Red Correct Colour: 0 Variables: P1,...Pn one variable for each peg Domains: R,G,B,Y Constraints: {P1,P2,P3: (Red, Green, Yellow ), (Red, Yellow, Blue), (Green, Red, Green), (Blue, Red, Green), (Yellow, Red, Green), (Blue, Blue, Red), (Yellow, Blue, Red)} ICS-271:Notes 5: 89