UMass Amherst Lecture Notes: Parsing and Grammar, Study notes of Computer Science

Download UMass Amherst Lecture Notes: Parsing and Grammar and more Study notes Computer Science in PDF only on Docsity! Andrew McCallum, UMass Amherst Chart Parsing Lecture #6 Introduction to Natural Language Processing CMPSCI 585, Fall 2007 University of Massachusetts Amherst Andrew McCallum (agglomeration of slides from Jason Eisner) Andrew McCallum, UMass Amherst Today’s Main Points • Hand back In-class Exercise #2 • Apologies: HW #1 not completed by grader. I’m giving you an extra day; now due Friday. • Motivations and applications of Parsing. • Dynamic Programming for Parsing: CYK – Some hands-on practice • Discuss Programming Assignment #3 “Implement CYK and build a grammar” Andrew McCallum, UMass Amherst The parsing problem P A R S E R Grammar s c o r e r correct test trees test sentences accuracy Andrew McCallum, UMass Amherst Applications of parsing (1/2) • Machine translation (Alshawi 1996, Wu 1997, ...) English Chinese tree operations • Speech synthesis from parses (Prevost 1996) The government plans to raise income tax. The government plans to raise income tax the imagination. • Speech recognition using parsing (Chelba et al 1998) Put the file in the folder. Put the file and the folder. Andrew McCallum, UMass Amherst Applications of parsing (2/2) • Grammar checking (Microsoft) • Indexing for information retrieval (Woods 1997) ... washing a car with a hose ... vehicle maintenance • Information extraction (Hobbs 1996) (Miller et al 2000) NY Times archive Database pattern Andrew McCallum, UMass Amherst Dynamic Programming for Parsing • Given CFG in Chomsky Normal Form, and an input string, we want to search for valid parse trees. • What are the intermediate sub-problems? • What would the dynamic programming table look like? Andrew McCallum, UMass Amherst CKY algorithm, recognizer version  Input: string of n words  Output: yes/no (since it’s only a recognizer)  Data structure: n x n table  rows labeled 0 to n-1  columns labeled 1 to n  cell [i,j] lists possible constituents spanning words between i and j Andrew McCallum, UMass Amherst CKY algorithm, recognizer version  for i := 1 to n  Add to [i-1,i] all (part-of-speech) categories for the ith word  for width := 2 to n  for start := 0 to n-width  Define end := start + width  for mid := start+1 to end-1  for every constituent X in [start,mid]  for every constituent Y in [mid,end]  for all ways of combining X and Y (if any)  Add the resulting constituent to [start,end] if it’s not already there. N 84 Det 13 P 2 V 52 NP 4 VP 41 NP 10NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 Det 13 P 2 V 52 NP 4 VP 41 NP 10 S 8 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 Det 13 P 2 V 52 NP 4 VP 41 NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 _P 2 V 52 _NP 4 VP 41 NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12_P 2 V 52 __NP 4 VP 41 _NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 __NP 4 VP 41 _NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 __NP 4 VP 41 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 NP 24 S 22 S 27 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 NP 24 S 22 S 27 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 ___NP 4 VP 41 NP 24 S 22 S 27 NP 24 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 NP 24 S 22 S 27 NP 24 S 27 S 22 S 27 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 NP 24 S 22 S 27 NP 24 S 27 S 22 S 27 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP SFollow backpointers … N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 NP 24 S 22 S 27 NP 24 S 27 S 22 S 27 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP S NP VP N 84 NP 10Det 13 PP 12 VP 16 _P 2 V 52 NP 18 S 21 VP 18 __NP 4 VP 41 NP 24 S 22 S 27 NP 24 S 27 S 22 S 27 __NP 10 S 8 S 13 NP 3 Vst 3 0 time 1 flies 2 like 3 an 4 arrow 5 1 S → NP VP 6 S → Vst NP 2 S → S PP 1 VP → V NP 2 VP → VP PP 1 NP → Det N 2 NP → NP PP 3 NP → NP NP 0 PP → P NP S NP VP VP PP P NP Det N Andrew McCallum, UMass Amherst CMPSCI 585 In-class Exercise #3 Name: __________________ Student ID: ____________ Fill in the CYK dynamic programming table to parse the sentence below. In the bottom right corner, draw the two parse trees. she eats fish with chop- sticks 0 1 2 3 4 5 0 1 2 3 4 S → NP VP NP → NP PP VP → V NP VP → VP PP PP → P NPNP NP → she NP → fish NP → fork NP → chopsticks V → eats V → fish P → with Andrew McCallum, UMass Amherst CMPSCI 591N In-class Exercise #3 Name: __________________ Student ID: ____________ Fill in the CYK dynamic programming table to parse the sentence below. In the bottom right corner, draw the two parse trees. she eats fish with chop- sticks 0 1 2 3 4 5 0 1 2 3 4 S → NP VP NP → NP PP VP → V NP VP → VP PP PP → P NPNP NP → she NP → fish NP → fork NP → chopsticks V → eats V → fish P → with V NP V P NP VP V NP PP P NP S NP,VP NP NP PP VP V NP VP VP PP S NP VP S NP VP Andrew McCallum, UMass Amherst Earley Parser (1970) • Nice combination of – dynamic programming – incremental interpretation – avoids infinite loops – no restrictions on the form of the context-free grammar. A → B C the D of causes no problems – O(n3) worst case, but faster for many grammars – Uses left context and optionally right context to constrain search. Andrew McCallum, UMass Amherst Overview of the Algorithm • Finds constituents and partial constituents in input – A → B C . D E is partial: only the first half of the A A B C D E A → B C . D E D+ = A B C D E A → B C D . E Andrew McCallum, UMass Amherst Overview of the Algorithm • Proceeds incrementally left-to-right – Before it reads word 5, it has already built all hypotheses that are consistent with first 4 words – Reads word 5 & attaches it to immediately preceding hypotheses. Might yield new constituents that are then attached to hypotheses immediately preceding them … – E.g., attaching D to A → B C . D E gives A → B C D . E – Attaching E to that gives A → B C D E . – Now we have a complete A that we can attach to hypotheses immediately preceding the A, etc. Andrew McCallum, UMass Amherst Earley’s Algorithm, recognizer version • Add ROOT → . S to column 0. • For each j from 0 to n: – For each dotted rule in column j, (including those we add as we go!) look at what’s after the dot: • If it’s a word w, SCAN: – If w matches the input word between j and j+1, advance the dot and add the resulting rule to column j+1 • If it’s a non-terminal X, PREDICT: – Add all rules for X to the bottom of column j, wth the dot at the start: e.g. X → . Y Z • If there’s nothing after the dot, ATTACH: – We’ve finished some constituent, A, that started in column I<j. So for each rule in column j that has A after the dot: Advance the dot and add the result to the bottom of column j. • Output “yes” just if last column has ROOT → S . • NOTE: Don’t add an entry to a column if it’s already there! Andrew McCallum, UMass Amherst Summary of the Algorithm • Process all hypotheses one at a time in order. (Current hypothesis is shown in blue.) • This may add to the end of the to-do list, or try to add again. new hypotheses old hypotheses • Process a hypothesis according to what follows the dot: • If a word, scan input and see if it matches • If a nonterminal, predict ways to match it • (we’ll predict blindly, but could reduce # of predictions by looking ahead k symbols in the input and only making predictions that are compatible with this limited right context) • If nothing, then we have a complete constituent, so attach it to all its customers Andrew McCallum, UMass Amherst A Grammar S → NP VP NP → Papa NP→ Det N N → caviar NP→ NP PP N → spoon VP→ V NP V → ate VP→ VP PP P → with PP→ P NP Det → the Det → a An Input Sentence Papa ate the caviar with a spoon. 0 NP . Papa 0 NP . NP PP 0 NP . Det N 0 S . NP VP 0 ROOT . S 0 predict the kind of NP we are looking for (actually we’ll look for 3 kinds: any of the 3 will do) 0 Det . a 0 Det . the 0 NP . Papa 0 NP . NP PP 0 NP . Det N 0 S . NP VP 0 ROOT . S 0 predict the kind of Det we are looking for (2 kinds) 0 Det . a 0 Det . the 0 NP . Papa 0 NP . NP PP 0 NP . Det N 0 S . NP VP 0 ROOT . S 0 predict the kind of NP we’re looking for but we were already looking for these so don’t add duplicate goals! Note that this happened when we were processing a left-recursive rule. 0 Det . a 0 Det . the 0 NP . Papa 0 NP . NP PP 0 NP . Det N 0 S . NP VP 0 NP Papa .0 ROOT . S 0 Papa 1 scan: failure 0 Det . a 0 Det . the 0 NP . Papa 0 NP . NP PP 0 NP NP . PP0 NP . Det N 0 S NP . VP0 S . NP VP 0 NP Papa .0 ROOT . S 0 Papa 1 attach the newly created NP (which starts at 0) to its customers (incomplete constituents that end at 0 and have NP after the dot) 0 Det . a 0 Det . the 1 VP . VP PP0 NP . Papa 1 VP . V NP0 NP . NP PP 0 NP NP . PP0 NP . Det N 0 S NP . VP0 S . NP VP 0 NP Papa .0 ROOT . S 0 Papa 1 predict 1 V . ate0 Det . a 1 PP . P NP0 Det . the 1 VP . VP PP0 NP . Papa 1 VP . V NP0 NP . NP PP 0 NP NP . PP0 NP . Det N 0 S NP . VP0 S . NP VP 0 NP Papa .0 ROOT . S 0 Papa 1 predict 1 P . with 1 V . ate0 Det . a 1 PP . P NP0 Det . the 1 VP . VP PP0 NP . Papa 1 VP . V NP0 NP . NP PP 0 NP NP . PP0 NP . Det N 0 S NP . VP0 S . NP VP 0 NP Papa .0 ROOT . S 0 Papa 1 predict 1 P . with 1 V . ate0 Det . a 1 PP . P NP0 Det . the 1 VP . VP PP0 NP . Papa 1 VP . V NP0 NP . NP PP 0 NP NP . PP0 NP . Det N 0 S NP . VP0 S . NP VP 1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 scan: success! 1 P . with 1 V . ate0 Det . a 1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP . NP PP1 VP . V NP0 NP . NP PP 2 NP . Det N0 NP NP . PP0 NP . Det N 1 VP V . NP0 S NP . VP0 S . NP VP 1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 predict 1 P . with 2 Det . a1 V . ate0 Det . a 2 Det . the1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP . NP PP1 VP . V NP0 NP . NP PP 2 NP . Det N0 NP NP . PP0 NP . Det N 1 VP V . NP0 S NP . VP0 S . NP VP 1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 predict (these next few steps should look familiar) 1 P . with 2 Det . a1 V . ate0 Det . a 2 Det . the1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP . NP PP1 VP . V NP0 NP . NP PP 2 NP . Det N0 NP NP . PP0 NP . Det N 1 VP V . NP0 S NP . VP0 S . NP VP 1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 predict 1 P . with 2 Det . a1 V . ate0 Det . a 2 Det . the1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP . NP PP1 VP . V NP0 NP . NP PP 2 NP . Det N0 NP NP . PP0 NP . Det N 1 VP V . NP0 S NP . VP0 S . NP VP 2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 1 P . with 2 Det . a1 V . ate0 Det . a 2 Det . the1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP . NP PP1 VP . V NP0 NP . NP PP 2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 1 P . with 2 Det . a1 V . ate0 Det . a 2 Det . the1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 1 P . with 2 Det . a1 V . ate0 Det . a 2 Det . the1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 attach 1 P . with 2 Det . a1 V . ate0 Det . a 2 Det . the1 PP . P NP0 Det . the 2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 attach (again!) 1 P . with 2 Det . a1 V . ate0 Det . a 1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 attach (again!) 0 ROOT S .1 P . with 4 PP . P NP2 Det . a1 V . ate0 Det . a 1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 4 P . with 0 ROOT S .1 P . with 4 PP . P NP2 Det . a1 V . ate0 Det . a 1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 4 P . with 0 ROOT S .1 P . with 4 PP . P NP2 Det . a1 V . ate0 Det . a 1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 4 P . with 0 ROOT S .1 P . with 4 PP . P NP2 Det . a1 V . ate0 Det . a 1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 5 NP . Papa0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 5 NP . NP PP2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 5 NP . Det N1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 4 PP P . NP2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 4 P with .3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 with 5 4 P . with 0 ROOT S .1 P . with 5 Det . a4 PP . P NP2 Det . a1 V . ate0 Det . a 5 Det . the1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 5 NP . Papa0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 5 NP . NP PP2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 5 NP . Det N1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 4 PP P . NP2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 4 P with .3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 with 5 4 P . with 0 ROOT S .1 P . with 5 Det . a4 PP . P NP2 Det . a1 V . ate0 Det . a 5 Det . the1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 5 NP . Papa0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 5 NP . NP PP2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 5 NP . Det N1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 4 PP P . NP2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 4 P with .3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 with 5 4 P . with 0 ROOT S .1 P . with 5 Det . a4 PP . P NP2 Det . a1 V . ate0 Det . a 5 Det . the1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 5 NP . Papa0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 5 NP . NP PP2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 5 NP . Det N1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 4 PP P . NP2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 5 Det a .4 P with .3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 with 5 a 6 4 P . with 0 ROOT S .1 P . with 5 Det . a4 PP . P NP2 Det . a1 V . ate0 Det . a 5 Det . the1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 5 NP . Papa0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 5 NP . NP PP2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 5 NP . Det N1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 5 NP Det . N4 PP P . NP2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 5 Det a .4 P with .3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 with 5 a 6 4 P . with 0 ROOT S .1 P . with 5 Det . a4 PP . P NP2 Det . a1 V . ate0 Det . a 5 Det . the1 VP VP . PP2 Det . the1 PP . P NP0 Det . the 5 NP . Papa0 S NP VP .2 NP . Papa1 VP . VP PP0 NP . Papa 6 N . spoon5 NP . NP PP2 NP NP . PP3 N . spoon2 NP . NP PP1 VP . V NP0 NP . NP PP 6 N . caviar5 NP . Det N1 VP V NP .3 N . caviar2 NP . Det N0 NP NP . PP0 NP . Det N 5 NP Det . N4 PP P . NP2 NP Det N .2 NP Det . N1 VP V . NP0 S NP . VP0 S . NP VP 5 Det a .4 P with .3 N caviar .2 Det the .1 V ate .0 NP Papa .0 ROOT . S 0 Papa 1 ate 2 the 3 caviar 4 with 5 a 6