Recursive Parsing: Understanding Grammar Rules and Generating Parse Trees, Study notes of Computer Science

Download Recursive Parsing: Understanding Grammar Rules and Generating Parse Trees and more Study notes Computer Science in PDF only on Docsity! 1 Grammars & Parsing Lecture 5 CS2110 – Summer 2008 2 Application of Recursion We have discussed two applications of recursion on integers: Factorial, fibonacci, combinations, an on lists: traversing, insertion, deletion, … Let us now consider a new application that shows off the full power of recursion: parsing Parsing has numerous applications: compilers, data retrieval, data mining,… 3 Motivation The cat ate the rat. The cat ate the rat slowly. The small cat ate the big rat slowly. The small cat ate the big rat on the mat slowly. The small cat that sat in the hat ate the big rat on the mat slowly. Not all sequences of words are legal sentences The ate cat rat the How many legal sentences are there? How many legal programs are there? Are all Java programs that compile legal programs? How do we know what programs are legal? http://java.sun.com/docs/books/jls/third_edition/html/syntax.html 4 Grammar: set of rules for generating sentences in a language. For example, Examples of valid Sentences: boys see bunnies bunnies like girls … The words boys, girls, bunnies, like, see are called tokens or terminals The words Sentence, Noun, Verb are called nonterminals How big is the set of valid Sentences? A Grammar A Sentence is a Noun followed by a Verb followed by a Noun A Noun can be ‘boys’ or ‘girls’ or ‘bunnies’ A Verb can be ‘like’ or ‘see’ Sentence → Noun Verb Noun Noun → boys Noun → girls Noun → bunnies Verb → like Verb → see 5 A Recursive Grammar Sentence → Sentence and Sentence Sentence → Sentence or Sentence Sentence → Noun Verb Noun Noun → boys Noun → girls Noun → bunnies Verb → like Verb → see Examples of Sentences in this language: boys like girls boys like girls and girls like bunnies boys like girls and girls like bunnies and girls like bunnies How big is the set of valid Sentences? 6 Detour What if we want to add a period at the end of every sentence? Sentence → Sentence and Sentence . Sentence → Sentence or Sentence . Sentence → Noun Verb Noun . Noun → … Does this work? No! This produces sentences like: girls like boys . and boys like bunnies . . Sentence Sentence Sentence 7 Sentences with Periods TopLevelSentence → Sentence . Sentence → Sentence and Sentence Sentence → Sentence or Sentence Sentence → Noun Verb Noun Noun → boys Noun → girls Noun → bunnies Verb → like Verb → see Add a new rule that adds a period only at the end of the sentence. The tokens here are the 7 words plus the period (.) 8 Grammar for Simple Expressions E → integer E → ( E + E )Simple expressions: An E can be an integer. An E can be ‘(’ followed by an E followed by ‘+’ followed by an E followed by ‘)’ Set of expressions defined by this grammar is a recursively-defined set Is language finite or infinite? Here are some legal expressions: 2 (3 + 34) ((89 + 23) + (23 + (34+12))) Here are some illegal expressions: (3 3 + 4 The tokens in this grammar are (, +, ), and any integer 9 Parsing Grammars can be used to: define a language (i.e., the set of properly structured sentences) parse a sentence (checking if the sentence is in the language) To parse a sentence is to build a parse tree This is much like diagramming a sentence Example: Show that ((4+23) + 89) is a valid expression E by building a parse tree E ( E )E+ 89 ( E )E+ 4 23 E → integer E → ( E + E ) 10 Recursive Descent Parsing Idea: Use the grammar to design a recursive program to check if a sentence is in the language To parse an expression E, for instance We look for each terminal (i.e., each token) Each nonterminal (e.g., E) can handle itself by using a recursive call Pseudo Code: boolean parseE( ): if first token is an integer: return true; if first token is “(”: parseE( ); Make sure there is a “+” token; parseE( ); Make sure there is a “)” token; return true; return false; E → integer E → ( E + E ) 11 Java Code for Parsing E public static boolean parseE(Scanner scanner) { if (scanner.hasNextInt()) { scanner.nextInt(); return true; } return check(scanner, "(") && parseE(scanner) && check(scanner, "+") && parseE(scanner) && check(scanner, ")"); } E → integer E → ( E + E ) 12 Using a Parser to Generate Code We can modify the parser so that it generates stack code to evaluate arithmetic expressions: 2 PUSH 2 STOP (2 + 3) PUSH 2 PUSH 3 ADD STOP Goal: Method parseE should return a string containing stack code for expression it has parsed Method parseE can generate code in a recursive way: For integer i, it returns string “PUSH ” + i + “\n” For (E1 + E2), Recursive calls for E1 and E2 return code strings c1 and c2, respectively For (E1 + E2), return c1 + c2 + “ADD\n” Top-level method should tack on a STOP command after code received from parseE