Instance based on Introduction to Machine Learning | CSI 5325, Study notes of Computer Science

Download Instance based on Introduction to Machine Learning | CSI 5325 and more Study notes Computer Science in PDF only on Docsity! Intro. to machine learning (CSI 5325) Lecture 18: Instance-based learning Intro. to machine learning (CSI 5325) Lecture 18: Instance-based learning Greg Hamerly Spring 2008 Some content from Tom Mitchell. 1 / 10 Intro. to machine learning (CSI 5325) Lecture 18: Instance-based learning Outline 1 k-Nearest Neighbor 2 / 10 Intro. to machine learning (CSI 5325) Lecture 18: Instance-based learning k-Nearest Neighbor When To Consider Nearest Neighbor Instances map to points in Rd Less than 20 attributes per instance Lots of training data Advantages: Training is very fast Learn complex target functions Don’t lose information Disadvantages: Slow at query time Easily fooled by irrelevant attributes 5 / 10 Intro. to machine learning (CSI 5325) Lecture 18: Instance-based learning k-Nearest Neighbor Voronoi Diagram + + − − − + − − + xq 6 / 10 Intro. to machine learning (CSI 5325) Lecture 18: Instance-based learning k-Nearest Neighbor Behavior in the Limit Consider p(x) defines probability that instance x will be labeled 1 (positive) versus 0 (negative). Nearest neighbor: As number of training examples →∞, approaches Gibbs Algorithm Gibbs: with probability p(x) predict 1, else 0 k-Nearest neighbor: As number of training examples →∞ and k gets large, approaches Bayes optimal Bayes optimal: if p(x) > .5 then predict 1, else 0 Note Gibbs has at most twice the expected error of Bayes optimal 7 / 10