Oracles, Sampling, Generative vs. Discriminative, Schemes and Mind Maps of Probability and Statistics

Download Oracles, Sampling, Generative vs. Discriminative and more Schemes and Mind Maps Probability and Statistics in PDF only on Docsity! Oracles, Sampling, Generative vs. Discriminative 1 10-601 Introduction to Machine Learning Matt Gormley Lecture 19 March 20, 2018 Machine Learning Department School of Computer Science Carnegie Mellon University Reminders • Midterm Exam – Thursday Evening 6:30 – 9:00 (2.5 hours) – Room and seat assignments will be announced on Piazza – You may bring one 8.5 x 11 cheatsheet 2 Oracles and Sampling Whiteboard – Sampling from common probability distributions • Bernoulli • Categorical • Uniform • Gaussian – Pretending to be an Oracle (Regression) • Case 1: Deterministic outputs • Case 2: Probabilistic outputs – Probabilistic Interpretation of Linear Regression • Adding Gaussian noise to linear function • Sampling from the noise model – Pretending to be an Oracle (Classification) • Case 1: Deterministic labels • Case 2: Probabilistic outputs (Logistic Regression) • Case 3: Probabilistic outputs (Gaussian Naïve Bayes) 6 In-Class Exercise 1. With your neighbor, write a function which returns samples from a Categorical – Assume access to the rand() function – Function signature should be: categorical_sample(theta) where theta is the array of parameters – Make your implementation as efficient as possible! 2. What is the expected runtime of your function? 7 Generative vs. Discrminative Whiteboard – Generative vs. Discriminative Models • Chain rule of probability • Maximum (Conditional) Likelihood Estimation for Discriminative models • Maximum Likelihood Estimation for Generative models 8