CS70 Cheatsheet, Cheat Sheet of Probability and Statistics

Download CS70 Cheatsheet and more Cheat Sheet Probability and Statistics in PDF only on Docsity! CS 70 CHEATSHEET ALEC LI Note 1 (Propositional Logic) • P =⇒ Q ≡¬P ∨Q • P =⇒ Q ≡¬Q =⇒ ¬P • ¬(P ∧Q) ≡¬P ∨¬Q • ¬(P ∨Q) ≡¬P ∧¬Q • P ∧ (Q ∨R) ≡ (P ∧Q)∨ (P ∧R) • P ∨ (Q ∧R) ≡ (P ∨Q)∧ (P ∨R) • ¬(∀xP (x)) ≡∃x¬P (x) • ¬(∃xP (x)) ≡∀x¬P (x) Note 2/3 (Proofs) • Direct proof • Proof by contraposition • Proof by cases • Proof by induction – Base case (prove smallest case is true) – Inductive hypothesis (assume n = k true for weak induction, assume n ≤ k true for strong induction) – Inductive step (prove n = k +1 is true) • Pigeonhole principle – Putting n +m balls in n bins =⇒≥ 1 bin has ≥ 2 balls Note 4 (Sets) • P(S) = powerset/set of all subsets; if |S| = k, |P(S)| = 2k • One to one (injection); f (x) = f (y) =⇒ x = y • Onto (surjection); (∀y∃x )( f (x) = y ) ; “hits" all of range • Bijection: both injective and surjective Note 5 (Countability & Computability) • Countable if bijection to N • Cantor-Schroder-Bernstein Theorem: bijection between A and B if there exists injections f : A → B and g : B → A • Cantor diagonalization: to prove uncountability, list out possibil- ities, construct new possibility different from each listed at one place (ex. reals ∈ (0,1), infinite binary strings, etc) • A ⊆ B and B is countable =⇒ A is countable • A ⊇ B and A is uncountable =⇒ B is uncountable • Infinite cartesian product sometimes countable (∅×∅×·· · ), some- times uncountable ({0,1}∞) • Halting Problem: can’t determine for every program whether it halts (uncomputable) • Reduction of TestHalt(P, x) to some task (here, TestTask) – define inner function that does the task if and only if P(x) halts – call TestTask on the inner function and return the result in TestHalt Note 6 (Graph Theory) • Kn has n(n−1) 2 edges • Handshaking lemma: total degree = 2e • Trees: (all must be true) – connected & no cycles – connected & has n −1 edges (n = |V |) – connected & removing an edge disconnects the graph – acyclic & adding an edge makes a cycle • Hypercubes: – n-length bit strings, connected by an edge if differs by exactly 1 bit – n-dimensional hypercube has n2n−1 edges, and is bipartite (even vs odd parity bitstring) • Eulerian walk: visits each edge once; only possible if connected and all even degree or exactly 2 odd degree • Eulerian tour: Eulerian walk but starts & ends at the same vertex; only possible if all even degree and connected • Planar graphs – v + f = e +2 – ∑ f i=1 si = 2e where si = number of sides of face i – e ≤ 3v −6 if planar (because si ≥ 3) – e ≤ 2v −4 if planar for bipartite graphs (because si ≥ 4) – nonplanar if and only if the graph contains K5 or K3,3 – all planar graphs can be colored with ≤ 4 colors Note 7 (Modular Artithmetic) • x−1 (modular inverse) exists mod m if and only if gcd(x,m) = 1 • Extended Euclidean Algorithm: x y ⌊ x/y ⌋ a b 35 12 2 −1 3 12 11 1 1 −1 11 1 11 0 1 1 0 gcd start answer – new a = old b – new b = a −b ⌊ x y ⌋ – if gcd(x, y) = 1, then a = x−1 mod y , b = y−1 mod x • Chinese Remainder Theorem: – find bases bi that are ≡ 1 mod mi and ≡ 0 mod m j for j ̸= i → bi = ci (c−1 i mod mi ) where ci = ∏ i ̸= j m j – x ≡∑ ai bi (mod ∏ mi ) – solution is unique mod ∏ mi – mi must be pairwise relatively prime in order to use CRT Note 8 (RSA) • Scheme: for primes p, q , find e coprime to (p −1)(q −1) – public key: N = pq and e – private key: d = e−1 mod (p −1)(q −1) – encryption of message m: me (mod N ) = y – decryption of encrypted message y : yd (mod N ) = m • Fermat’s Little Theorem (FLT): xp ≡ x (mod p), or xp−1 ≡ 1 (mod p) if x coprime to p • Prime Number Theorem: π(n) ≥ n lnn for n ≥ 17, where π(n) = # of primes ≤ n • Breaking RSA if we know d : – we know de − 1 = k(p − 1)(q − 1), where k ≤ e because d < (p −1)(q −1) – so de−1 k = pq −p −q +1; pq = N , so we can find p, q because we know d ,e,k Note 9 (Polynomials) • Property 1: nonzero polynomial of degree d has at most d roots • Property 2: d + 1 pairs of points (xi distinct) uniquely defines a polynomial of degree at most d • Lagrange Interpolation: – ∆i (x) =∏ i ̸= j x−x j xi−x j – P (x) =∑ i yi∆i (x) • Secret Sharing (normally under GF (p)): – P (0) = secret, P (1), . . . ,P (n) given to all people – P (x) = polynomial of degree k −1, where k people are needed to get the secret • Rational Root Theorem: for P (x) = an xn +·· ·+a0, the roots of P (x) that are of the form p q must have p | a0, q | an Note 10 (Error Correcting Codes) • Erasure Errors: k packets lost, message length n; need to send n +k packets because P (x) of degree n −1 needs n points to define it • General Errors: k packets corrupted, message length n; send n +2k packets • Berlekamp Welch: – P (x) encodes message (degree n −1) – E(x) constructed so that roots are where the errors are (degree k); coefficients unknown – Q(x) = P (x)E(x) (degree n +k −1) – substitute all (xi ,ri ) into Q(xi ) = ri E(xi ), make system of equa- tions – solve for coefficients; P (x) = Q(x) E(x) 1 CS 70 CHEATSHEET ALEC LI Note 11 (Counting) • 1st rule of counting: multiply # of ways for each choice • 2nd rule of counting: count ordered arrangements, divide by # of ways to order to get unordered • (n k )= n! k !(n−k)! = # ways to select k from n • Stars and bars: n objects, k groups → n stars, k −1 bars → (n+k−1 k−1 )= (n+k−1 n ) • Zeroth rule of counting: if bijection between A and B , then |A| = |B | • Binomial theorem: (a +b)n =∑n k=0 (n k ) ak bn−k • Hockey-stick identity: ( n k+1 )= (n−1 k )+ (n−2 k )+·· ·+ (k k ) • Derangements: Dn = (n −1)(Dn−1 +Dn−2) = n! ∑n k=0 (−1)k k ! • Principle of Inclusion-Exclusion: |A1 ∪ A2| = |A1|+ |A2|− |A1 ∩ A2| More generally, alternate add/subtract all combinations • Stirling’s approximation: n! ≈p 2πn ( n e )n Note 12 (Probability Theory) • Sample points = outcomes • Sample space =Ω= all possible outcomes • Probability space: (Ω,P(ω)); (sample space, probability function) • 0 ≤P(ω) ≤ 1, ∀ω ∈Ω; ∑ ω∈ΩP(ω) = 1 • Uniform probability: P(ω) = 1 |Ω| , ∀ω ∈Ω • P(A) =∑ ω∈A P(ω) where A is an event • If uniform: P(A) = # sample points in A # sample points inΩ = |A| |Ω| • P(A) = 1−P(A) Note 13 (Conditional Probability) • P(ω | B) = P(ω) P (B) for ω ∈ B • P(A | B) = P(A∩B) P(B) →P(A∩B) =P(A | B)P(B) • Bayes’ Rule: P(A | B) = P(B | A)P(A) P(B) = P(B | A)P(A) P(B | A)P(A)+P(B | A)P(A) . • Total Probability Rule (denom of Bayes’ Rule): P(B) = n∑ i=1 P ( B ∩ Ai )= n∑ i=1 P ( B | Ai ) P ( Ai ) for Ai partitioningΩ • Independence: P(A∩B) =P(A)P(B) or P(A | B) =P(A) • Union bound: P (⋃n i=1 Ai ) ≤∑n i=1P ( Ai ) Note 14 (Random Variables) • Bernoulli distribution: used as an indicator RV • Binomial distribution: P(X = i ) = i successes in n trials, success probability p – If X ∼ Bin(n, p), Y ∼ Bin(m, p) independent, X +Y ∼ Bin(n+m, p) • Hypergeometric distribution: P(X = k) = k successes in N draws w/o replacement from size N population with B objects (as suc- cesses) • Joint distribution: P(X = a,Y = b) • Marginal distribution: P(X = a) =∑ b∈B P(X = a,Y = b) • Independence: P(X = a,Y = b) =P(X = a)P(Y = b) • Expectation: E[X ] =∑ x∈X xP(X = x) • LOTUS: E [ g (X ) ]=∑ x∈X g (x)P(X = x) • Linearity of expectation: E[aX +bY ] = aE[X ]+bE[Y ] • X , Y independent: E[X Y ] = E[X ]E[Y ] Note 15 (Variance/Covariance) • Variance: Var(X ) = E[ ( X −µ)2] = E[X 2]−E[X ]2 – Var(c X ) = c2 Var(X ), Var(X +Y ) = Var(X )+Var(Y )+2Cov(X ,Y ) – if indep: Var(X +Y ) = Var(X )+Var(Y ) • Covariance: Cov(X ,Y ) = E[X Y ]−E[X ]E[Y ] • Correlation: Corr(X ,Y ) = Cov(X ,Y ) σXσY , always in [−1,1] • Indep. implies uncorrelated (Cov = 0), but not other way around, ex. X = { 1 p = 0.5 −1 p =−0.5 , Y =  1 X =−1, p = 0.5 −1 X =−1, p = 0.5 0 X = 1 Note 16 (Geometric/Poisson Distributions • Geometric distribution: P(X = i ) = exactly i trials until success with probability p; use X −1 for failures until success – Memoryless Property: P(X > a +b | X > a) = P(X > b); i.e. wait- ing > b units has same probability, no matter where we start • Poisson distribution: λ= average # of successes in a unit of time – X ∼ Pois(λ), Y ∼ Pois(µ) independent: X +Y ∼ Pois(λ+µ) – X ∼ Bin(n, λn ) where λ> 0 is constant, as n →∞, X → Pois(λ) Note 20 (Continuous Distributions) • Probability density function: – fX (x) ≥ 0 for x ∈R – ∫ ∞ −∞ fX (x) dx = 1 • Cumulative density function: FX (x) = P(X ≤ x) = ∫ x −∞ fX (t) dt , fX (x) = d dx FX (x) • Expectation: E[X ] = ∫ ∞ −∞ x fX (x) dx • LOTUS: E [ g (X ) ]= ∫ ∞ −∞ g (x) fX (x) dx • Joint distribution: P(a ≤ X ≤ b,c ≤ Y ≤ d) – fX Y (x, y) ≥ 0, ∀x, y ∈R – ∫ ∞ −∞ ∫ ∞ −∞ fX Y (x, y) dx dy = 1 • Marginal distribution: fX (x) = ∫ ∞ −∞ fX Y (x, y) dy ; integrate over all y • Independence: fX Y (x, y) = fX (x) fY (y) • Conditional probability: fX |A(x) = fX (x) P(A) , fX |Y (x | y) = fX Y (x,y) fY (y) • Exponential distribution: continuous analog to geometric distribu- tion – Memoryless property: P(X > t + s | X > t ) =P(X > s) – Additionally, P(X < Y | min(X ,Y ) > t ) =P(X < Y ) – If X ∼ Exp(λX ), Y ∼ Exp(λY ) independent, then min(X ,Y ) ∼ Exp(λX +λY ) and P(X ≤ Y ) = λX λX +λY • Normal distribution (Gaussian distribution) – If X ∼N (µX ,σ2 X ), Y ∼N (µY ,σ2 Y ) independent: Z = aX +bY ∼N (aµX +bµY , a2σ2 X +b2σ2 Y ) • Central Limit Theorem: if Sn = ∑n i=1 Xi , all Xi iid with mean µ, variance σ2, Sn n ≈N ( µ, σ2 n ) ; Sn −nµ σ p n ≈N (0,1). Note 17 (Concentration Inequalities, LLN) • Markov’s Inequality: P(X ≥ c) ≤ E[X ] c , if X nonnegative, c > 0 • Generalized Markov: P(|Y | ≥ c) ≤ E[|Y |r ] cr for c,r > 0 • Chebyshev’s Inequality: P (∣∣X −µ∣∣≥ c )≤ Var(X ) c2 for µ= E[X ], c > 0 – Corollary: P (∣∣X −µ∣∣≥ kσ )≤ 1 k2 for σ=p Var(X ), k > 0 • Confidence intervals: – For proportions, P (∣∣p̂ −p ∣∣≥ ε)≤ Var(p̂) ε2 ≤ δ, where δ is the confi- dence level (95% interval → δ= 0.05) – p̂ = proportion of successes in n trials, Var(p̂) = p(1−p) n – =⇒ n ≥ 1 4ε2δ – For means, P (∣∣∣ 1 n Sn −µ ∣∣∣≥ ε)≤ σ2 nε2 = δ – Sn =∑n i=1 Xi , all Xi ’s iid mean µ, variance σ2 – =⇒ ε= σp nδ , interval = Sn ± σp nδ – With CLT, P (∣∣An −µ∣∣≤ ε)=P(∣∣∣ (An−µ) p n σ ∣∣∣≤ ε p n σ ) ≈ 1−2Φ ( − ε p n σ ) = 1−δ. Here, An = 1 n Sn and CLT gives An ≈N ( µ, σ 2 n ) ; use inverse cdf to get ε • Law of large numbers: as n → ∞, sample average of iid X1, . . . Xn tends to population mean 2