Local Decodable Codes: Hadamard and Reed-Muller Codes, Study notes of Number Theory

Download Local Decodable Codes: Hadamard and Reed-Muller Codes and more Study notes Number Theory in PDF only on Docsity! CS225: Pseudorandomness Prof. Salil Vadhan Lecture 20: Locally Decodable Codes April 24, 2007 Based on scribe notes by Sasha Schwartz, Adi Akavia, and Kevin Matulef. 1 Recap Last time we saw that locally decodable codes, if they exist, could be used to convert worst-case hard functions to average-case hard functions. Today, we will see how to obtain such codes. Recall that we wanted a locally decodable code encoding messages f : {0, 1}` → {0, 1} to codewords f̂ : {0, 1} ˆ̀ → {0, 1}: 1. ˆ̀= O(`), ie the block length (2 ˆ̀ ) is polynomial in the message length (2`), and moreover the encoding should be computable in time 2O(`) (i.e. polynomial in the message length). 2. A local (1/2 − ε)-decoding algorithm that runs in time poly(`, 1/ε). That is, given oracle access to a received word r : {0, 1} ˆ̀ → {0, 1} at distance less than (1/2− ε) from a codeword f̂ and any index x ∈ {0, 1}`, we should be able to compute f(x) with high probability in time poly(`, 1/ε). However, we will begin by working with the following modifications of the second requirement: 1. We will give a local δ-decoding algorithm for a small constant δ > 0, rather than a local (1/2− ε)-decoding algorithm. The reason is that achieving the latter will require a notion of local list-decoding, which we will introduce later. 2. We will give an algorithm for decoding codeword symbols rather than message symbols, as formalized by the following definition. Definition 1 A local δ-decoding algorithm for codeword symbols for a code C ⊂ ΣL̂ is a prob- abilistic oracle algorithm Dec with the following property. Let f̂ ∈ C be any codeword, and let g : {0, 1} ˆ̀ → Σ be such that ∆(g, f̂) < δ. Then for all x ∈ [L̂] we have Pr[Decg(x) = f̂(x)] ≥ 2/3, where the probability is taken over the coins flips of Dec. This implies the standard definition of locally decodable codes under the mild constraint that the message symbols are explicitly included in the codeword. Definition 2 An encoding algorithm Enc : {0, 1}L → ΣL̂ for the code C = Im(Enc) is systematic if there is a polynomial-time computable function I : [L] → [L̂] such that for all f ∈ {0, 1}L, f̂ = Enc(f), and all x ∈ [L], we have f̂(I(x)) = f(x), where we interpret 0 and 1 as elements of Σ in some canonical way. 1 Lemma 3 If Enc : {0, 1}L → ΣL̂ is systematic and C = Im(Enc) has a local δ-decoding algorithm for codeword symbols running in time t, then Enc has a local δ-decoding algorithm (in the standard sense) running in time t + poly(log L). 2 Hadamard Code Recall the m-variate Hadamard code, which consists of the truth tables of all Z2-linear functions L : {0, 1}m → {0, 1}. Proposition 4 The m-variate Hadamard code has a local (1/4−ε)-decoding algorithm for codeword symbols running in time poly(m, 1/ε). Proof: We are given oracle access to g : {0, 1}m → {0, 1} that is at distance less than 1/4 − ε from some (unknown) linear function L, and we want to compute L(x) at an arbitrary point x ∈ {0, 1}m. The idea is random self-reducibility: we can reduce computing L at an arbitrary point to computing L at uniformly random points, where g is likely to give the correct answer. Specifically, L(x) = L(x⊕ r)⊕ L(r) for every r, and both x⊕ r and r are uniformly distributed if we choose r R ← {0, 1}m. The probability that g differs from L at either of these points is at most 2 · (1/4− ε) = 1/2− 2ε. Thus g(x⊕ r)⊕ g(r) gives the correct answer with probability noticeably larger than 1/2. We can amplify this success probability by repetition. Specifically, if we define Decg(x) = maj1≤j≤t{g(rj) ⊕ g(rj ⊕ x)}, where r1, . . . , rt are chosen independently at random and where t = O(1/ε2), then we get error probability at most 1/3 in computing L(x). This local decoding algorithm is optimal in terms of its decoding distance and running time, but the problem is that the Hadamard code has exponentially small rate. 3 Reed–Muller Code Recall that the m-variate Reed–Muller code of degree d over Fq consists of all multivariate poly- nomials p : Fmq → Fq of total degree at most d. This code has minimum distance δ = 1 − d/q. Reed–Muller Codes are a common generalization of both Hadamard and Reed–Solomon codes, and thus we can hope that for an appropriate setting of parameters, we will be able to get the best of both kinds of codes. That is, we want to combine the efficient local decoding of the Hadamard code with the good rate of Reed-Solomon codes. Theorem 5 The Reed-Muller Code of degree d in m variables has a local 1/12-decoding algorithm for codeword symbols running in time poly(m, |F|) provided d ≤ |F|/9 and |F| ≥ 36. Note the running time of the decoder is roughly the m’th root of the block length L̂ = |F|m. When m = 1, our decoder can query the entire string and we simply obtain a global decoding algorithm for Reed-Solomon Codes (which we already know how to achieve). But for large enough m, the decoder can only access a small fraction of the received word. In fact, one can improve the running time to poly(m, d, log |F|), but the weaker result is sufficient for our purposes. 2 Proof: We prove the existence of the polynomial f̂ . Define f̂(x1, ..., x`) = ∑ α∈{0,1}` f(α)δα(x) for δα(x) = ( ∏ i : αi=1 xi )( ∏ i : αi=0 (1− xi) ) Note that for x ∈ {0, 1}`, δα(x) = 1 only when α = x, therefore f̂|{0,1}` ≡ f . Uniqueness of f̂ is derived from counting degrees of freedom. The bound on the total degree is by inspection. Thinking of f̂ as an encoding of f , let’s inspect the properties of this encoding. • For the local 1/12-decoding algorithm above, we need |F| ≥ 9`, therefore, we can take |F| = Θ(`). • The encoding time is 2O(`), as computing a single point of f̂ requires summing over 2O(`) elements. • The code is systematic, since f̂ is an extension of f . • However, we run into troubles in terms of the input length ˆ̀= ` log |F| = Θ(` log `). This is slightly too large; we aim for having ˆ̀= O(`). 4.2 Low-Degree Extension To solve the problem of the input length ˆ̀ in the multi-linear encoding, we reduce the dimension of the polynomial f̂ by changing the embedding of the domain of f : Instead of interpreting {0, 1}` ⊆ F ˆ̀ as an embedding of the domain of f in F ˆ̀ , we map {0, 1}` to Hm for some subfield H ⊆ F, and as such embed it in Fm. More precisely, we fix a subfield H ⊆ F of size (roughly) |H| = √ |F| . Choose m = `log|H| , and fix some efficient one-to-one mapping from {0, 1}` into Hm. With this mapping, view f as a polynomial f : Hm → F. Analogously to before, we have the following theorem. Theorem 7 There exists a (unique) f̂ : Fm → F of degree at most |H| − 1 in each variable s.t. f̂ |Hm ≡ f . The total degree of f̂ is m(|H| − 1) ≤ m |H| ≤ ` √ |F|. So we can 112 -decode, as long as |F| ≥ 81` 2 (recall that our decoding algorithm requires that the degree is at most |F| /9). Let’s inspecting the properties of f̂ as an encoding of f : • The input length is m log |F| = `log|H| log |F| = 2`. Namely, ˆ̀= O(`) as desired. • The code is systematic as long as our mapping from {0, 1}` to Hm is efficient. 5 5 Implications for Worst-Case/Average-Case Connections Plugging the local decoding algorithm for Reed–Muller codes given above into the general result from last time, we obtain the following Theorem 8 If there exists f : {0, 1}` → {0, 1} in E that is worst-case hard against (non-uniform) time t(`), then there exists f̂ : {0, 1}O(`) → {0, 1}O(log `) in E that is 1/12 average-case hard against (non-uniform) time t(`)/poly(`). This differs from our original goal in two ways: f̂ is not Boolean, and we only get hardness 1/12 (instead of 1/2 − ε). These can be remedied by concatenating the Reed–Muller code with a Hadamard code. Note that the Hadamard code is for message space F, so it can be 1/4-decoded by brute-force in time poly(|F|) (which is the amount of time already taken by our decoder).1 Using this, we obtain: Theorem 9 If there exists f : {0, 1}` → {0, 1} in E that is worst-case hard against (non-uniform) time t(`), then there exists f̂ : {0, 1}O(`) → {0, 1} in E that is 1/48 average-case hard against (non-uniform) time t(`)/poly(`). Hardness Amplification. Now our goal is to boost this constant hardness 1/48 to 1/2 − ε. There are some generic techniques for doing this, known as Direct Product Theorems or Yao’s XOR lemma (for Boolean functions). In those methods we use independent copies of the function at hand. For example, it Yao’s XOR lemma, we let f ′ consist of k independent copies of f̂ , f ′(x1, ..., xk) = (f̂(x1), ..., f̂ (xk)). Intuitively, if f̂ is 1/12 average-case hard, then f ′ should (1 − ( 1112 ) k)-average case hard. Similarly, if we take the XOR of k independent copies of a Boolean function, the hardness should approach 1/2 exponentially fast. These statements are (basically) true, though proving them turns out to be harder than one might expect. The main disadvantage of this approach (for our purposes) is that the input length is k` while we aim for input length of O(`). To overcome this problem, it is possible to use derandomization, namely, evaluate f̂ on dependent inputs instead of independent ones. In the next lecture, we will take a different approach, namely to generalize our notion and algorithms for locally decodable codes to locally list-decodable codes. (In fact, the aforementioned results on hardness amplification can be interpreted in a coding-theoretic language as well.) 6 Other Connections As you will see on PS6, locally decodable codes are closely related to protocols for private in- formation retrieval. Another connection, and actually the setting in which these local decoding 1This concatenation step is equivalent to applying the Goldreich–Levin hardcore predicate to f̂ , for those familiar with it. However, for the parameters we are using (where the message space is small and we are doing unique decoding), we do not need the sophisticated Goldreich–Levin algorithm (which can be interpreted as a “local list- decoding algorithm,” a notion we will define next time). 6 algorithms were first discovered, is to program self-correctors. Suppose you have a program for computing a function, such as the determinant, which happens to be a codeword in a locally de- codable code (e.g. the determinant is a low-degree multivariate polynomial). Then, even if this program has some bugs and gives the wrong answer on some small fraction of inputs, you can use the local decoding algorithm to obtain the correct answer on all inputs with high probability. 7