Linear Space Theory - Lecture Notes | ECON 712, Study notes of Introduction to Macroeconomics

Download Linear Space Theory - Lecture Notes | ECON 712 and more Study notes Introduction to Macroeconomics in PDF only on Docsity! Economics 712 Steven N. Durlauf Fall 2007 Lecture Notes 1. Linear Space Theory Much of time series analysis is based on the construction of linear decompositions of the processes under study. These ideas are based on some results based that are taken from the mathematics of linear spaces. Linear spaces are also sometimes called vector spaces. These lecture notes describe some basic definitions and results. The results are all standard. I follow Royden (1988) and Simmons (1963) closely in terms of presentation. Ash (1976) chapter 3 is also useful. 1. Basic definitions Definition 1.1.1 Linear space Let denote a non-empty set. For each pair of elements Γ i γ and jγ contained in Γ , assume that ∀ an operation called addition, denoted as +, such that i jγ γ+ is an element of . Suppose that the addition operation obeys Γ i. i j j iγ γ γ γ+ = + ii. ( ) ( )i j k i j kγ γ γ γ γ γ+ + = + + iii. Γ contains a unique element, 0, such that 0γ γ+ = γ∀ ∈Γ iv. For each γ ∈Γ ∀ an element γ− such that ( ) 0γ γ+ − = 1 Further, for any element of the set of complex numbers C, assume that c may be combined with any c γ ∈Γ to produce another element of Γ , cγ . Suppose that this operation, called scalar multiplication, obeys v. ( )i j ic c jcγ γ γ+ = + γ vi. ( )c d c dγ γ γ+ = + vii. ( ) ( )cd c dγ γ= viii. 1γ γ= Then, the set and the addition and multiplication operations define a linear space. Γ If multiplication is defined with respect to elements of the real line, R, rather than C, then the set Γ and the addition and multiplication operations define a real linear space. The properties which will developed for linear spaces will apply to real linear spaces. I will switch between the two as appropriate. In essence, the definition of a linear space requires that the space is closed with respect to addition of any pair of elements of the space and that multiplying any element by a scalar produces another element of the space. This definition does not require the space to be closed in any sense. In order to discuss whether the space is closed, one associates a linear space with a norm. A norm is a function that defines distances between elements of a space. This definition is required to fulfill certain properties. Definition 1.1.2. Norm For a linear space Γ , a norm, denoted as ⋅ , is a mapping from Γ to R such that i. 0; 0 0γ γ γ≥ = → = ii. c cγ γ= ∀ c C∈ 2 Our final definition describes spaces where the inner product is defines the norm. Definition 1.1.9. Hilbert space A Hilbert space is a Banach space with an inner product such that 2,i i iγ γ γ〈 〉 = . Hilbert spaces are the foundations for the theory of time series. 2. Properties of Hilbert spaces This section discusses a few properties of the general structure of Hilbert space. Thes properties will depend critically on the notion of orthogonality between two elements of a Hilbert space. Hilbert spaces possess a natural notion of orthogonality between two elements. Definition 1.2.1 Orthogonality Two elements iγ and jγ are orthogonal if , 0i jγ γ〈 〉 = i. Projections Orthogonality allows one to construct various decompositions of elements of Hilbert spaces and hence of the spaces themselves. To see this, take any two elements of a space, iγ and jγ and consider the element , ,i i j j j j γ γ γ γ γ γ 〈 〉 − 〈 〉 . 5 This element is also a member of the Hilbert space since the space is linear. Further, this element is orthogonal to jγ since , , , , ,i i j j j i j i j j j 0 γ γ γ γ γ γ γ γ γ γ γ 〈 〉 〈 − 〉 = 〈 〉 − 〈 〉 = 〈 〉 This construction is suggestive of a fundamental property of Hilbert spaces, namely that there exist ways to decompose these spaces into mutually orthogonal subspaces. Theorem 1.2.1. Decomposability of Hilbert space into orthogonal subspaces Suppose that and are Hilbert spaces such that Γ 1Γ 1Γ ⊆ Γ . Then there exists a unique Hilbert space such that 2Γ ⊆ Γ i. 1 2Γ = Γ ⊕Γ ii. 1 2Γ ⊥ Γ The operator “ ” is called the direct sum. The direct sum of two spaces produces a third space whose elements consist of all linear combinations of elements of the original spaces and the limits of all such combinations. ⊕ Corollary 1.2.1. Decomposition of an element of a Hilbert space into orthogonal components Suppose that and are Hilbert spaces such that Γ 1Γ 1Γ ⊆ Γ . For a given element γ ∈Γ , there exist unique elements 1 1γ ∈Γ and 2 2γ ∈Γ , (where 2Γ is defined in Theorem 1.2.1) such that i. 1 2γ γ γ= + 6 ii. 1 2γ γ⊥ In this corollary, 1γ is called the projection of γ onto 1Γ . This projection has an interpretation in terms of an optimization problem. Suppose one want to solve the problem 1 minξ γ ξ∈Γ − In other words, one wants to find that element of the subspace 1Γ that is closest to an element of . The solution to this problem is Γ 1ξ γ= . This follows immediately from recognizing that 1 2 1 2γ ξ γ γ ξ γ ξ γ− = + − = − + where the second equality follows immediately from the orthogonality of and . 1Γ 2Γ ii. Orthogonal Bases for Hilbert space Definition 1.2.3. Orthonormal set A subset of a given Hilbert space is called an orthonormal set if S i. each element of is orthogonal to every other element S ii. the norm of every element equals 1 Definition 1.2.4. Orthonormal basis An orthonormal set is an orthonormal basis with respect to a given Hilbert space if it is not a proper subset of any other orthonormal set in the space. S H 7