Page 266 - DMTH502_LINEAR_ALGEBRA

Linear Algebra

Notes Thus , and are orthornormal set of polynomials in V.
1 2 3
In essence, the Gram-Schmidt process consists of repeated applications of a basic geometric
operation called orthogonal projection, and it is best understood from this point of view. The
method of orthogonal projection also arises naturally in the solution of an important
approximation problem.

Suppose W is a subspace of an inner product space V, and let be an arbitrary vector in V. The
problem is to find a best possible approximation to by vectors in W. This means we want to
find a vector for which is as small as possible subject to the restriction that should
belong to W. Let us make our language precise.

A best approximation to by vectors in W is a vector in W such that

for every vector in W.
3
2
By looking at this problem in R or in R , one sees intuitively that a best approximation to by
vectors in W ought to be a vector in W such that – is perpendicular (orthogonal) to W and
that there ought to be exactly one such . These intuitive ideas are correct for finite-dimensional
subspace and for some, but not all, indefinite-dimensional subspaces. Since the precise situation
is too complicated to treat here, we shall only prove the following result.
Theorem 4: Let W be a subspace of an inner product space V and let be a vector in V.

1. The vector in W is a best approximation to by vectors in W if and only if – is
orthogonal to every vector in W.
2. If a best approximation to by vectors in W exists, it is unique.

3. If W is finite-dimensional and { . . . , } is orthonormal basis for W, then the vector
1 n
( | )
= k
2 k
k k
is the (unique) best approximation to by vectors in W.
Proof: First note that if is any vector in V, then – = ( – ) + ( – ), and

2 2 2
2 Re ( | ) .
Now suppose – is orthogonal to every vector in W, that is in W and that . Then, since
– is in W, it follows that
2 2 2
=

> 2 .

Conversely, suppose that for every in W. Then from the first equation above it
follows that
2
2 Re( | ) 0

for all in W. Since every vector in W may be expressed in the form – with in W, we see that
2 Re ( | ) 2 0

260 LOVELY PROFESSIONAL UNIVERSITY