# Inner Product Spaces

Page Contents

## Inner Product

Let V be a vector space over a field F ( = &reals. or &complex.). An #{~{inner product}} on V is a map p#: V # V -> F , _ written _ p(#~u,#~v) _ = _ #~u #. #~v _ such that

1. (#~u + #~v) #. #~w _ = _ #~u #. #~w + #~v #. #~w
2. (λ#~u) #. #~v _ = _ λ(#~u #. #~v)
3. #~u #. #~v _ = _ \${(#~v #. #~u)} _ (complex conjugate)
4. #~u #. #~u _ &ge. _ 0 _ &forall. #~u ; _ _ #~u #. #~u _ = _ 0 _ <=> _ #~u = #0

[~{Typographical note}: in general the notation \${~a} is used to denote the complex conjugate of ~a. However as the HTML overline function \${~a} does not work too well with more complex (no pun intended!) expressions for example _ \${#{~u_~i} #. #{~v_~j^T}} _ the notation (#{~u_~i} #. #{~v_~j^T})^c will be used instead.]

The following can be deduced

1. #~u #. #0 _ = _ #0 #. #~u _ = _ 0
2. #~u #. (#~v + #~w) _ = _ \${((#~v + #~w) #. #~u)} _ = _ \${(#~v #. #~u + #~w #. #~u)} _ = _ \${(#~v #. #~u)} + \${(#~w #. #~u)} _ = _ #~u #. #~v + #~u #. #~w
3. #~u #. (λ#~v) _ = _ \${((λ#~v) #. #~u)} _ = _ \${(λ(#~v #. #~u))} _ = _ \${λ}(#~u #. #~v)

## Inner Product Space

An #{~{inner product space}} over a field F is vector space V over a field F with a specific inner product, p say (there may be other inner products on V). Let (V,p) designate the inner product space.

## Norm

Let (V,p) be an inner product space over F, write _ p(#~u,#~v) _ = _ #~u #. #~v . The #{~{norm}} of #~u &in. V is defined as _ || #~u || _ = _ &sqrt.(#~u #. #~u)

#{Lemma}:

1. ||λ#~u|| _ = _ |λ| || #~u ||
2. || #~u || _ &ge. _ 0 _ &forall. #~u ; _ _ || #~u || _ = _ 0 _ <=> _ #~u = #0
3. | #~u #. #~v | _ &le. _ || #~u || || #~v ||
4. ||#~u + #~v|| _ &le. _ || #~u || + || #~v || _ (Cauchy-Schwartz &triangle. inequality.)

Proof:

1. ||λ#~u|| _ = _ &sqrt.(λ#~u #. λ#~u) _ = _ &sqrt.(λ\${λ}(#~u #. #~u)) _ = _ |λ| || #~u || .
_
2. Follows from definition of inner product part 4.
_
3. #~u = #0 then the result is trivial,
else #~u ≠ #0 _ consider #~v - λ#~u

(#~v - λ#~u) #. (#~v - λ#~u) _ _ >= _ _ 0

so

#~v #. #~v - \${λ}#~v #. #~u - λ#~u #. #~v + λ\${λ}#~u #. #~u _ _ >= _ _ 0 _ _ _ _ _ _ _ _ &forall. λ

take

λ _ = _ fract{#~v #. #~u,#~u #. #~u} _ _ so _ _ \${λ} _ = _ fract{#~u #. #~v,#~u #. #~u}

substituting:

#~v #. #~v - fract{(#~u #. #~v)(#~v #. #~u),#~u #. #~u} - fract{(#~v #. #~u)(#~u #. #~v),#~u #. #~u} + fract{(#~v #. #~u)(#~u #. #~v),#~u #. #~u} _ _ >= _ _ 0

#~v #. #~v - fract{(#~u #. #~v)(#~v #. #~u),#~u #. #~u} _ _ >= _ _ 0

&imply.

(#~v #. #~v)(#~u #. #~u) _ _ >= _ _ (#~u #. #~v)(#~v #. #~u) _ = _ | #~u #. #~v |^2

|| #~v ||^2 || #~u ||^2 _ _ >= _ _ | #~u #. #~v |^2 _ _ _ _ _ _ hence result.

4. || #~u + #~v ||^2 _ = _ (#~u + #~v) #. (#~u + #~v) _ = _ #~u #. #~u + #~u #. #~v + #~v #. #~u + #~v #. #~v

_ _ _ _ _ _ _ _ _ _ _ = _ || #~u ||^2 + || #~v ||^2 + #~u #. #~v + \${(#~u #. #~v)}

_ _ _ _ _ _ _ _ _ _ _ = _ || #~u ||^2 + || #~v ||^2 + 2 ~R(#~u #. #~v) _ _ _ _ (~R = real part)

_ _ _ _ _ _ _ _ _ _ _ =< _ || #~u ||^2 + || #~v ||^2 + 2 | #~u #. #~v |

_ _ _ _ _ _ _ _ _ _ _ =< _ || #~u ||^2 + || #~v ||^2 + 2 || #~u || || #~v || _ _ _ _ _ (by iii)

_ _ _ _ _ _ _ _ _ _ _ = _ ( || #~u || + || #~v || )^2 _ _ _ _ _ _ _ _ _ _ hence result.

## Orthogonality

In the following, V is an inner product space with inner product _ #~u #. #~v

### Orthogonal Vectors

#~u &in. V and #~v &in. V are _ #{~{orthogonal}} _ if _ #~u #. #~v = 0

### Orthogonal Set

\{#~u_~i \} is an _ #{~{orthogonal set}} _ if _ #~u_~i #. #~u_~j = 0 _ &forall. ~i ≠ ~j . _ _ #~u_~i #. #~u_~i ≠ 0 _ for any ~i , _ i.e. _ #~u_~i ≠ #0 _ any ~i .

### Orthonormal Set

\{#~u_~i \} is an _ #{~{orthonormal set}} _ if _ #~u_~i #. #~u_~j = &delta._{~i ~j} _ (Kroenecker delta). _ I.e. a set of orthogonal vectors, each with unit norm.

### Linear Independence

\{#~u_~i \} is an _ orthogonal set _ &imply. _ the #~u_~i are linearly independent.

Proof:
Suppose &sum._~i λ_~i #~u_~i = #0 for some λ_~i not all zero. _ In particular let λ_~k ≠ 0. _ Now _ (&sum._~i λ_~i #~u_~i ) #. #~u_~k = #0 _ i.e. _ &sum._~i λ_~i (#~u_~i #. #~u_~k) = #0.
But _ &sum._~i λ_~i (#~u_~i #. #~u_~k) = λ_~k (#~u_~k #. #~u_~k) _ since #~u_~i #. #~u_~k = 0 , _ ~i ≠ ~k. _ &imply. _ #~u_~k #. #~u_~k = 0 _ contradicting orthogonality conditions.