Inner Product Spaces

Page Contents

Inner Product

Let V be a vector space over a field F ( = &reals. or &complex.). An #{~{inner product}} on V is a map p#: V # V -> F , _ written _ p(#~u,#~v) _ = _ #~u #. #~v _ such that

  1. (#~u + #~v) #. #~w _ = _ #~u #. #~w + #~v #. #~w
  2. (λ#~u) #. #~v _ = _ λ(#~u #. #~v)
  3. #~u #. #~v _ = _ ${(#~v #. #~u)} _ (complex conjugate)
  4. #~u #. #~u _ &ge. _ 0 _ &forall. #~u ; _ _ #~u #. #~u _ = _ 0 _ <=> _ #~u = #0

[~{Typographical note}: in general the notation ${~a} is used to denote the complex conjugate of ~a. However as the HTML overline function ${~a} does not work too well with more complex (no pun intended!) expressions for example _ ${#{~u_~i} #. #{~v_~j^T}} _ the notation (#{~u_~i} #. #{~v_~j^T})^c will be used instead.]

The following can be deduced

  1. #~u #. #0 _ = _ #0 #. #~u _ = _ 0
  2. #~u #. (#~v + #~w) _ = _ ${((#~v + #~w) #. #~u)} _ = _ ${(#~v #. #~u + #~w #. #~u)} _ = _ ${(#~v #. #~u)} + ${(#~w #. #~u)} _ = _ #~u #. #~v + #~u #. #~w
  3. #~u #. (λ#~v) _ = _ ${((λ#~v) #. #~u)} _ = _ ${(λ(#~v #. #~u))} _ = _ ${λ}(#~u #. #~v)

Inner Product Space

An #{~{inner product space}} over a field F is vector space V over a field F with a specific inner product, p say (there may be other inner products on V). Let (V,p) designate the inner product space.

Norm

Let (V,p) be an inner product space over F, write _ p(#~u,#~v) _ = _ #~u #. #~v . The #{~{norm}} of #~u &in. V is defined as _ || #~u || _ = _ &sqrt.(#~u #. #~u)

#{Lemma}:

  1. ||λ#~u|| _ = _ |λ| || #~u ||
  2. || #~u || _ &ge. _ 0 _ &forall. #~u ; _ _ || #~u || _ = _ 0 _ <=> _ #~u = #0
  3. | #~u #. #~v | _ &le. _ || #~u || || #~v ||
  4. ||#~u + #~v|| _ &le. _ || #~u || + || #~v || _ (Cauchy-Schwartz &triangle. inequality.)

Proof:

  1. ||λ#~u|| _ = _ &sqrt.(λ#~u #. λ#~u) _ = _ &sqrt.(λ${λ}(#~u #. #~u)) _ = _ |λ| || #~u || .
    _
  2. Follows from definition of inner product part 4.
    _
  3. #~u = #0 then the result is trivial,
    else #~u ≠ #0 _ consider #~v - λ#~u

    (#~v - λ#~u) #. (#~v - λ#~u) _ _ >= _ _ 0

    so

    #~v #. #~v - ${λ}#~v #. #~u - λ#~u #. #~v + λ${λ}#~u #. #~u _ _ >= _ _ 0 _ _ _ _ _ _ _ _ &forall. λ

    take

    λ _ = _ fract{#~v #. #~u,#~u #. #~u} _ _ so _ _ ${λ} _ = _ fract{#~u #. #~v,#~u #. #~u}

    substituting:

    #~v #. #~v - fract{(#~u #. #~v)(#~v #. #~u),#~u #. #~u} - fract{(#~v #. #~u)(#~u #. #~v),#~u #. #~u} + fract{(#~v #. #~u)(#~u #. #~v),#~u #. #~u} _ _ >= _ _ 0

    #~v #. #~v - fract{(#~u #. #~v)(#~v #. #~u),#~u #. #~u} _ _ >= _ _ 0

    &imply.

    (#~v #. #~v)(#~u #. #~u) _ _ >= _ _ (#~u #. #~v)(#~v #. #~u) _ = _ | #~u #. #~v |^2

    || #~v ||^2 || #~u ||^2 _ _ >= _ _ | #~u #. #~v |^2 _ _ _ _ _ _ hence result.

  4. || #~u + #~v ||^2 _ = _ (#~u + #~v) #. (#~u + #~v) _ = _ #~u #. #~u + #~u #. #~v + #~v #. #~u + #~v #. #~v

    _ _ _ _ _ _ _ _ _ _ _ = _ || #~u ||^2 + || #~v ||^2 + #~u #. #~v + ${(#~u #. #~v)}

    _ _ _ _ _ _ _ _ _ _ _ = _ || #~u ||^2 + || #~v ||^2 + 2 ~R(#~u #. #~v) _ _ _ _ (~R = real part)

    _ _ _ _ _ _ _ _ _ _ _ =< _ || #~u ||^2 + || #~v ||^2 + 2 | #~u #. #~v |

    _ _ _ _ _ _ _ _ _ _ _ =< _ || #~u ||^2 + || #~v ||^2 + 2 || #~u || || #~v || _ _ _ _ _ (by iii)

    _ _ _ _ _ _ _ _ _ _ _ = _ ( || #~u || + || #~v || )^2 _ _ _ _ _ _ _ _ _ _ hence result.

Orthogonality

In the following, V is an inner product space with inner product _ #~u #. #~v

Orthogonal Vectors

#~u &in. V and #~v &in. V are _ #{~{orthogonal}} _ if _ #~u #. #~v = 0

Orthogonal Set

\{#~u_~i \} is an _ #{~{orthogonal set}} _ if _ #~u_~i #. #~u_~j = 0 _ &forall. ~i ≠ ~j . _ _ #~u_~i #. #~u_~i ≠ 0 _ for any ~i , _ i.e. _ #~u_~i ≠ #0 _ any ~i .

Orthonormal Set

\{#~u_~i \} is an _ #{~{orthonormal set}} _ if _ #~u_~i #. #~u_~j = &delta._{~i ~j} _ (Kroenecker delta). _ I.e. a set of orthogonal vectors, each with unit norm.

Linear Independence

\{#~u_~i \} is an _ orthogonal set _ &imply. _ the #~u_~i are linearly independent.

Proof:
Suppose &sum._~i λ_~i #~u_~i = #0 for some λ_~i not all zero. _ In particular let λ_~k ≠ 0. _ Now _ (&sum._~i λ_~i #~u_~i ) #. #~u_~k = #0 _ i.e. _ &sum._~i λ_~i (#~u_~i #. #~u_~k) = #0.
But _ &sum._~i λ_~i (#~u_~i #. #~u_~k) = λ_~k (#~u_~k #. #~u_~k) _ since #~u_~i #. #~u_~k = 0 , _ ~i ≠ ~k. _ &imply. _ #~u_~k #. #~u_~k = 0 _ contradicting orthogonality conditions.