Banach Spaces

Norm on a Vector Space

Suppose that

V

is a vector space a subfield

F

of the complex numbers

ℂ

a norm is a function

‖ \cdot ‖ : V \to ℝ

such that

$‖ x + y ‖ \leq ‖ x ‖ + ‖ y ‖$ for all $x, y \in X$ .
for any $c \in F$ , we have $‖ c x ‖ = | c | ‖ x ‖$ for all $x \in V$ and
for all $x \in X$ , if $‖ x ‖ = 0$ then $x = 0$ .

Norm Is Zero Iff Input Is Zero

Suppose that

‖ \cdot ‖ : V \to ℝ

is a norm, then we have for any

v \in V

‖ v ‖ = 0_{F} ⟺ v = 0_{V}

We already know that if $‖ v ‖ = 0$ then $v = 0$ by property 3 of the norm. Now we verify that the norm of zero is zero, but it is because we have: $‖ 0_{V} ‖ = ‖ 0_{F} \cdot 0_{V} ‖ = 0_{F} ‖ 0_{V} ‖ = 0_{F}$ .

Non-negativity of the Norm

Suppose that

‖ \cdot ‖ : V \to ℝ

is a norm, then the following is true for every

v \in V

0 \leq ‖ v ‖

Using properties 1 and 2 we have

0 = ‖ v - v ‖ \leq ‖ v ‖ + ‖ - v ‖ = 2 ‖ v ‖

thus by division by 2 we deduce that

0 \leq ‖ v ‖

as needed.

Norm Induced Metric

Suppose that

‖ \cdot ‖ : V \to ℝ

is a norm, then the function

d (x, y) = ‖ x - y ‖

is a metric.

TODO: Add the proof here.

Normed Vector Space

A normed vector space is a vector space

V

over

K \in {ℝ, ℂ}

, with a norm over

V

We will usually refer to normed vector spaces as an ordered pair $(V, ‖ \cdot ‖)$

Norm Topology

Suppose that

‖ \cdot ‖ : V \to ℝ

is a norm, then we define the norm topology as the metric topology generated by the norm induced metric

Norm Closure

Suppose that

X

is a normed vector space, if

A \subseteq X

then we denote

A^{=}

to be the norm closure of

A

X

which is the closure of

A

in the norm topology

Sequential Compactness of a Metric Space

A metric space

X

is said to be sequentially compact if for every sequence in

X

has a convergence subsequence.

Also note that given any subset $M \subseteq X$ we say that it compact if $M$ is considered as a subspace of $X$ , that is every sequence in $M$ has a convergent subsequence whose limit is in $M$ .

A Compact Subset of a Metric Space Is Closed and Bounded

A compact subset

M

of a metric space is closed and bounded

TODO: Add the proof here.

Linear Independence Coefficient Inequality

Let

{x_{1}, \dots, x_{n}}

be a linearly independent set of vectors in a normed space

X

(of any dimension). Then there is a number

c > 0

such that for every choice of scalars

α_{1}, \dots, α_{n}

we have

‖ α_{1} x_{1} + \dots + α_{n} x_{n} ‖ \geq c (| α_{1} | + \dots + | α_{n} |)

We write

s = | α_{1} | + \dots + | α_{n} |

. If

s = 0

, all

α_{i}

are zero, so that (1) holds for any

c

. Let

s > 0

. Then (1) is equivalent to the inequality which we obtain from (1) by dividing by

s

and writing

β_{i} = α_{i} / s

, that is,

‖ β_{1} x_{1} + \dots + β_{n} x_{n} ‖ ≧ c (\sum_{j = 1}^{n} | β_{j} | = 1) .

Hence it suffices to prove the existence of a

c > 0

such that (2) holds for every

n

-tuple of scalars

β_{1}, \dots, β_{n}

with

\sum | β_{j} | = 1

. Suppose that this is false. Then there exists a sequence

(y_{m})

of vectors

y_{m} = β_{1}^{(m)} x_{1} + \dots + β_{n}^{(m)} x_{n} (\sum_{j = 1}^{n} | β_{i}^{(m)} | = 1)

such that

‖ y_{m} ‖ ⟶ 0 as m ⟶ \infty

Now we reason as follows. Since

\sum | 𝜷_{j}^{(m)} | = 1

, we have

| 𝜷_{j}^{(m)} | ≦ 1

. Hence for each fixed

j

the sequence

(β_{j}^{(m)}) = (β_{j}^{(1)}, β_{j}^{(2)}, \dots)

is bounded. Consequently, by the Bolzano-Weierstrass theorem,

(𝜷_{1}^{(m)}

) has a convergent subsequence. Let

β_{1}

denote the limit of that subsequence, and let (

y_{1, m}

) denote the corresponding subsequence of

(y_{m})

. By the same argument,

(y_{1, m})

has a subsequence

(y_{2, m})

for which the corresponding subsequence of scalars

β_{2}^{(m)}

converges; let

β_{2}

denote the limit. Continuing in this way, after

n

steps we obtain a subsequence

(y_{n, m}) = (y_{n, 1}, y_{n, 2}, \dots)

(y_{m})

whose terms are of the form

y_{n, m} = \sum_{j = 1}^{n} γ_{j}^{(m)} x_{j} (\sum_{j = 1}^{n} | γ_{j}^{(m)} | = 1)

with scalars

γ_{j}^{(m)}

satisfying

γ_{i}^{(m)} ⟶ β_{i}

m ⟶ \infty

. Hence, as

m ⟶ \infty

y_{n, m} ⟶ y = \sum_{j = 1}^{n} β_{i} x_{i}

where

\sum | β_{j} | = 1

, so that not all

β_{j}

can be zero. Since

{x_{1}, \dots, x_{n}}

is a linearly independent set, we thus have

y \neq 0

. On the other hand,

y_{n, m} ⟶ y

implies

‖ y_{n, m} ‖ ⟶ ‖ y ‖

, by the continuity of the norm. Since

‖ y_{m} ‖ ⟶ 0

by assumption and

(y_{n, m})

is a subsequence of

(y_{m})

, we must have

‖ y_{n, m} ‖ ⟶ 0

. Hence

‖ y ‖ = 0

, so that

y = 0

by (N2) in Sec. 2.2. This contradicts

y \neq 0

, and the lemma is proved.

In a Finite Dimensional Normed Space Compactness Is Equivalent to Closed and Bounded

Suppose that

X

is a normed vector space, then given any

M \subseteq X

then

M

is compact iff it is closed and bounded.

Compactness implies closedness and boundedness by the above lemma, so we prove the converse. Let

M

be closed and bounded. Let

\dim X = n

and

{e_{1}, \dots, e_{n}}

a| basis for

X

. We consider any sequence

(x_{m})

M

. Each

x_{m}

has a representation

x_{m} = ξ_{1}^{(m)} e_{1} + \dots + ξ_{n}^{(m)} e_{n} .

Since

M

is bounded, so is

(x_{m})

, say,

‖ x_{m} ‖ ≦ k

for all

m

. By the above lemma

k \geq ‖ x_{m} ‖ = ‖ \sum_{j = 1}^{n} ξ_{j}^{(m)} e_{i} ‖ \geq c \sum_{j = 1}^{n} | ξ_{j}^{(m)} |

where

c > 0

. Hence the sequence of numbers

(ξ_{i}^{(m)})

(

j

fixed) is bounded and, by the Bolzano-Weierstrass theorem, has a point of accumulation

ξ_{i}

; here

1 ≦ i ≦ n

. As in the proof of Lemma

2.4 - 1

we conclude that

(x_{m})

has a subsequence

(z_{m})

which converges to

z = \sum ξ_{j} e_{j}

. Since

M

is closed,

z \in M

. This shows that the arbitrary sequence

(x_{m})

M

has a subsequence which converges in

M

. Hence

M

is compact.

Riesz's

Let

Y

and

Z

be subspaces of a normed space

X

(of any dimension), and suppose that

Y

is closed and is a proper subset of

Z

. Then for every real number

θ

in the interval

(0,1)

there is a

z \in Z

with

‖ z ‖ = 1

such that for each

y \in Y

we have that:

‖ x - y ‖ \geq θ

TODO: Add the proof here.

Every Finite Dimensional Subspace of a Normed Space Is Complete

TODO: Add the content for the proposition here.

TODO: Add the proof here.

In particular every finite dimensional normed space is complete

A Subspace of a Complete Metric Space Is Complete Iff It Is Closed

A subspace

M

of a complete metric space

X

is itself complete if and only if the set

M

is closed in

X

TODO: Add the proof here.

Every Finite Dimensional Subspace of a Normed Space Is Closed

Every finite dimensional subspace

Y

of a normed space

X

is closed in

X

Since

Y

is finite dimensional, then we know that it is complete therefore

Y

is closed in

X

If the Closed Unit Ball Is Compact Then the Space Is Finite Dimensional

If a normed space

X

has the property that the closed unit ball

M = {x : ‖ x ‖ \leq 1}

is compact, then

X

is finite dimensional

For the sake of contradiction assume that

M

is compact but

\dim X = \infty

. We choose any

x_{1} \in X

of norm 1. This

x_{1}

generates a one dimensional subspace

X_{1}

X

, and is therefore closed and is a proper subspace of

X

since

\dim X = \infty

. By Riesz's lemma there is an

x_{2} \in X

of norm 1 such that

‖ x_{2} - x_{1} ‖ ≧ θ = \frac{1}{2} .

The elements

x_{1}, x_{2}

generate a two dimensional proper closed subspace

X_{2}

X

. By Riesz's lemma there is an

x_{3}

of norm 1 such that for all

x \in X_{2}

we have

‖ x_{3} - x ‖ ≧ \frac{1}{2} .

In particular,

\begin{matrix} ‖ x_{3} - x_{1} ‖ \geq \frac{1}{2}, \\ ‖ x_{3} - x_{2} ‖ \geq \frac{1}{2} . \end{matrix}

Proceeding by induction, we obtain a sequence (

x_{n}

) of elements

x_{n} \in M

such that

‖ x_{m} - x_{n} ‖ ≧ \frac{1}{2} (m \neq n) .

Obviously,

(x_{n})

cannot have a convergent subsequence. This contradicts the compactness of

M

. Hence our assumption

\dim X = \infty

is false, and

\dim X < \infty

Complete Metric Space

We say that

(X, d)

is complete if every cauchy sequence is convergent.

A Normed Vector Space is Complete iff Every Absolutely Summable Sequence is Summable

Let

X

be a normed vector space, then it is complete iff every absolutely convergent sequence is summable.

$⟹$ Suppose that $\sum ‖ a_{n} ‖ < \infty$ , since $X$ is complete, then we just have to show that $s_{n} : = \sum_{i = 1}^{n}$ is cauchy, so let $ϵ \in ℝ^{+}$ since $\sum ‖ a_{n} ‖ < \infty$ then by the cauchy criterion for series we know that there is some $N^{'}$ such that for all $n, m \geq N^{'}$ we have that $‖ \sum_{k = n + 1}^{m} a_{i} ‖ < ϵ$ but note that $\sum_{k = n + 1}^{m} a_{i} = s_{m} - s_{n}$ so select $N = N^{'}$ for any $j, k \geq N$ we have $| s_{j} - s_{k} | = ‖ \sum_{k = n + 1}^{m} a_{i} ‖ < ϵ$ therefore $(s_{n})$ is cauchy and therefore it converges so that $\sum a_{i} < \infty$

$⟸$ Suppose that $\sum ‖ a_{i} ‖ < \infty ⟹ \sum a_{i} < \infty$ now we want to prove that $X$ is complete, so let $(x_{n}) : ℕ_{1} \to X$ be a cauchy sequence, and let's prove that it converges to some limit.

Since $(x_{n})$ is cauchy then we can construct the following subsequence inductively, for each $k \in ℕ_{1}$ we obtain some $M_{k}$ such that for all $n, m \geq M_{k}$ we have $‖ x_{n} - x_{m} ‖ < \frac{1}{2^{k}}$ . Now let $i \in ℕ_{1}$ to obtain some $M_{i}$ and also consider $i + 1$ to obtain $M_{i + 1}$ , and in this case we will define $N_{i + 1} = \max (M_{i}, M_{i + 1}) + 1$ and $N_{1} = M_{1}$ .

Notice that it is true that $N_{j} < N_{j + 1}$ (as we used +1) therefore $x_{N_{i}}$ is a subsequence, and then also note that since $N_{j}, N_{j + 1} \geq N_{j}$ we have that $‖ x_{N_{j}} - x_{N_{j + 1}} ‖ < \frac{1}{2^{j}}$ From here we note that $x_{N_{k}} = x_{N_{1}} + \sum_{i = 1}^{k - 1} (x_{N_{i + 1}} - x_{N_{i}})$ and that $\sum_{i = 1}^{k - 1} (x_{N_{i + 1}} - x_{N_{i}})$ converges absolutely because $\sum_{i = 1}^{k - 1} | (x_{N_{i + 1}} - x_{N_{i}}) | \leq \sum_{i = 1}^{k - 1} \frac{1}{2^{k}}$ and so by the comparison test it converges and so $x_{N_{1}} + \sum_{i = 1}^{k - 1} (x_{N_{i + 1}} - x_{N_{i}})$ converges so that $(x_{N_{i}})$ converges, as needed.

Banach Space

Every complete normed vector space is called a banach space.

These spaces are named after Stefan Banach.

Any Finite Dimensional Vector Space With a Norm Is Complete

TODO: Add the content for the corollary here.

TODO: Add the proof here.

Standard Norm on Little L P

p \in ℕ_{1}

, and

x \in 𝔽^{n}

then we define

{‖ x ‖}_{p} = {(\sum_{n = 1}^{\infty} {| x_{n} |}^{p})}^{\frac{1}{p}}

Linear Operator

A linear operator is a synonym for a linear transformation where

T : X \to Y

and both

X, Y

are normed vector spaces.

Bounded Linear Operator

Suppose that

L : X \to Y

is a linear operator, then we say that it is bounded diff

\exists M \in ℝ^{+}

such that for any

x \in X

, we have

{‖ L x ‖}_{Y} \leq M {‖ x ‖}_{X}

The intuitive meaning is that there is some scalar such that the linear operator can only increase the length of the vector by that scalar multiple. For example a linear transform that multiplies all vector by some constant, will trivially satisfy this. We can re-obtain our familiar notion of bounededness by seeing that any bounded subset of $X$ becomes a bouneded subset of $Y$ .

But note that even though the above paragraph is true, it's still not the same as the regular notion of boundedness we encounter in calculus, where we had that the range of the function is a bounded set. This is different because our definition of bounded above may not have the range being bounded.

On a new note, if we ask ourselves what value of $M$ is the smallest such that the equation ${‖ L x ‖}_{Y} \leq M {‖ x ‖}_{X}$ holds true for all $x \in X$ . For a given value $x_{0}$ if the smallest value of $M$ where the equation holds true is $M_{0}$ then the smallest value that works for all $x \in X$ is at least $M_{0}$ .

Therefore by first noting that when $x = 0$ then $T x = 0$ and therefore any value of $M$ works for $x = 0$ , specifically $M = 0$ works, and thus yields no constraint on the value of $M$ that works for all $x \in X$ then we may restrict our search on values of $x \in X$ such that $X \neq 0$ therefore we are looking for an $M$ such that $\frac{‖ T x ‖}{‖ x ‖} \leq M$ for all $x \in X$ .

This implies that $M$ must be at least as large as $\sup_{x \in X ∖ {0}} (\frac{‖ T x ‖}{‖ x ‖})$ , thus this motivates the following definition:

Operator Norm

Let

T

be a bounded linear operator between two normed vector spaces

(V, {‖ \cdot ‖}_{V})

and

(W, {‖ \cdot ‖}_{W})

the the operator norm written as

{‖ T ‖}_{op}

such that

{‖ T ‖}_{op} = \sup_{v \in V ∖ {0}} (\frac{{‖ T (v) ‖}_{W}}{{‖ v ‖}_{V}})

Operator Norm Restricted to the Circle

Suppose that

T : V \to W

is a linear operator, then we have:

{‖ T ‖}_{op} = \sup_{v \in V, ‖ v ‖ = 1} {‖ T v ‖}_{W}

\begin{matrix} {| T |}_{op} & = \sup_{v \in V ∖ {0}} (\frac{{‖ T (v) ‖}_{W}}{{‖ v ‖}_{V}}) \\ = \sup_{v \in V ∖ {0}} ({‖ T (\frac{v}{{‖ v ‖}_{V}}) ‖}_{W}) \\ = \sup_{x \in V, ‖ x ‖ = 1} ({‖ T (x) ‖}_{W}) \end{matrix}

The Operator Norm of a Bounded Operator Is The Smallest Number Satisfying an Inequality

Suppose that

C \in ℝ

is the smallest number such that for any

v \in V

{‖ L v ‖}_{W} \leq C {‖ v ‖}_{V}

Then

C = {‖ L ‖}_{op}

Just to note, the inequality which we know from the above theorems: $‖ T x ‖ \leq {‖ T ‖}_{op} ‖ x ‖$ is one to remember.

The Image of a Linear Opertor Is a Vector Space

T

is a linear operator then

im (T)

is a vector space

TODO: Add the proof here.

If a Linear Operator Has a Finite Dimensional Domain Then the Domain of Its Image Is Less Than It

T

is a linear operator, then if

\dim (dom (T)) = n < \infty

then we have

\dim (im (T)) \leq n

TODO: Add the proof here.

The Nullspace of a Linear Operator Is a Vector Space

TODO: Add the content for the proposition here.

TODO: Add the proof here.

Proof. (a) We take any

y_{1}, y_{2} \in ℛ (T)

and show that

α y_{1} + β y_{2} \in ℛ (T)

for any scalars

α, β

. Since

y_{1}, y_{2} \in ℛ (T)

, we have

y_{1} = T x_{1}

y_{2} = T x_{2}

for some

x_{1}, x_{2} \in 𝒪 (T)

, and

α x_{1} + β x_{2} \in 𝒟 (T)

because

𝒟 (T)

is a vector space. The linearity of

T

yields

T (α x_{1} + β x_{2}) = α T x_{1} + β T x_{2} = α y_{1} + β y_{2} .

Hence

α y_{1} + β y_{2} \in ℛ (T)

. Since

y_{1}, y_{2} \in ℛ (T)

were arbitrary and so were the scalars, this proves that

ℛ (T)

is a vector space. (b) We choose

n + 1

elements

y_{1}, \dots, y_{n + 1}

ℛ (T)

in an arbitrary fashion. Then we have

y_{1} = T x_{1}, \dots, y_{n + 1} = T x_{n + 1}

for some

x_{1}, \dots, x_{n + 1}

𝒟 (T)

. Since

\dim 𝒟 (T) = n

, this set

{x_{1}, \dots, x_{n + 1}}

must be linearly dependent. Hence

α_{1} x_{1} + \dots + α_{n + 1} x_{n + 1} = 0

for some scalars

α_{1}, \dots, α_{n + 1}

, not all zero. Since

T

is linear and

T 0 = 0

, application of

T

on both sides gives

T (α_{1} x_{1} + \dots + α_{n + 1} x_{n + 1}) = α_{1} y_{i} + \dots + α_{n + 1} y_{n + 1} = 0.

This shows that

{y_{1}, \dots, y_{n + 1}}

is a linearly dependent set because the

α_{i}

's are not all zero. Remembering that this subset of

𝒜 (T)

was chosen in an arbitrary fashion, we conclude that

ℛ (T)

has no linearly independent subsets of

n + 1

or more elements. By the definition this means that

\dim ℜ (T) ≦ n

. (c) We take any

x_{1}, x_{2} \in 𝒩 (T)

. Then

T x_{1} = T x_{2} = 0

. Since

T

is linear, for any scalars

α, β

we have

T (α x_{1} + β x_{2}) = α T x_{1} + β T x_{2} = 0.

This shows that

α x_{1} + β x_{2} \in 𝒩 (T)

. Hence

𝒩 (T)

is a vector space.

Continuity of a Linear Operator

Suppose that

L : X \to Y

is a linear operator between two normed vector spsces , then

L

is continuous at

a \in X

iff

\forall p \in X, \forall ϵ \in ℝ^{+}, \exists δ \in ℝ^{+} st

‖ p ‖ < δ ⟹ ‖ L p ‖ < ϵ

$⟹$ Let $ϵ \in ℝ^{+}$ therefore since $L$ is continuous we obtain some $δ \in ℝ^{+}$ such that for any $x \in X$ if $‖ x - a ‖ < δ$ we have $‖ f (x) - f (a) ‖ < ϵ$ .

Now let $p \in X$ and assume that $‖ p ‖ < δ$ and take $x = p - a$ therefore we know that $‖ L (p - a) - L (a) ‖ < ϵ$ which implies that $‖ L p ‖ < ϵ$ as needed.

$⟸$ Now we prove that $L$ is continuous, so let $ϵ \in ℝ^{+}$ ,

Continuity Boundedness Equivalence of Linear Operators

Suppose that

L : X \to Y

is a linear operator then the following are equivalent so long as

X

is non-empty:

Proving $1 ⟹ 2$ is easy since $X$ is non-empty, so we'll prove $2 ⟹ 3$ . Suppose that $L$ is continuous at some point $x \in X$ so by letting $ϵ = 1$ we obtain some $δ$ such that for any $y \in X$ if $‖ x - y ‖ < δ$ then we have $‖ L x - L y ‖ < 1$

But then note that for any $z \in X$ $‖ δ \frac{z}{‖ z ‖} + x - x ‖ \leq δ$ therefore we have $‖ L (δ \frac{z}{‖ z ‖} + x) - L (x) ‖ = ‖ L (δ \frac{z}{‖ z ‖}) ‖ \leq 1$ to conclude $‖ T z ‖ \leq \frac{1}{δ} ‖ z ‖$ So that $L$ is bounded.

$3 ⟹ 1$ We need to show that $L$ is continuous, and so let $a \in X$ and $ϵ \in ℝ^{+}$ , since $L$ is bounded, we have some $M \in ℝ^{+}$ such that for any $z \in X$ we have $‖ L z ‖ \leq M ‖ z ‖$ , now take $δ = \frac{ϵ}{M}$ and let $x \in X$ and assume that $‖ x - a ‖ < δ = \frac{ϵ}{M}$ $‖ L (x - a) ‖ \leq M ‖ x - a ‖ < M \frac{ϵ}{M} = ϵ$ so $L$ is continuous, therefore all three statements are equivalent.

Adjoint of a Linear Operator

Suppose that

L

is a linear operator on a vector space

V

then the adjoint of

L

is a function

L^{*} : V \to V

such that:

⟨ L (x), y ⟩ = ⟨ x, L^{*} (y) ⟩

The Adjoint Is a Unique Linear Operator

The adjoint of a linear opeartor

L

is also a linear operator and is unique

TODO: Add the proof here.

The Adjoint of a Linear Operator Exists When the Vector Space Is Finite Dimensional

As per title.

TODO: Add the proof here.

Lemma 6.2 (one-dimensional extension, real case) Let

X

be a real normed linear space, let

M \subseteq X

be a linear subspace, and let

ℓ \in M^{*}

be a bounded linear functional on

M

. Then, for any vector

x_{1} \in X \ M

, there exists a linear functional

ℓ_{1}

M_{1} = span {M, x_{1}}

that extends

ℓ

(i.e.

ℓ_{1} ↾ M = ℓ

) and satisfies

{‖ ℓ_{1} ‖}_{M_{1}^{*}} = ‖ ℓ ‖_{M^{*}}

A Linear Operator With a Finite Dimensional Domain Is Bounded

If a normed space

X

is finite dimensional, then every linear operator on

X

is bounded.

Proof. Let

\dim X = n

and

{e_{1}, \dots, e_{n}}

a basis for

X

. We take any

x = \sum ξ_{j} e_{j}

and consider any linear operator

T

X

. Since

T

is linear,

‖ T x ‖ = ‖ \sum ξ_{i} T e_{i} ‖ ≦ \sum | ξ_{i} | ‖ T e_{i} ‖ ≦ \max_{k} ‖ T e_{k} ‖ \sum | ξ_{j} |

(summations from 1 to

n

). To the last sum we apply Lemma 2.4-1 with

α_{i} = ξ_{i}

and

x_{i} = e_{j}

. Then we obtain

\sum | ξ_{i} | ≦ \frac{1}{c} ‖ \sum ξ_{i} e_{i} ‖ = \frac{1}{c} ‖ x ‖ .

Together,

‖ T x ‖ ≦ γ ‖ x ‖ where γ = \frac{1}{c} \max_{k} ‖ T e_{k} ‖ .

From this and (1) we see that

T

is bounded.