Equivalent Properties of a Linear Operator

This is a summary of material from the course ``Advanced topics in
machine learning’’ by Arthur Gretton with slightly more intermediate
steps.

Theorem
Let and
be normed linear spaces. If $L:\mathcal{F}\mapsto\mathcal{G}$ is
a linear operator, then the following three conditions are equivalent.

$L$ is a bounded operator.

$L$ is continuous on $\mathcal{F}$.

$L$ is continuous at one point of $\mathcal{F}$.

Proof
Equivalence of these conditions can be proved by proving $1\Rightarrow2,2\Rightarrow3$
and $3\Rightarrow1$.

Firstly we prove $1\Rightarrow2$. Assume $L$ is bounded. Then by
definition of a bounded operator, for all $f\in\mathcal{F}$, ,
where $|L|$ is the operator norm. Set $f:=f_{1}-f_{2}$. So, we
have
which is the definition of Lipschitz continuity with Lipschitz constant
$|L|$. Since Lipschitz continuity implies continuity, $1\Rightarrow2$
holds.

The implication from 2 to 3 is obvious since by definition an operator
is continuous if it is continuous at every point.

Proving $3\Rightarrow1$ is a bit tricky. Let us recall the definition
of continuity. An operator is said to be continuous at $f_{0}\in\mathcal{F}$
if for all $\epsilon>0$, there exists $\delta(\epsilon,f_{0})>0$
such that for all $f\in\mathcal{F}$, we have

Assume $L$ is continuous at $f_{0}$. Then, for $\epsilon=1$, there
exists $\delta>0$, such that
for any $\Delta$ implies
.
Here $f$ in the definition of continuity is set to $f_{0}+\Delta$,
and $\epsilon$ is chosen to be 1. Now we try to show that $L$ is
bounded.

Here we used the fact that $L$ is linear. Since
,
if we set , then we have
$|\Delta|_{\mathcal{F}}\leq\delta$. So,

where we used the fact that $\left\Vert L\Delta\right\Vert \leq1$.
Since $f$ is arbitrary, we have the definition of a bounded operator
with operator norm $|L|=\frac{1}{\delta}$.