Consider, you have a problem in an infinite dimensional Hilbert or Banach space (think of a PDE or an optimization problem in such a space) and you have an algorithm that converges weakly to a solution. If you discretize the problem and apply the corresponding discretized algorithm to the problem, then weak convergence is convergence in every coordinate and hence also strong. My question is:

Does this kind of strong convergence feel or look any different from convergence obtained from good old plain strong convergence of the original infinite algorithm?

Or, more concrete:

What kind of bad behaviour can happen with a "discretized weakly convergence method"?

I myself are usually not quite happy when I can only prove weak convergence but up to now I could not observe some problem with the outcome of the methods even if I scale the problem discretized problems to higher dimensions.

Note that I am not interested in the "first discretize than optimize" vs. "first optimize than discretize" problem and I am aware of problems that can occur if you apply an algorithm to a discretized problem that does not share all properties with the problem for which the algorithm was designed for.

Update: As a concrete example consider an optimization problem with a variable in $L^2$ and solving it with something like (an inertial) forward-backward splitting or some other method for which only weak convergence in $L^2$ is known. For the discretized problem you can use the same method and with the correct discretization you get the same algorithm is if you discretized the algorithm directly. What can go wrong when you increase the discretization accuracy?

$\begingroup$What kind of method are you thinking of where the convergence is analyzed before the infinite-dimensional problem is discretized? You mention optimization, so are you thinking of PDE-constrained optimization problems, mostly, or is there something else?$\endgroup$
– Bill BarthJun 5 '15 at 23:01

2 Answers
2

It is true that weak convergence is most crucial in the continuum limit as $h\to 0$ (e.g., by not being able to observe any convergence rate). At least in Hilbert spaces, it is also closely tied to non-uniqueness of the limit and hence only subsequential convergence (e.g., where you can alternate between approaching different limit points, again destroying rates), and it is difficult to separate the influence of the two on the convergence.

Specifically for weak convergence in $L^2$, you also have the fact that the convergence need not be pointwise, and this you can actually observe in a (sufficiently fine) discretization. Here's an example from a sequence of minimizers $\{u_\varepsilon\}_{\varepsilon>0}$ that converges as $\varepsilon\to 0$ to
$$ u(x) = \begin{cases} -1 & x<\frac13 \\ 0 &x\in [\frac13,\frac23] \\ 1 &x>\frac23\end{cases}$$
where the convergence is weak but not pointwise on $[\frac13,\frac23]$ (but pointwise almost everywhere else). The following figures show three representative elements from the sequence (for $\varepsilon$ already quite small).

This phenomenon is known as "chittering" in the approximation of bang-bang control problems for differential equations (i.e., problems with box constraints where the solution almost everywhere attains either the lower or upper bound).

$\begingroup$Excellent example! However, I did not get the point how weak convergence is tied to non-uniqueness. In general one can not upgrade weak convergence to strong convergence when the limit is unique, right? But agree, frequently one has both only weak convergence and non-uniqueness.$\endgroup$
– DirkJun 9 '15 at 6:47

$\begingroup$Sorry, that was poorly phrased; I didn't mean that this is always the case. I had in mind problems where you usually get convergence of the norm as well, so as long as you have convergence of the full sequence, you can "upgrade" to strong convergence (i.e., the only thing that can prevent strong convergence is subsequential convergence).$\endgroup$
– Christian ClasonJun 9 '15 at 15:02

The question you ask is often of not much practical concern because weak convergence in one norm may imply strong convergence in another one, for the same sequence of solutions.

To give you one example, let's assume we solve the Laplace equation with a sufficiently smooth right hand side on a convex polygonal domain with standard finite elements. Then the solution $u$ is in $H^2$, but of course the finite element solution $u_h$ is only in $H^1$. We do know that $u_h \rightarrow u$ strongly in both the $L_2$ and $H^1$ norms as the maximal mesh size $h\rightarrow 0$ because we have the a priori error estimates $\|u-u_h\|_{L_2} \le Ch^2$ and $\|u-u_h\|_{H^1} \le Ch$.

But clearly we cannot expect $u_h \rightarrow u$ strongly in $H^2$ because the $u_h$ are only in $H^1$. But we might have $u_h \rightharpoonup u$ weakly in $H^2$ (in fact, I think that that holds). This would then probably imply a statement such as
$$
\left< \nabla^2 (u-u_h), \nabla^2 v\right> \le {\cal o}(1) \qquad \forall v \in H^2.
$$

The point is that the question of weak vs strong convergence is typically a question of what norm you look at, and not a property of the sequence of solutions you get from your method.

$\begingroup$This is true, but at some point the norm becomes too weak to be practically useful (for example, when you only have weak convergence in $L^2$, which might imply strong convergence in negative Sobolev norms, which are not localizable).$\endgroup$
– Christian ClasonJun 6 '15 at 22:47

$\begingroup$@ChristianClason, can you speak to what this is like when such a method is discretized. Do they work? Etc?$\endgroup$
– Bill BarthJun 7 '15 at 2:56

$\begingroup$The case I had in mind is when the discretized norm actually approximates the norm in which only weak convergence happens (usually $L^2$).$\endgroup$
– DirkJun 9 '15 at 6:49