I am trying to calculate $$\int_{\pi^2}^{4\pi^2} \frac{\cos(\sqrt{x})}{\sqrt{x}} \,dx.$$
I calculated the integral and got $2\sin(\sqrt{x})$ as a result, but for $x=\pi^2$ and $x=4\pi^2$ we get that $2\sin(\sqrt{\pi^2})=0$ and $2\sin(\sqrt{4\pi^2})=0$ So the Riemann integral will be $0-0=0$ which is not true, as you can see from ploting $2\sin(\sqrt{x})$.

I simply changed the bounds of integration, so there's no need to "back-substitute". So it seems, as you proceeded in your evaluation of the definite integral, that your answer is indeed correct.

See, e.g. Wolfram|Alpha's computation:

$\quad\quad\quad\quad$

Visual representation of the integral:

*If we are looking to calculate the area between the x-axis and the curve, then we need to split the integral to compute the area below the x-axis, and the area above the x-axis: for $u$, the dividing point will be $\large\frac{3\pi}{2}$ (for $x$: $\large\frac{9\pi}{4}).$

You are right. Here is a slightly different approach, if you want.
Using the substitution $u=\sqrt{x}$, we have $du=\frac{1}{2\sqrt{x}}dx$ so the integral becomes
$$
\int_\pi^{2\pi} 2\cos u du=2\sin (2\pi)-2\sin(\pi)=0 .
$$

Although @julien got the right fact about the definite integral, we should find a positive value for that. I mean we should consider the following integrals as well to find a real area: $$
\Big|\int_\pi^{3\pi/2}\Big|\;\;+\;\;\Big|\int_{3\pi/2}^{2\pi}\Big|
$$