There is a problem on Page 60 of Book of B.M.Budak, A. A. Samarskii and A. N. Tikhonov, whose title is A Collection of Problems in Mathematical Physics (New York, Dover, 1964). The problem (the problem number is 64) is: Applying the Fourier transform with the kernel $$K(x,\lambda)=\sqrt{\frac{2}{\pi}}\frac{\lambda \cos (\lambda x)+h\sin(\lambda x)}{\lambda^2+h^2},$$
solve the boundary-value problem

Apparently, this transform is just the joint sine and cosine transform. But I have never seen this kind of Fourier transform before, and therefore I do not how to use this kind of integral transform to solve the problem here, although I've tried several days. And I do not even know what is its inverse transform:

If we denote the image of $f(x)$ of this transform by $\hat u(\lambda)=\int_0^{+\infty} K(x,\lambda)u(x)dx,$ is it true that $u(x)=\int_0^{+\infty} K(x,\lambda)\hat{u}(\lambda)d\lambda$ ?

I could not find any other books about this kind of Fourier transform. Can anyone help me solve this problem? or just give me some hints on the properties of the Fourier transform with the above kernel $K(x,\lambda)$?

hmm, there is no square root in the denominator? if there was a square root, then we could define $\alpha=\arccos(\lambda/\sqrt{\lambda^2+h^2})$ and your kernel would simply be $K(x,\lambda)=\sqrt{2/\pi}\cos(\lambda x-\alpha)$
–
Carlo BeenakkerMay 11 '13 at 19:36

There is indeed no square root in the denominator of the kernel $K(x,\lambda)$. I have tried the method just as you thought, but the result is no use.
–
azhiMay 12 '13 at 0:51