I am not sure if I understand you correctly, but if you have situation, where you fill the variable of My structure with data on host(your program using OpenCL) and then you pass this struct (or pointer to it) via buffer to kernel like this:

__kernel void myKernel(__global My* myVar)
{
...
}

...then the pointer members of My structure (ps1, ps2) will still contain address which points somewhere to host memory, while the kernel is (I presume) executed on GPU, in completely different address space.

I remember getting myself into similar trap, resulting in various side effects like freezing up the PC.