This happens many time with RPF using Z-depth. AFAIK The problem is because of nasty edges in alpha channel.
I do have a separate Zdepth layer but I am not familiar wth the workflow
Precompose the whole scene, then apply Lens Blur effect and Choose your Depth Map in Depth Map channel instead of Luminance.