Yes, Perl is correctly installed in this context. And all the other IM functions I use are working fine.
Of those it's only Annotate that gives this strange behaviour.
To give some more info, I Read images, use Blur, Resize and use ImageToBlob which all work fine.

I did read the PerlMagick pages, but no hints to solve my problem.

I feel like it's a bug, because it occured after a release upgrade of Ubuntu, before that, the same Annotate function worked fine.
For now I leave it to the workaround with convert and see if the problem is gone when a new version of PerlMagick is released.