In your audio example the pitch shift is modulated by hand or using automation so that it's shifting sometimes upwards and sometimes downwards.

I had a quick look at kvraudio.com which has a very good overview of free vst effects. However I did not find a vst doing this effect out of the box. But I really did not search for very long. You might want to look some more yourself..

I have myself created a vst effect doing exactly this. It is an enhanced version of the Echolive VST from my vst page (http://www.interruptor.ch/vst_overview.shtml) which was never released as I just built it for my own purpose.
If you are interested I can dig it up for you.

As discussed by private message here are two screenshots of my own plug which I modified for these "wandering pitch" delays.

One is the setting for upward shifting and one for downward shifting.
I also uploaded a small demo of what it sounds like. First a clean vocal snippet from your file, then an example of each setting shown in the screen shots..

I think the pitch shifting sounds grainier than in your sound file. Probably because I used a freely available pitch shift algorithm and did not spend much time on fine tuning this special version of the plug-in.
The pot for the feedback pitch is on the far left of the GUI. As mentioned this is an inofficial version of the plug which I created for myself, that's also why the two additional pots are simply "glued" on top of the user interface and do not look polished at all..