I think you would have to not move the VST window, and then crop a display capture or window capture to just capture this part of the screen. You can crop by SHIFT+ALT+drag the red circles handles when you select a source.