I would have though the common pronunciation is indistinguishable between 'of' and 'have': [əv] is the normal unstressed form of both words. 'Have' has an alternative, less common unstressed form [həv], particularly in England, but this sounds very unnatural to me after another modal like this.

The test is in those situations where modals assume their full forms, that is finally with the rest of the verb phrase omitted: 'They said I won't make it, but I will'; 'You think I can't win, but I can', etc. Problem is, 'should[əv]' is usually repeated as a whole: 'You shouldn't have done that, I should've.' So this wouldn't usually tease out whether the word was an underlying 'of' or 'have'. That said, I have sometimes heard people use a full-form 'of' here.

This is one of those where the misspelling more accurately reflects common (I would say accepted) pronunciation than the correct spelling. That doesn't mean it should be misspelled; it just can't be counted as a mispronunciation.