Well as far as the game is concerned, it's not explained why The Hunt exchanged Yennefer for Geralt. I guess we're supposed to assume they valued him more for some reason. The amnesia is also never explained, and neither is Geralt escaping/released by The Hunt.

Afaik, the plots of both Witcher games are unique, and not directly related to any of the books.

I know that I was disappointed with the revelation of what/who The Hunt actually are though. I was expecting something more interesting than elves from another dimension.