That's actually a good point to bring up I think it was that Oly sent them the notes and said they had business @ the asylum. So when they went there they got ambushed by Loboto. Then their brains were taken. And that would wrap it up.

Oleander DID tell them they all had official Psychonauts business. They were kidnapped by Linda, who took them to Loboto and yadayadayada.