Private forum with every user using group function possible?

I have a wordpress site and would like to interact with my users – however, I would like this interaction to be private from every other user on the forum. Ideally I envision a part of the forum that is for all registered users, but a separate section where I can interact with an individual – send files, chat etc.

Can this be done by using the group function – where I make a group of myself and the other individual and then keep this group private to myself and the individual? Or is this likely to become unsustainable for myself – having to keep track of what is being written on all the groups I have created?

Any help would really be appreciated! Maybe I am over complicating it if bbPress can have a secure messaging system that copes with attachments. Thanks