Last part solved. When creating groups don’t add for which roles as this part will feed user setup. If you already have created groups and users just go back to each group and tick off all roles and all users has now been ticked off from usergroups and can be manually ticked in for which groups they should have access or not. This part can only be manage if you have few users and not several 100s….