Am trying to get external user integrated with our Azure AD and Tableau. For internal users it is working fine but when we have external users, we some times get an error that is 'xxxx@...com is not assigned to a role for the application 'xxxx'

Sequence of how to reproduce the error is as follows

1) Enter the tableau url

2) It routes to our Azure AD

3) I enter my user name and password

4) I get the above error

Is there any configuration on the Azure AD side that we should be aware of when configuring Tableau and Azure AD?