The Holy Prophet sent everybody out in his last moments and had confidential talk with Imam Ali. Please also check in Tirmidhi the Holy Prophet conducted secret and confidential talk with Imam Ali on one more occasion at the day of Taa’if.