The situation and the believability of the claim. (mods from the way that bluff works)

The ability of the recipient to discern truth from lies (the sense motive mods of the recipient)

Random variance (the d20 rolls).

Note that this can also go any which way: When the players roll their sense motive checks on what some person is saying, then depending on whether the person is telling the truth, just apply the mods the other way or not. They cannot even rely on whether the d20 roll is high - it just determines how suspicious they are. (They can, however, do the math and logic thing to figure it out, but that's a bit complicated.)