White spot on tonsil

Patient: Hi, I have a white spot on my tonsil, its been swabbed and it not bacterial. Its been there for a month, its also not tonsil stones as I used to get them and know what they are. Sometimes that tonsil hurts, the spot is largish and has been there for months. Doctors didnt seem overly concerned but it is really freaking my out, wont come off with gargling or toothpick. What could it be?? Could it be connected to allergies??