The reflective person in me says you're embodying a fantasy they wish they could participate in but don't have the courage or desire to enact. I used to spend a lot of time online talking to fetishists who had the same issue ("my wife doesn't understand.. you're wonderful and open.. but I'd never leave her, though..")

My advice is to take it as a compliment to your attractiveness as a sexually aware and liberated woman, to politely decline, and to move on. There are a lot of wonderful poly men out there who'll love you for who you are instead of who they idealize you to be.