Is it considered "weird" to remember certain details?

Sometimes I'll say something that people [apparently] don't expect someone to remember, and I feel like it makes me seem creepy, as though it's something I've been dwelling on. Is it somehow weird to pay attention when someone talks?