If you were using the bucket fill on a transparent layer (except the white), then

- don't use a selection (or use a selection that includes the existing antialiasing pixels)
- bucket fill the grey after putting the bucket-fill tool in "Behind" mode.

Another (completely equivalent) solution:
- add a layer of the same size under the first
- bucket-fill that whole layer with gray
- merge the layers if strictly necessary (if yo uleave them distinct, you can change the colors later)

If you are trying to paint over an existing color, then use color-to-alpha (or bucket fill with that color in color-erase mode) to make that color transparent, then see above.

I didn't realize it was that simple of a solution, I completely understand about how it makes it look a lot better.