I have noticed an uptick in bee activity recently. I have walked past this nest every day not knowing it was just a foot away.
Will the wasps smell the fear on me now that I know where they are? There might be 2,000 of them in there.

However, if you do not find the wasps' nest until late summer or fall, destroying the nest may not be necessary (depending on your location) as freezing nighttime temperatures will kill the colony off naturally. Then you can simply dispose of the nest come winter.