Humans are annoyed when they are exposed to environmental noise. Traditional measures such as sound pressure levels may not correlate well with how humans perceive annoyance, therefore it is important to investigate psychoacoustic metrics that may correlate better with the perceived annoyance of environmental noise than the A-weighted equivalent sound pressure level. This study examined whether the use of binaural recordings of sound events improves the correlation between the objective metrics and the perceived annoyance, particularly for road traffic noise. Metrics based on measurement with a single microphone and on binaural sound field recordings have been examined and compared. In order to acquire data for the subjective perception of annoyance, a series of listening tests has been carried out. It is concluded that binaural loudness metrics from binaural recordings are better correlated with the subjective annoyance assessment.