The history of the village can be traced as far back as the Roman times based on a site containing a pottery kiln dated around the third century.

According to the Oxford Dictionary of English Place-Names the origin of Elmswell or in its Old English form Elmswella, as referred to in the Domesday Book of 1086 comes from 'Spring or Stream where elm-trees grow. The place-name Elmswella is formed by the conjunction of elm + wella, where wella is Old English for stream.

A huge village green – Butten Haugh Green – once formed the centre of Elmswell. However, the arrival of the railway in 1846 and the bacon factory in 1911, meant the green now has houses built on it.

The 1881 census showed that the number of dwellings in the village was 196 and the population was 761.

The village has had a railway station since 1846, on the line between Bury St Edmunds and Ipswich.
There is a wildlife area at Kiln Meadow, and two pubs – The Tavern and The Fox - exist in the village.

Elmswell was named Suffolk's village of the year in 2008 beating Cockfield and Fressingfield into second and third place respectively.