The recession is at the root of most of these high small business failure rates, Vielehr said.

The downturn hurt Colorado's tourism businesses and hit Tennessee's manufacturing industry badly, Vielehr said. In fact, Tennessee has been among the five states with the worst failure rates since 2007, according to the report.

On the other side of the coin,North Dakota, Vermont, Iowa, Wyoming, and Kansas have the lowest rate of small business failures.

North Dakota had a 67% lower failure rate than the national average, followed by Vermont with 47%, Wyoming and Iowa (40%) and Kansas (39%).

North Dakota, Wyoming and Iowa have all been in the top five for the past four years.

These rural states did not have the economic boom that other areas of the country had, so the fall during the recession was not so stark, said Alla Kramskaia, senior director of statistical consulting at Dun & Bradstreet Global Analytics.

North Dakota and Wyoming's economies were also supported by their rich energy resources, said Kramskaia.