Now, atheists are pushing to get their literature in the cabins too. The non-profit group American Atheists say they're taking Governor Deal up on his offer for any religious groups to donate literature to be placed in bedside drawers in Georgia's state park cabins.

"I didn't like that they were taking the Bibles out," Cloudland Canyon State Park visitor Calvin Johns said.

Calvin and Clara Johns are staying in a cabin at Cloudland Canyon State Park near Trenton. Though the Gideon Bibles have not yet been returned to their cabin, officials with the Georgia Department of Natural Resources say they started the process of putting them back last week, after Governor Deal ordered them to. He said since they were not paid for by the state, it doesn't constitute as a state establishment of religion.

"In a state park cabin? That cabin is owned by me. It's owned by every Georgia citizen and tax payer," Ed Buckner said.

Buckner, an atheist, says he was shocked to find nine Gideon Bibles in the three bedroom cabin he rented at Amicalola Falls State Park a few week ago. He filed the complaint that sparked controversy after the GDNR agreed to remove the bibles from all cabins while they reviewed the situation.

"I'm not attacking Christianity, I'm attacking government's apparent endorsement of any religious point of view, even atheist point of view," Buckner said.

Governor Deal did invite any religious groups to also donate literature to the parks.

"I think he's opened Pandora's box," Buckner said.

"If he insists on having Bibles in state owned cabins, then of course, we want our material alongside them," American Atheists spokesperson Dave Muscato said.

American Atheists says its working with the Georgia DNR to coordinate the distribution of four atheist books: "Why I Am An Atheist," by Madalyn Murray O'Hair, "god Is Not Great," by Christopher Hitchens, "Why I Am Not Muslim," by Ibn Warraq, and "The GOD Delusion," by Richard Dawins.

Several park visitors told Channel 3 that they're fine with that, even though they believe in the Bible.

"They have rights just like anybody else does, I think they should," Calvin Johns said.

Georgia state park officials tell Channel 3 no other religious groups have contacted them over donating literature yet. They say they're in the process of writing a policy for interested groups on how to go about it, outlining details like how to deliver the books.