The home, called Stonor Lodge, had been undergoing a major renovation.

Joel Macedo, whose mother lives nearby, was one of the first people to the scene.

"It was something I've never seen before. It looked like it was out of a movie. It seemed like in seconds it just went down," Macedo told NBC 10 News.

One construction worker was still on site when the fire started at about 4:30 p.m.

Patrick Keith ran out from his home after seeing smoke.

"It was just pandemonium. People were freaking out, going crazy thinking their house was going to catch fire. And I saw the worker coming out. Have was on the phone with 911. I was, 'Are you OK?' He was fine. He was a little shooken up and he said he just heard a big bang of some sort," Keith said.

The Stonor Lodge, which tax records show had nine bedrooms and 11 bathrooms, was in the midst of a significant construction project. More workers had been there earlier in the day.

The cause of the fire is not known.

"We're not sure. It was called in, I believe by the constructions workers. No injuries. Everyone got out safely," Newport Fire Department Capt. Stephen Knapman said.

The work was being done by Kirby Perkins Construction, which is known for major home building and estate renovation projects in the area.

Owner Jerry Kirby told NBC 10 News that his workers don't know what started the fire and the workers on the crew "feel sick" about it but they are glad everyone is OK.

Now, the house is gone.

The fire brought back memories for Keith, whose friend grew up at the home.

"Spent a lot of time horsing around as kids in there. It's just sad to see it go," he said.

The current owners of the home, who bought it a few years ago, are from New York.