Jindo, South Korea (CNN) -- Passengers aboard a sinking South Korean ferry faced a terrifying choice as the vessel rolled: obey commands barked over loudspeakers to stay in place, or don life vests and jump into the chilly ocean water.

That announcement, some witnesses worried, may have cost some passengers on the ferry Sewol their lives.

"Kids were forced to stay put," one survivor told CNN affiliate YTN, "so only some of those who moved survived."

As rescuers searched the frigid water for nearly 300 people who remained unaccounted for, witnesses told horrific stories of their final minutes on the sinking ship.

"We were told to stay where you are, so we kept staying," survivor Hyun Hung Chang told YTN. "But later on, the water level came up. So we were beside ourselves. Kids were screaming out of terror, shouting for help."

At least some of those who jumped or made their way to the top of the ship were rescued. Helicopter crews plucked some from the deck.

Others were pulled from the water by crews aboard the multitude of fishing boats and military vessels that raced to the scene as the ship rolled over and capsized.

"I had to swim a bit to get to the boat to be rescued," Lim Hyung Min, one of more than 300 Seoul high school students who was aboard the ship for a four-day field trip, told CNN affiliate YTN. "The water was so cold and I wanted to live."

CNN affiliate YTN cited South Korean emergency response officials in reporting that six people were dead. The semiofficial Yonhap News Agency was still reporting four deaths early Thursday, mirroring what officials had told CNN on Wednesday.

At least 164 had been rescued, although some media reports put the number at as many as 179.

A massive rescue operation resumed early Thursday after a delay of several hours, the South Korean Coast Guard told CNN. The reason for the delay was unclear.

Dozens of military divers, sailors, marines and police officers were assisting in the effort.

But cold water, swift currents and low visibility appeared to be complicating the rescue operation.

Water temperatures in the area are between 50 and 55 degrees Fahrenheit (about 10 to 13 degrees Celsius), CNN meteorologist Brandon Miller said.

Divers from the South Korean navy searched three of the ship's compartments but found no survivors or bodies, Yonhap reported.

The U.S. Navy ship USS Bonhomme Richard, on routine patrol in the area, diverted to the scene and was standing by in case South Korean officials ask for help, said Lt. Arlo Abrahamson, spokesman for the U.S. Navy in South Korea.

"The Republic of Korea has done a great job in their rescue efforts thus far," he said.

Rescuers are "up against every sort of obstacle," said David Gallo, director of special projects at the Woods Hole Oceanographic Institution.

It began as a routine ocean trip on calm seas. Passengers bound for a resort island were just stirring, some eating breakfast.

The ship ran into trouble just before 9 a.m. as it steamed toward to Jeju, a resort island considered the Hawaii of Korea.

Passenger Kim Sung-Mook told YTN that he was eating breakfast in the ship's main hall when he felt the ferry begin to tilt.

Someone made the announcement telling passengers to stay in place. Then, he said, he heard a loud bang.

Lim, the rescued student, told YTN he heard the bang before the ship began to list. The tremors knocked shipping containers off balance, he said.

"The students were falling over and crashing into things and bleeding," Lim said.

Many passengers screamed.

He obeyed orders to stay on the ship until rescuers arrived, threw him a life jacket and told him to jump.

The water, he said, was "unbearably cold."

At first, survivor Ji Chul Song said she listened to orders to stay put.

"But suddenly, the water came up to my face," she told YTN. "So I think it was a narrow escape from dying."

Massive damage?

What caused the ship to sink is unknown. It capsized within two hours of its first distress call, which came just before 9 a.m., Yonhap reported

Peter Boynton, a retired U.S. Coast Guard captain, said the speed with which the ship sank suggested it had sustained "major damage."

It most likely struck something in the water, Boynton said.

Making matters worse, the ferry carried dozens of vehicles. Once an auto deck is breached, "it's typically open to very significant flooding," Boynton said. That could explain "why the ferry in just a matter of hours began to roll onto its side so quickly."