Two children named Gracie Jenkins and Daniel Pender are best friends living on the island of Bryher. They enjoy sailing wooden boats that they make off the numerous beaches, but the tide carries their favorite one across to a forbidden bay. This is the home of a local hermit nicknamed the Birdman, whom all the children are forbidden to go near and whom none of the adults trust. This is partly due to him being born on a nearby island named Samson which all locals avoid, as they say it is cursed. On stormy nights he goes over to Samson and can be seen sending lantern signals out to sea.

Daring to go to the Birdman's beach, Daniel and Gracie find the missing boat on the sand with a message written in stones, left by the hermit. Daniel leaves a further message saying thank you and tells him their names. Later the Birdman leaves them a beautiful carving of a seabird and the children hide it at Gracie's house.

Life on Bryher is hard and very rustic. The villagers survive on fishing and growing their own produce. Daniel's father is a violent man who has to work extra hard in order to feed his large family of seven children. Gracie's parents, Jack and Clem, are kinder; Gracie is an only child as their parents have not been able to have more children. However, her father doesn't see school as important for Gracie in their lives as fishermen, while her mother wants her to have an education.

The local school is on a nearby island and ruled by Mr Welbeloved, who struggles to get the rustic students to work. He is worried over the coming war and warns children to be vigilant against any who might be sending messages to enemies.

Daniel and Gracie soon meet the Birdman in person, discover that he is kind, gentle and profoundly deaf. They begin a secret friendship. Born as Mr Woodcock on another nearby island named Samson, he tells the children that when he was a small boy, a group of NarWhales were beached and slaughtered by the islanders for their valuable horns. The NarWhales cursed them and the boat shipping the islanders and horns to the mainland was sunk, the Birdman's father among them. Next illness and death struck the island, then the plants began to fail, chickens stopped laying and the islanders left all except the Birdman and his mother. Mrs Woodcock had sworn she would not leave her husband's ghost alone, but when the Well runs dry, she is forced to leave. Before they do, the curse claims the Birdman's hearing, making him deaf.

Daniel asks the Birdman to teach him to carve, and soon begins to become expert.

One day the Birdman tells the children about a large amount of valuable timber which has washed up on one of the island's shores. The islanders all gather to collect and hide this timber before the coastal authorities arrive and try to claim it for themselves. While searching the island they ask to see in Gracie's attic, where they find nothing except the carved sea bird that the Birdman gave Gracie.

Jack asks where it came from and Gracie insists that Daniel made it. When Daniel shows several of his new carvings, Jack apologises and the Seabird is proudly displayed in their home.

The war begins and Gracie's father decides to go and fight in the Navy; as an experienced sailor, he feels it is his duty. The only man to go from Bryher, it leaves Gracie's mother in charge of feeding the two of them. Clem is unable to catch enough fish so Daniel and Gracie take the boat out themselves. Getting lost in a thick fog they accidentally land on Samson, discovering the dry well, the graves of the old islanders and one of the Narwhale horns. When they arrive home, Daniel's father is furious as he insists that they have brought a curse back with them.

Days later a telegram comes saying that Gracie's father has been lost at sea and is presumed dead. Gracie sees it as the Samson curse and blames herself. The islanders gather around to comfort Clem and Gracie and the Birdman delivers gifts on produce, fish, honey and eggs. Clem suspects that the Birdman has been talking to the children but leaves well alone.

After a great storm, Daniel, Gracie and the Birdman discover a beached NarWhale and attempt to get it back into the sea, but it is too heavy. The other Narwhales gather in the bay and the Birdman knows that soon they will come to the beach themselves and then the curse of Samson will begin again on Bryher.

Daniel's oldest brother and some local boys set fire to the Birdman's cottage because they think he is signalling to the enemy- he is actually stopping ships from getting beached on the dangerous rocks or getting caught by the curse. They see the Narwhales and go back to the village. Soon the entire island comes to slaughter the whales but Daniel insists that they listen to the Birdman. Mr Woodcock finally tells them the true story of Samson and the islanders work together to get the whales back into the sea and away from the island.

Daniel leaves the Birdman watching the sea but returning in the morning he finds the Birdman gone, never to be seen again. He takes a boat over to Samson and discovers that the Well is full again and that plants have started to grow.

Later that day, a naval boat arrives with Jack on board, found at last and not drowned. The curse is lifted by the islanders and Gracie, Jack and Clem return home celebrating their experiences.