Prince Harry has been greeted by huge crowds in Sydney on his first official visit to Australia – but also had to endure a gentle ribbing from the country's new prime minister, Tony Abbott, about Australian republicanism.

The prince was representing the Queen at the International Fleet Review, which features around 40 warships from 17 countries.

Harry arrived in the city on Friday night to start his two-day visit. On Saturday morning he was greeted by a crowd of hundreds as he boarded the survey ship HMAS Leeuwin at the city's Garden Island naval base before inspecting the flotilla.

The prince wore a white British Army tropical dress uniform with the sky blue beret of the Army Air Corps as he took the royal salute. He was joined on board by Abbott and the governor general, Quentin Bryce, the Queen's representative in Australia.

Abbott braved the issue of Australian republicanism during a speech at Kirribilli House, one of his official residences.

In front of Harry and an audience which included disabled Australian war veterans, Paralympians and young entrepreneurs, he said: "Prince Harry, I regret to say not every Australian is a monarchist. But today everyone feels like a monarchist.

"You grace us as your family has graced our nation from its beginning, as the crown is a symbol of our stability, continuity and decency in public life."

Arriving at Kirribilli amongst a host of boats and circling helicopters, Harry was greeted by Abbott, his wife, Margie, and two of his three daughters, Frances and Bridget. He then chatted with members of the public who had gathered at the gates of the house before heading inside.

Harry asked one: "What do you think of your new neighbour?", in reference to Abbott's recent general election win.

Earlier, thousands of people turned out to greet the prince as he went on a harbourside walkabout. There were screams from those at Campbell's Cove, in the shadow of the Sydney Harbour Bridge, as Harry chatted, shook hands and posed for photos before boarding a small boat to cross the harbour for the reception at Kirribilli, on the north side of the bay.

Among the crowd were naval trainee medics Ayesha Sweeney, 19, and Renee Simmons, 21, who live six hours' drive away at their base in Aubrey, New South Wales.

Simmons described Harry as gorgeous: "I'm in love. He wears a uniform very well."

Sweeney said: "It was good to get a royal visit today for the review, it adds that extra excitement to it."

Sydneysider Meagan Roberton, 50, who went to the street outside the base with her family, said: "We didn't expect to see him today so it's an added bonus. He's gorgeous, we all love him in Australia. I remember when he was born."

The review is being held to commemorate 100 years since the Royal Australian Navy's first modern warships sailed into Sydney, seen as a defining moment in the country's maritime history.

The RAN spokesman Commodore Paul Kingshorne said the force was pleased Harry had come to mark the event.

"It is his first official visit here representing the Queen," he said. "The fact he is here on such a significant day for the navy and for Australia is fantastic for us."

The 29-year-old is one of the younger royals being looked at to increasingly take on the more long-haul parts of the monarchy's workload. He last visited Australia in 2003, and leaves the country on Sunday to attend a charity event in Dubai.