Taro Aso referred to elderly patients who are no longer able to feed themselves as 'tube people'. Photograph: Yoshikazu Tsuno/AFP/Getty Images

Japan's new government is barely a month old, and already one of its most senior members has insulted tens of millions of voters by suggesting that the elderly are an unnecessary drain on the country's finances.

Taro Aso, the finance minister, said on Monday that the elderly should be allowed to "hurry up and die" to relieve pressure on the state to pay for their medical care.

"Heaven forbid if you are forced to live on when you want to die. I would wake up feeling increasingly bad knowing that [treatment] was all being paid for by the government," he said during a meeting of the national council on social security reforms. "The problem won't be solved unless you let them hurry up and die."

Aso's comments are likely to cause offence in Japan, where almost a quarter of the 128 million population is aged over 60. The proportion is forecast to rise to 40% over the next 50 years.

The remarks are also an unwelcome distraction for the new prime minister, Shinzo Abe, whose first period as Japan's leader ended with his resignation after just a year, in 2007, partly due to a string of gaffes by members of his cabinet.

The 72-year-old, who doubles as deputy prime minister, said he would refuse end-of-life care. "I don't need that kind of care," he said in comments quoted by local media, adding that he had written a note instructing his family to deny him life-prolonging medical treatment.

To compound the insult, he referred to elderly patients who are no longer able to feed themselves as "tube people". The health and welfare ministry, he added, was "well aware that it costs several tens of millions of yen" a month to treat a single patient in the final stages of life.

Cost aside, caring for the elderly is a major challenge for Japan's stretched social services. According to a report this week, the number of households receiving welfare, which include family members aged 65 or over, stood at more than 678,000, or about 40% of the total. The country is also tackling a rise in the number of people who die alone, most of whom are elderly. In 2010, 4.6 million elderly people lived alone, and the number who died at home soared 61% between 2003 and 2010, from 1,364 to 2,194, according to the bureau of social welfare and public health in Tokyo.

The government is planning to reduce welfare expenditure in its next budget, due to go into force this April, with details of the cuts expected within days.

Aso, who has a propensity for verbal blunders, later attempted to clarify his comments. He acknowledged his language had been "inappropriate" in a public forum and insisted he was talking only about his personal preference.

"I said what I personally believe, not what the end-of-life medical care system should be," he told reporters. "It is important that you be able spend the final days of your life peacefully."

It is not the first time Aso, one of Japan's wealthiest politicians, has questioned the state's duty towards its large elderly population. In 2008, while serving as prime minister, he described "doddering" pensioners as tax burdens who should take better care of their health.

"I see people aged 67 or 68 at class reunions who dodder around and are constantly going to the doctor," he said at a meeting of economists. "Why should I have to pay for people who just eat and drink and make no effort? I walk every day and do other things, but I'm paying more in taxes."

In 2001, he said he wanted Japan to become the kind of successful country in which "the richest Jews would want to live".

He once likened an opposition party to the Nazis, praised Japan's colonial rule in Taiwan and, as foreign minister, told US diplomats they would never be trusted in Middle East peace negotiations because they have "blue eyes and blond hair".

While figures released on Monday showed a record 2.14 million Japanese were receiving welfare in October 2012, Aso has led a life of privilege few of his compatriots could hope to match.

He is the grandson of Shigeru Yoshida, an influential postwar prime minister, and is married to the daughter of another former premier.