At least 26 states, including California, Florida, Illinois and New York, are rolling out mandatory programs that put billions of public dollars into privately managed long-term care plans, in hopes of keeping people in their homes longer, and expanding alternatives to nursing homes.

Subway advertisements and highway billboards feature smiling old people as plans jockey for shares of this vast new market. Companies promise profits for investors and taxpayer savings, too. And some states say the new system is already working.

“It’s a success story,” said Patti Killingsworth, director of long-term services and supports in Tennessee, pointing out that the state was serving a quarter more people with inexpensive home and community services.

But a closer look at Tennessee, widely cited as a model, reveals hidden pitfalls as the system of caring for the frail comes under the twin pressures of cost containment and profit motive. In many cases, care was denied after needs grew costlier — including care that people would have received under the old system.

Like many advocates, he originally supported managed long-term care, seeing it as a way to break the stranglehold of nursing home lobbies that opposed shifting more Medicaid money to home and community-based care. But now he says too high a price is being paid by very debilitated people denied care when they need it most — people like Billy Scarlett II, who was 33 in 2005 when he sustained severe brain injuries in an A.T.V. accident, and Glenn McClanahan, who is 79.

Mr. McClanahan’s case illustrates both the appeal and the perils of the new system. Once a high school quarterback, a successful car salesman and a ladies’ man, he was living alone on Social Security, already hobbled by arthritis and emphysema, when at 75 he abruptly lost nearly all of his sight. For years, Tennessee residents like him had to move to nursing homes, with Medicaid paying the bills from a mix of state and federal money.

But in 2010 the new program gave Mr. McClanahan another choice: Stay at home with daily help, and go to a nursing home later if he needed it. Medicaid paid a fixed monthly sum to an insurance company to cover and coordinate his future care. For about 30 months, Mr. McClanahan was happy to manage at home with four hours of help daily. The government and the insurance company benefited, too, because his care cost much less than the monthly Medicaid sum paid to the plan — $3,820, which was less than the $4,583 a nursing home would have cost.

But when he developed dementia and his health fell apart in the fall of 2012, the state and the insurer denied his application for nursing home placement and told him he would lose his home care, too. Under tighter rules adopted by the state to serve more people without spending more, Mr. McClanahan was one of thousands of applicants deemed not disabled enough for Medicaid to pay for any help.

The change was new scoring that sharply raised the disability threshold required to get into a nursing home, or to get equivalent care at home. Such thresholds vary from state to state. But in Tennessee, 41 percent of 34,000 applications for care were denied over the 13 months after the change, compared with under 10 percent previously.

In the real world of budget cuts, state officials say, this was the only way to double the proportion of Medicaid recipients served outside nursing homes, to 40 percent. “Yes,” Ms. Killingsworth said, “sometimes that means that not everybody is going to get everything that they think they need.”

In Mr. McClanahan’s case, the day after an official letter scored his need for care at zero, he fell from his short-stay convalescent bed, gashing his face and breaking his nose.

“It’s all about the money,” his son, David McClanahan, said. “I wouldn’t want anybody to have to go through what I went through with my dad.”

Changes, and Audits

For years, efforts to curb fast-rising Medicaid costs centered on welfare mothers and children, even though Medicaid spends more than five times as much on an aged or severely disabled person in long-term care as it does on a poor child.

Long-term care cases traditionally were considered too vulnerable and politically sensitive to be assigned to a managed care company. But between recession-starved budgets and the looming costs of an aging population, many states have decided the old system is unsustainable. About 4.2 million people receive long-term services paid by Medicaid, representing only 6 percent of Medicaid beneficiaries, but about $136 billion, or one-third of all Medicaid spending. They include many formerly well-off people in nursing homes who have “spent down” their “countable” assets — the primary home is the major exclusion — to less than $2,000, the maximum for Medicaid eligibility in many states.

David and Debby McClanahan with his father, Glenn. Managed care officials denied him coverage for nursing home placement and home care.

Chris Berkey for The New York Times

Under the old system, providers bill Medicaid directly, a model plagued by perverse incentives for expensive, unnecessary and even fraudulent care. Despite arguments that people should not have to enter high-priced institutions to get help with activities of daily life like bathing and eating, relatively little Medicaid money was available for cheaper alternatives. Nursing homes have often used political muscle to keep it that way.

Managed care promises more predictable, controlled spending. From a fixed sum per enrollee, plans pay networks of providers to deliver care, which could be as cheap as a recorded medication reminder, or as costly as a nursing home stay.

Like the rationale behind health maintenance organizations, the idea is that plans will benefit financially by keeping costs lower and people healthier, and that the expense of customers who need more care will be counterbalanced by those who need less.

But now, as the formula is applied to a more fragile population, some states have already run into problems that marred the early history of H.M.O.s.

In Wisconsin, which Gov. Andrew M. Cuomo of New York has called a model for his Medicaid reforms, the price of expanding managed long-term care rose by 43 percent in three years, as more people signed up than expected. Further expansion was suspended. The program, which relies on homegrown nonprofits, saw two of nine plans go broke; others cut caregivers’ hours and pay, shifting burdens to relatives.

Still, Kitty Rhoades, Wisconsin’s Medicaid chief, said, “We’re closer to getting it right than most other states.”

Northeast of Nashville, in the house on his father’s farm where he grew up, Mr. Scarlett, who was severely injured in an A.T.V. accident nine years ago, is living proof that high-quality care at home can be better than care in a nursing home. But his family has had to struggle to keep it, under the financial pressures inherent in the shift to managed long-term care.

In 2005, Tennessee shrank Medicaid from one of the most expansive versions in the country to one of the most restrictive. That bitterly contested move, made amid spiraling costs in a state without an income tax, eliminated coverage for more than 170,000 people, many with severe chronic illnesses.

Though the state had experienced more than its share of managed care scandals in the 1990s, it embraced that approach for long-term care, under tight rules and a governor who had been a managed care executive. Officials say it helped keep increases in the state’s Medicaid budget to half the national trend line.

Before his family signed him up for the new program, Mr. Scarlett spent a year in a nursing home, with relatives keeping vigil. Whenever a mucous plug threatened to choke him, his sister, Kimberly Maynard, recalls, she dashed to the nurse’s station to beg for someone to suction the tracheostomy tubing. He was sent to the hospital six times with pneumonia and battled two antibiotic-resistant infections linked to institutional health care. At home, he has been tended 24 hours a day, mostly by licensed practical nurses, and had to go to the hospital only once.

One weekday last fall, propped in a recliner with tubes linking him to life, he struggled to raise a thumb so his father could kiss it, and moved one bare foot against a beach ball that his sister had gently aimed there. “Attaboy,” said his father, Billy Scarlett, 75, who still hopes his son will emerge from what doctors call a persistent vegetative state. “Knock it over here to Daddy!”

But since 2010, when the state expanded the program, called TennCare Choices, and AmeriGroup, a major insurer, took over the case, the program has been trying to drop Mr. Scarlett. The form letters began with a mistaken claim that the family had asked to quit. A letter last fall ordered him “disenrolled” on Dec. 1 and added, “You can’t appeal again.”

The problem: His home care, while it served him best, cost $330,000 a year. If he were dropped from TennCare Choices, he would most likely end up in a nursing home, at an average annual tracheostomy rate of $144,000, potentially a big savings.

In a nursing home “he’d have been gone a long time ago,” Billy’s sister said.

“I just can’t do that,” she added, “because now if you were to do that, you’d actually be murdering him.”

A customer service representative, Barbara Murphy, center, and a coach, Theresa McMahon, right, at AmeriGroup, a major insurer in Tennessee's managed long-term-care program.

Luke Sharrett for The New York Times

The program offered an alternative: Instead of paying an agency $37 an hour for 24-hour care, AmeriGroup would pay the family about $15 an hour to hire caregivers earning less than agency employees, without benefits like health insurance.

The family withdrew its appeal after the state’s lawyer warned at a hearing, “By trying to get something better, they could get nothing.”

Still, AmeriGroup said late last year that the rules had changed again: A nursing home was now the only choice — an ultimatum withdrawn eight hours later, after The Times inquired about it.

WellPoint, which recently acquired AmeriGroup for $4.9 billion, referred questions to TennCare, where officials said privacy laws did not allow discussion of the case. But Kelly Gunderson, a TennCare spokeswoman, added that in any long-term care program, “difficult public policy decisions must be made, including whether to provide an unlimited array of benefits to a few, or a reasonable package of benefits sufficient to safely serve individuals in the community to many.”

Tennessee has chosen to be as cost-effective as possible, she said, and that has allowed the state to eliminate waiting lists for community-based services, which now serve nearly 13,000 people, up from 5,000, while keeping the number of nursing home residents flat at 19,200.

Beneficiaries include Sara West, 64, a former medical records worker who was in rehabilitation centers for months after infections from operations left her in a wheelchair. “I would be in a nursing home if it wasn’t for this program,” she said, calling it a godsend.

Through UnitedHealthcare, TennCare provided a roll-in-shower and pays for about five hours of daily help by aides Ms. West hires herself. Despite having recent amputations and needing nightly bedpan help, Ms. West, a diabetic, has stayed within a $55,000 annual cap for three years, she said, by relying first on her husband, a retiree with dementia, and now on cousins who take turns spending the night.

Assisted Living

Last spring, Mr. McClanahan, in his late 70s and nearly blind, had already cycled through hospitals, rehab centers and geriatric-psychiatric units when he became one of 5,283 people who were told that they did not qualify even for a new, temporary category of home services limited to $15,000 a year — one-third of the state’s Medicaid rate for a nursing home.

Officials acknowledge that among the 1,451 of those denials that were appealed, more than a third turned out to be based on inadequate information and were later reversed. Members have 30 days to appeal denials.

When David McClanahan threatened to publicize a gruesome photo of his father’s face after his fall, UnitedHealthcare, the managed care company, offered placement in an assisted living center, costing a third of the nursing home rate. Such centers are not regulated or equipped for people with serious impairments.

The McClanahans were unaware that the center, Elmcroft of Twin Hills, had been under state investigation for resident deaths linked to neglect, and for complaints that it kept residents who needed more care than it could deliver.

Elmcroft, acquired by a 19-state chain in 2011, said it had fixed the problems by last spring, when it passed a state inspection. But relatives noticed unwashed sheets and pills scattered on Mr. McClanahan’s floor, and said the center demanded $460 more a month from Mr. McClanahan because he needed more care than expected. It settled for his full Social Security income. “He remained a viable assisted living resident” under state rules, Bob Goyette, a spokesman for Elmcroft, said, “even though he required more care.”

Mr. McClanahan soon ended up back in a hospital, and the whole process began again. Eventually, after his son threatened to sue the state, another denial was reversed and Mr. McClanahan secured a nursing home bed. “They’re trying to see what they can get by with,” his son said.

Alice Ferreira, a spokeswoman for the insurer, said, “UnitedHealthcare has worked very closely with the family and the TennCare to ensure the member has the appropriate care.”

Ms. Gunderson, the state spokeswoman, said: “The due process procedures we have in place work effectively to ensure that members are able to receive the appropriate level of services in the appropriate setting.”

For raising the level of impairment required to qualify for nursing home-level care, Ms. Killingsworth said, “I make no apology.”

She called the previous threshold — a significant deficiency in one activity of daily life — one of the lowest in the country, while the new threshold is based on a weighted point system that typically requires serious deficiencies in three activities.

Nationwide, publicly traded companies like UnitedHealthcare are replacing nonprofits. There are trade-offs, said Michael J. McCue, a professor of health administration at Virginia Commonwealth University, whose comparative study found that publicly traded plans focusing primarily on Medicaid enrollees reported the highest percentage of administrative expenses, and received lower scores for quality of care.

“They have to make sure that they meet earnings expectations to help improve their stockholders’ wealth,” Professor McCue said. “They could argue that, hey, maybe we have a more effective way of managing the care or cost. And one can ask, hey, are they denying care?

“We’ll have to collect the data on that and see.”

Correction: March 20, 2014

An article on March 7 about problems that states have encountered in shifting to privately managed long-term care programs misstated one reason Minnesota was criticized for its handling of the transition. The state came under congressional fire for shifting state costs to Medicaid, which has partial federal funding — not for shifting Medicaid costs to federal Medicare.