Didas Yuryahewa, bent double and waist-deep in water, holds his breath as he struggles to gouge out another shovel of stinking black mud. The air is thick with the bad-egg stench of hydrogen sulphide mixed with ammonia. The equatorial sun beats down on his naked back, leaving a salty sheen.

In the good times, Yuryahewa – and hundreds of other salt miners at Lake Katwe in western Uganda – can make a reasonable living, but it is a casino existence. Salt production turns rapidly from boom to bust with the seasons, leaving the workers struggling to make ends meet, and climate change is starting to load the dice against them.

The gathering of environment ministers and officials at UN climate talks in Cancun, Mexico, on 29 November may seem a world away, but development campaigners say progress towards a deal to raise $100bn (£62bn) a year by 2020 to help poorer countries such as Uganda adapt to climate change is essential.

The climate change fund was seen as one of the few successes to come out of the ill-fated Copenhagen talks last December. Two weeks ago, leading economists, finance ministers and heads of state came up with a draft plan for how the money could be raised – including a mixture of carbon taxes, aviation and shipping taxes, and the redirection of fossil fuel subsidies.

At Cancun, negotiators are expected to inch closer to an agreement on finance, but progress on reducing the world's carbon emissions looks to be as far away as ever. With President Barack Obama's climate bill killed off in the US Senate, America will come with little to offer. In the absence of firm pledges from the world's largest economy, China, India and other emerging economies are unlikely to move their positions.

Poverty campaigners say that a deal on both finance and emissions cuts will be vital to improve the lot of Yuryahewa and people like him. Oxfam's campaigns and policy director, Phil Bloomer, said: "Lake Katwe is symptomatic of poverty around the world today … World leaders have so far failed to tackle the difficult decisions needed to confront the threat of climate change, but the longer they shy away from it, the worse and more widespread the problem is going to be."

The salt miners, meanwhile, are gambling with their health. The hypersaline water sucks moisture from their bodies and infuses them with toxic chemicals – the dead lake taking a toll on the living.

Like many workers, 18-year-old Yuryahewa is an orphan. His father – also a miner – died of a chest infection in 2004 and his mother abandoned him and his 16-year-old brother to fend for themselves. Yuryahewa struggles to pay his brother Reuben's school fees from the 500,000 shillings (£155) a month he earns from the lake. Now Reuben misses four school days each month to work in the saltpans as well to help cover the fees.

Yuryahewa works in the salty water for more than two hours a day. He knows it is affecting his health but he cannot afford to buy protective clothing. "When you go into these waters when you are not properly dressed you will of course have a problem," he said. "If you have a small wound then … it will eventually turn into a very big wound." His hands and feet are rough and covered in sores.

Many workers improvise as best they can. Men tie plastic bags around their genitals or wear condoms in an effort to avoid the extreme desiccating effects of the brine, while some women put flour inside their vaginas as a barrier to the toxic water. These improvised strategies have little effect and sexual health problems are rife among the Katwe workers.

"It's very dangerous," said Rutura Rwankangi, head porter at Katwe, who describes impotence as a major problem for workers. Salt from Katwe has been central to the local economy for centuries; but the miners, salt-processors and porters who carry bags of salt to waiting trucks see few of the spoils.

"It is extremely labour-intensive and they sell what they get out of it at very low prices," said Clovis Kabaseke, an environmental scientist at Mountains of the Moon University in Fort Portal, the regional capital. "This is why they have remained poor, especially those who actually do the mining."

Katwe, a volcanic crater fed by several streams, has no outlet, meaning that intense evaporation during the dry seasons concentrates minerals in the hypersaline water. To extract salt, the miners have constructed large semi-permanent pools around the edges of the lake to intensify the evaporation. These pools, or saltpans, are held together by sticks and mud. Their ownership is crucial. The saltpan's proprietor takes a cut of whatever a miner working the pan can extract.

The stench and grime of the mining is a stone's throw away from another world. Just across the hippo-filled waters of neighbouring Lake Edward, and within tantalising sight of the shantytown where the workers live, is the exclusive Mweya Safari Lodge. A night here will set you back up to $840 a night (the Queen Mother stayed here in 1954). Few of the well-heeled international guests relaxing at Mweya's poolside bar will be aware of the hardship on the hotel's doorstep.

Both Mweya and Katwe are inside Queen Elizabeth National Park, a vast area of savannah that is home to elephants, lions, leopards and more. Being marooned in the park only adds to the salt miners' isolation and marginalisation. The hour-long drive to the nearest town, Kasese, costs the same by public transport as the six or seven-hour trip from Kasese to the capital Kampala, so they rarely make the trip.

Reuben Mbauta, chief executive of the Foundation for Urban and Rural Advancement (Fura), a local NGO that is funded by Oxfam to improve the workers' lot, said the Ugandan government has neglected the salt-mining community because it is so cut off. "The death rate in Katwe is very high," he said, "It is an under-served place. Health delivery systems are not very clear. Not very good at all." Mbauta estimates that around 20,000 people share one health clinic.

"The water system is broken. They get water from [Lake Edward]. Therefore they are more affected by water-related diseases," he added.

Through Fura's activities, Oxfam has been helping to improve sanitation and educating the community in how to avoid diseases such as cholera.

The salt yield from Katwe, however, has dwindled in recent years and become more unpredictable because of Uganda's increasingly uncertain climate.

"Climatic changes have been very evident here in Uganda," said Kabaseke. "The months of June and July used to be a very dry season and December to February would be another very dry season … but these days things are not like that."

Climate scientists predict that weather patterns in Uganda will shift as a result of global warming, although it is hard to say with absolute certainty whether the unpredictable rains are part of that shift. What is certain is that it is hitting the salt miners hard. Too much rain means not enough evaporation to produce salt.

Bloomer said a global deal on climate finance and emissions cuts is vital to help improve the lot of the salt miners and millions like them whose livelihoods are vulnerable to a change in the climate. "The UN climate talks at Cancun next month must move decisively towards raising the money to help people like Didas Yuryahewa adapt and protect themselves," he said.

• The flights and accommodation for James Randerson's trip to Uganda were paid for by Oxfam. The charity did not have any control over content of the article.