The poll found that 59 percent of the billionaires believe the U.S. elections in November will “significantly impact the tax policy, affecting both the climate and the level of philanthropic activity.”

President Obama has called (unsuccessfully so far) for limiting charitable deductions to 28 percent for couples with incomes of more than $250,000. The gift tax exemption is scheduled to fall to $1 million from $5 million, and the estate tax could go to 35 percent with an exemption of $5 million. All of these deductions affect how and how much donors give.

As billionaire Stephen Ross, chairman of The Related Companies, said in the report: “While tax laws haven’t impacted me, nor are they a reason I’ve given [to charity], they will have a major impact if giving is no longer a deduction.”

Mr. Perez said that “Without private giving, the arts and theater would close down. What happens then? How can a new generation grow up without these?”

Not all billionaire philanthropists are as tax-driven. The survey found that 70 percent of the billionaires say they are prompted to give by "personal values." More than half said that they are "setting an important example for future generations." (Read more: The Billionaire Who Stopped Giving)

Taxes, in other words, are just one factor – and perhaps not even among the top factors for giving for many.

As the heiress Liesel Pritzker Simmons said “I don’t care what the political situation is. I would still give. That has to do with the size of resources.”