Here's a breakdown from YouGov's Michael Bailey of the poll question and its response:

It is therefore interesting to get a handle on what Americans perceive when they hear "income tax." The most recent YouGov/Economist poll asked, "Do you pay federal income taxes?" Seventy-eight percent said they do. Given that about 54 percent of households pay federal income tax that suggests roughly 24 percent of respondents report that they pay federal income taxes, but do not.

The disparity comes partly from people that incorrectly identify payroll taxes — such as Social Security and Medicare taxes — as income taxes. Thirty-five percent of respondents in the YouGov/Economist poll made that mistake:

From the Tax Policy Center, a look at who does not pay federal income taxes. Most people, if employed, are paying payroll taxes. The two primary sources of people who pay neither federal income tax nor payroll taxes are either very poor or elderly, as the last pie chart displays: