"In particularly, the main piece of unfinished business from the Great Recession is the plight of the long-term unemployed," Perez said at the 2014 conference of the National Coalition for Homeless Veterans.

"Unfortunately there's a little bit of [unemployment insurance] fatigue among some in Congress. We're going to make sure that fatigue doesn't exist, because too many people are suffering," he said.

Perez said he regularly meets with people who have been out of work for an extended period and recalled visiting with a military veteran in Northern Virginia who lives in a 275-square foot trailer with his wife and three children.

"[He is] remarkably talented and has been unemployed for almost two years, doing odd jobs just to get food. Ran out of unemployment benefits — effectively homeless, and he has been stripped of his dignity," Perez said.

"That's really what I hear from so many people: 'I have a poverty of spirit,' " he said.

Perez, whose father was a veteran, said long-term unemployment is the issue that keeps him up at night.

"We need to make sure that we don't ever quit on them," he said. "It is not a lifestyle, it is a lifeline for people who have lost their jobs through no faults of their own."