NEW YORK (TheStreet) -- The fiscal cliff is not about Republicans or Democrats, it's about avoiding a recession, Jim Cramer told Matt Lauer on NBC's "Today" Monday. Cramer said there has to be some common ground, that's what politics is supposed to be about.

Cramer outlined some of the many spending cuts and tax increases, over $600 billion in total, that will take effect on Jan 1. He said that everyone will feel some of the $174 billion in overall tax increases, but many may not even be aware of the $59 billion being raised as the alternative minimum tax gets readjusted for inflation.

Cramer said he's most worried about the $100 billion in social security tax increases, as those increases will damper hiring and lead to a recession. Yet of all the expected changes, it's the $40 billion tax increase for the wealthy that has most of Congress' attention, noted Cramer, which shows how misguided Washington still is and how unlikely a deal is to occur.

"Someone has to give," Cramer concluded, or tens of thousands of people will lose their jobs, benefits will be cut and our country will be seriously damaged.

To sign up for Jim Cramer's free Booyah! newsletter with all of his latest articles and videos please click here.