If oil prices keep heading higher, that could translate into a jump of more than 20 cents at the pump within the next couple weeks, said Schork. Currently, the national average of gasoline is $3.65 per gallon.

Rising gas prices hurting Ukrainians

The issue is supply: Investors have worried about tight inventory ever since the Libyan conflict limited that country's production, so a decline in Iraq's production could be a big problem for oil markets.

"This is the first time in a while where we do have a legitimate concern about the availability of crude oil from a geopolitical standpoint," said Schork. "The market is already missing Libyan oil, and the market has been able to compensate -- but now if you add in a loss of Iraqi exports, that is certainly a concern and should logically push oil prices higher."

Still, Kloza doesn't think gas prices will cross the $4 mark at a national level.

"The country will see modestly higher prices than we saw one year ago, but right now, I don't believe we are on the threshold of $4, which can spark outrage, malaise, and a bit of demand destruction," he said.