One of the most popular out there is the 1955 double die. Close up, you can see the date stamp looks like a double image. If you're lucky enough to find this one, the going price varies.

"We've sold them as cheap as $800 and up to $50,000. It just depends on the collector really," Duncan said. "Another is the three-legged buffalo. It's a 1937 Denver mint coin."

With that coin, the buffalo on the back appears to only have three legs. The error was discovered after the coins entered circulation. The value of those vary based on grade quality. Some have gone for $50,000.

A piece from the 1920s to watch out for is the 1922 no D-cent. The letter D wore off during the time the coins were pressed and can go for around $200.

If you're lucky enough to find a 1943 bronze penny, Duncan said it's like hitting the lottery. Due to a copper shortage during World War II, zinc coated steel was used to produce cents in 1943. However, a small fraction of coins were mistakenly struck on bronze planchets. Only 26 have been found to date, and a select few have sold for over a million.

It's certainly worth keeping an eye open.

If you think you've found a 1943 bronze cent, there's a test you can do. Grab a magnet and hold it to the coin. If it doesn't stick, you may want to have it checked by a reputable coin expert.