"You wouldn't have the same key for your home, car, office, safe, etc.," Grossman wrote. "For the same reason you shouldn't use the same password for all your online accounts."

He recommends picking passwords that are hard to guess, not found in the dictionary, six characters or more in length, and that have a mix of numbers and letters. Two examples are y77Vj6t or JX0r21b.

Since having multiple passwords can be hard to remember, you can write down the passwords on a piece of paper that fits in your wallet or on index cards that can be locked in your desk. Or, you can use a password manager, which is software that stores your password and encrypts the data Grossman says.

Related stories

Chris Wysopal, of Veracode, said it's also good to keep a password manager, like the Password Wallet app, on your phone so you can access them easily if you are away from your computer. Additionally, he said it's important to change passwords if they have similar patterns. For instance, he said one of the hacked passwords he saw was "scottlinkedin" which could potentially be a security risk for Scott's other accounts.

"Someone might go to Facebook and try 'scottfacebook,'" he said. "It's good to have unique passwords for each one, but the pattern is so obvious, it's good to change the other passwords."

Prior to confirming the breach, LinkedIn offered similar advice on its blog, adding that passwords should be changed frequently, at least once a quarter or every few months.