Website password strength meters, like a spouse asked to assess your haircut or outfit, often tell you only what you want to hear.

That’s the finding from researchers at Concordia University in Montreal, who examined the usefulness of those pesky and ubiquitous red-yellow-green password strength testers on websites run by big names such as Google, Yahoo, Twitter and Microsoft/Skype. The researchers used algorithms to send millions of “not-so-good” passwords through these meters, as well as through the meters of password management services such as LastPass and 1Password, and were largely underwhelmed by the results.

"We found the outcomes to be highly inconsistent. What was strong on one site would be weak on another," says Mohammad Mannan, an assistant professor with Concordia's Institute for Information Systems Engineering, in a statement. He collaborated on the study with Ph.D student Xavier de Carné de Carnavalet.

But that doesn’t mean the meters have necessarily been designed well, according to the Concordia researchers, whose study (A Large-Scale Evaluation of High-Impact Password Strength Meters) will be published in the journal ACM Transactions on Information and System Security. The study asserts that most of the meters studied “are quite simplistic in nature and apparently designed in an ad-hoc manner.”

And just because a meter rates a password as strong, doesn’t mean that it is, the researchers say.

In their study, the researchers singled out cloud file-sharing service Dropbox as having among the stronger password checkers – and an open source one that includes an explanation of its design. Among other things, the checker puts the kibosh on any words found in the dictionary. Dropbox rated “Password1” as very weak, but another site, Yandex, okayed it as secure.

Overall, password strength gateways are inconsistent, with some allowing all letters and others requiring different character sets to gain approval, the researchers found. That sends a mixed message to online users accessing many different websites.

Mannan says that despite warning most of the website operators about the study findings, few have made changes, but the researchers are hopeful their work will encourage website operators as well as other academics to take a harder look at this issue.

Bob Brown is a news editor for Network World, blogs about network research, and works most closely with our staff's wireless/mobile reporters. Email me at bbrown@nww.com with story tips or comments on this post.