Why Recommendation Engines Are About To Get Much Better

Expanding data sources, including social media sources, are making recommendation engines much more powerful.

5 Big Wishes For Big Data Deployments

(click image for larger view and for slideshow)

Amazon.com certainly deserves credit for bringing the term "recommendation engine" into the general lexicon. But recommendation engines have expanded well beyond consumer-facing shopping sites like Amazon as programmatic ways of making accurate recommendations.

Recommendation systems are now common in insurance and finance, where they can suggest coverage plans and investment opportunities, and they are also showing up in B2B settings, recommending prospects and strategies to salespeople.

So-called recommender systems analyze a mass of past choices, from one user or many users, to make new suggestions, or take attributes of one item to suggest new items with similar properties. The first approach is called collaborative filtering; the second is known as content-based filtering. The two methods are often combined.

ACM RecSys, an annual international conference on recommender systems, will hold its seventh meeting Oct. 12-16 in Hong Kong.

"Companies are learning to capture and learn from more data sources," Dan Mallinger, data science lead at Think Big Analytics, wrote InformationWeek in an email. "As big data supports both structured and unstructured data, companies can understand everything from customer transactions, to call center data, to what's selling on secondary auctions."

Recommendation engines come into play "anywhere options proliferate and search alone does not sufficiently limit results," he wrote.

Figuring out what kind of artwork to suggest to online shoppers is the goal of a home-grown recommendation engine being developed at startup Suede Lane

"We're developing a new way for people to develop their taste without needing to know any industry jargon," founder and CEO Yinka Abdu told InformationWeek. These recommendations will, for instance, help shoppers find, buy and enjoy a painting in the impressionist style without needing to know that term.

Abdu also sees an opportunity to use recommendation engines to educate shoppers. "There's a huge swath of people who want to learn about art or design," she said, imagining a future recommendation engine that might point Suede Lane visitors to educational resources once her website detects -- based on explicit or implicit choices -- an interest in, say, cubism.

Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.

Why should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.