Facebook's New Open Source Software Can Learn 1 Billion Words in 10 Minutes

FastText is the tool that powers Facebook's AI--and now it's available for anyone.

Back in June, Facebook announced that it was working behind the scenes on creating bots that would understand everything its users type. So, if you message a friend about needing a ride, for example, it will prompt the social-media app to bring up a button offering an Uber or a Lyft.

Now Facebook is letting anyone use the software that powers that artificial intelligence. In a blog post Thursday, Facebook announced that it is open sourcing FastText, a library of code used to identify words and makes AI stronger.

Facebook's AI has become central to the social network. It's why your news feed almost never contains graphic content and why searching "posts by my friends about Ryan Lochte" gives pretty accurate results. The company also recently announced it would be making a stronger effort to block clickbait headlines that over-promise and under-deliver.

FastText makes all these features possible. It works by using a classification system, placing words into categories and subcategories instead of trying to understand them the way a human brain would. It also learns prefixes, suffixes, and root words. These shortcuts let it learn crazy fast--what would take days for other machine learning systems, FastText can do in seconds. Facebook says it can process 1 billion words in 10 minutes.

Other businesses could have big uses for this technology. Any site that includes a search engine or uses predictive text could become far more accurate. It can also effectively block spam, profanity, or anything else that users wouldn't want to see. And as more and more sites use chat bots as a means of customer service, there's a huge need to make them smarter.

So why is Facebook giving it away? For starters, offering it up might help Facebook gain favor with the coding community, maybe making it more likely that some of the best developers will want to work for Facebook. But maybe more importantly, opening FastText up to coders and engineers all over the world means more people will have the opportunity to study it and offer feedback--giving Facebook an easy way to continually improve its technology.

FastText is available in Facebook's GitHub. In addition to English, it currently works with German, Spanish, French, and Czech.