Disney Research taught AI how to judge short stories

The researchers used social question and answer site Quora for a large database to feed into its AI algorithms. Many of the answers on Quora come in the form of stories, so reader upvotes can be used as a measure of popularity, and as “a proxy for narrative quality.” The team gathered almost 55,000 answers and classified more than 28,000 of them as stories, each with an average of 369 words. Then they developed a couple of different neural networks — one to look at different sections of each story and one to take a more holistic view of a story’s meaning. Each AI made predictions about the relative popularity of a given story. Both neural nets were better at choosing a story’s popularity over a baseline text evaluation, but the holistic network showed an 18 percent improvement over the one that focused on sections.

It’s not hard to imagine a movie studio, for example, using a future version of this type of technology to choose scripts for production, of course, but the tech is still in its infancy. Let’s just hope that researchers find a way to filter stories for quality, and not just popularity. No one needs another Transformers movie.