The team behind Baidu’s first smart speaker is now using AI to make films

The HBO sci-fi blockbuster
Westworld has been an inspiring look into what humanlike robots can
do for us in the meatspace. While current technologies are not
quite advanced enough to make Westworld a reality, startups are
attempting to replicate the sort of human-robot interaction it
presents in virtual space.

Rct studio, which just
graduated from Y Combinator and ranked among TechCrunch’snine
favorite picks from the batch, is one of them. The
“Westworld” in the TV series, a far-future theme park staffed
by highly convincing androids, lets visitors live out their heroic
and sadistic fantasies free of consequences.

There are a few reasons why rct studio, which is keeping mum
about the meaning of its deliberately lower-cased name for later
revelation, is going for the computer-generated world. Besides the
technical challenge, playing a fictional universe out virtually
does away the geographic constraint. The Westworld experience, in
contrast, happens within a confined, meticulously built park.

“Westworld is built in a physical world. I think in this age
and time, that’s not what we want to get into,” Xinjie Ma, who
heads up marketing for rct, told TechCrunch. “Doing it in the
physical environment is too hard, but we can build a virtual world
that’s completely under control.”

The startup appears suitable to undertake the task. The
eight-people team is led by Cheng Lyu, the 29-year-old entrepreneur
who goes by Jesse and helped Baidu build up
its smart speaker unit from scratch after the Chinese search
giant acquired his
voice startup Raven in 2017. Along with several of Raven’s
core members, Lyu left Baidu in 2018 to start rct.

“We appreciate a lot the support and opportunities given by
Baidu and during the years we have grown up dramatically,” said
Ma, who previously oversaw marketing at Raven.

Let AI write the script

Immersive films, or games, depending on how one wants to
classify the emerging field, are already available with pre-written
scripts for users to pick from. Rct wants to take the experience to
the next level by recruiting artificial intelligence for
screenwriting.

At the center of the project is the company’s proprietary
engine, Morpheus. Rct feeds it mountains of data based on
human-written storylines so the characters it powers know how to
adapt to situations in real time. When the codes are sophisticated
enough, rct hopes the engine can self-learn and formulate its own
ideas.

“It takes an enormous amount of time and effort for humans to
come up with a story logic. With machines, we can quickly produce
an infinite number of narrative choices,” said Ma.

To venture through rct’s immersive worlds, users wear a
virtual reality headset and control their simulated self via voice.
The choice of audio came as a natural step given the team’s
experience with natural language processing, but the startup also
welcomes the chance to develop new devices for more lifelike
journeys.

“It’s sort of like how the film Ready Player One built its
own gadgets for the virtual world. Or Apple, which designs its own
devices to carry out superior software experience,” explained
Ma.

On the creative front, rct believes Morpheus could be a
productivity tool for filmmakers as it can take a story arc and
dissect it into a decision-making tree within seconds. The engine
can also render text to 3D images, so when a filmmaker inputs the
text “the man throws the cup to the desk behind the sofa,” the
computer can instantly produce the corresponding animation.

Path to monetization

Investors are buying into rct’s offering. The startup is about
to close its Series A funding round just months after banking seed
money from Y Combinator and
Chinese venture capital firm Skysaga, the startup told
TechCrunch.

The company has a few imminent tasks before achieving its
Westworld dream. For one, it needs a lot of technical talent to
train Morpheus with screenplay data. No one on the team had
experience in filmmaking, so it’s on the lookout for a creative
head who appreciates AI’s application in films.

Rct studio’s software takes a story arc and dissects it into a
decision-making tree within seconds. / Image: rct studio

“Not all filmmakers we approach like what we do, which is
understandable because it’s a very mature industry, while others
get excited about tech’s possibility,” said Ma.

The startup’s entry into the fictional world was less about a
passion for films than an imperative to shake up a traditional
space with AI. Smart speakers were its first foray, but making
changes to tangible objects that people are already accustomed to
proved challenging. There has
been some interest in voice-controlled speakers, but they are
far from achieving ubiquity. Then movies crossed the team’s
mind.

“There are two main routes to make use of AI. One is to target
a vertical sector, like cars and speakers, but these things have
physical constraints. The other application, like Alpha Go, largely exists
in the lab. We wanted something that’s both free of physical
limitation and holds commercial potential.”

The Beijing and Los Angeles-based startup isn’t content with
just making the software. Eventually, it wants to release its own
films. The company has inked a long-term partnership with Future Affairs Administration, a
Chinese sci-fi publisher representing about 200 writers, including
the Hugo award-winning Cixin Liu. The pair is expected to start
co-producing interactive films within a year.

Rct’s path is reminiscent of a giant that precedes it:Pixar Animation Studios .
The Chinese company didn’t exactly look to the California-based
studio for inspiration, but the analog was a useful shortcut to
pitch to investors.

“A confident company doesn’t really draw parallels with
others, but we do share similarities to Pixar, which also started
as a tech company, publishes its own films, and has built itsown engine,” said Ma.
“A lot of studios are asking how much we price our engine at, but
we are targeting the consumer market. Making our own films carry so
many more possibilities than simply selling a piece of
software.”