SAN FRANCISCO — A growing throng of Americans revel in being able to call out "What's the weather?" to their Amazon Echo or "Is 'The Shawshank Redemption' on?" to their Samsung SmartTV and get the immediate answers.

But these first forays into a world where digital servants always listen for our commands raise red flags for privacy and security experts, who see too many ways it could all go horribly wrong.

“In our homes, there are all sorts of conversations that are going on and are meant to be personal and private,” said Lynn Terwoerds, executive director of the Executive Women’s Forum on Information Security and Risk Management.

On Wednesday, she and others launched the Voice Privacy Industry Group at the RSA computer security conference. Their goal is to set a voice privacy agenda for developers "early on, when they’re starting to think about it, not later when they have to patch something that’s fundamentally broken," she said.

Most voice-command devices listen for a "wake word" that tells them to start paying attention, such as "Alexa!" or "Hey, Siri." Simple commands can be processed on the device while more complex requests are uploaded via wireless to the cloud where they're translated into text the program can understand and act upon.

Questions that concern the group: How much the device records, whether the audio stream is encrypted as it zips through the cloud, how long it's stored and who has access to the information.

The worry is that this trickle of helpful adjuncts could become a flood of invasive devices bent on listening and learning from everything we say around them.

"Now is the time for setting privacy expectations," said Michelle Dennedy, chief privacy officer for Cisco and founding member of the group.

It was initially an outgrowth of the Executive Women’s Forum, a professional organization for women in information security. The new Voice Privacy Industry Group is open to both men and women.

Siri can now name songs, via Shazam.(Photo: Apple)

Always listening

Americans have grown inured to the idea that their computers are always tracking them. Search for a kitchen faucet online and faucet ads seem to appear on every Website you visit for days.

But could there come a day when talking about buying a faucet in the kitchen could be overheard by your TV in the living room, changing the types of commercials that show up when you’re watching your favorite program the next night?

That's the type of scenario the group would like to get ahead of, while the list of voice control devices is still short. Today, the most widely-deployed are Apple's Siri, Google's OK, Amazon's Echo and Fire TV, and some Samsung Smart TVs.

They represent "an enormous sea change to our relationship with the computer. And this is where privacy comes into the mix,” said Kelly Fitzsimmons, co-founder of Custom Reality Services, a virtual reality production company and also a part of the working group.

The aim isn't to shut voice control down but to nudge it in the right directions. “We don’t want to kill the innovation cycle, but I care about whether my TV is listening to me,” said Joyce Brocaglia of Alta Associates, an executive cybersecurity search firm that helped launch the group.

No legal cases, yet

One concern has been that law enforcement might subpoena sound files recorded in a home when investigating a crime, or that they could be discoverable in a divorce proceeding.

So far no actual cases seem to have hit the courts, though "we are finding that evidence from voice technology can be quite valuable if the information leads to some kind of tracking mechanism that places a spouse at a different location than they might have claimed," said Joslin Davis, president of the American Academy of Matrimonial Lawyers.

Privacy experts disagree whether this kind of voice tracking would be illegal.

“These devices are microphones already installed in people’s homes, transmitting data to third parties. So reasonable privacy doesn’t exist. Under the Fourth Amendment, if you have installed a device that’s listening and is transmitting to a third party, then you’ve waived your privacy rights under the Electronic Communications Privacy Act,” said Joel Reidenberg, director of the Center on Law and Information Policy at Fordham Law School in New York City.

But what about people who aren’t the device’s owner themselves, and who never agreed to anything, who find themselves in a room where their voices are being listened to, asks Marc Rotenberg, executive director of the Electronic Privacy Information Center in Washington. Last year the group filed a request that the Federal Trade Commission investigate Samsung's voice-operated TVs. So far, it hasn't received a response.