Justice Anthony Kennedy will likely cast the deciding vote. He has written all the court's gay rights rulings, but he is also one of the court's most ardent advocates of free speech rights. And both sides of Kennedy's legacy were on display.

"If you prevail," he asked the lawyer for Phillips, "and bakeries put signs in the window saying, 'We don't bake cakes for gay weddings,' wouldn't that be an affront to the gay community?"

Kennedy seemed to take the opposite side of the case when he told a lawyer for the gay couple, "Tolerance is essential in a free society," but added that Colorado wasn't very tolerant of Phillips' religious beliefs when the state's human rights commission ruled against him.

The incident that brought the case to the high court lasted only a few minutes, the day David Mullins and Charlie Craig of Denver walked into Masterpiece Cakeshop.

When Phillips found out they wanted a cake for a reception to celebrate their wedding, he said, "I'm sorry, guys, I can't do that."

After the couple filed a formal complaint, the Colorado courts ruled that the state's public accommodation law, which bans discrimination by companies offering their services to the public, did not allow Phillips to refuse the gay couple's request. So he appealed.

Requiring him to make a wedding cake for a same-sex couple, he said, would violate his First Amendment rights by requiring him to express a view, through his cakes, that is counter to his religious beliefs.

Much of Tuesday's 80-minute argument involves what-ifs. If a cake is a form of expression, Justice Ruth Bader Ginsburg asked, what about flowers, the invitation or the wedding menu? Justice Elena Kagan, in the same vein, asked what about the make-up artist or the hairdresser?

"That's not speech," said Kristen Waggoner, appearing on behalf of Phillips.

Colorado courts said a reasonable person would assume that the cake expressed the message of the couple, not the baker. And the ACLU's David Cole, arguing for the gay couple, said that was correct.

"If a mother asks a bakery for a cake that says 'Happy Birthday' and serves it at her child's birthday, no one thinks that's the baker's wishes. They think it's the mom's."

While the court struggled with the issue of whether a cake is a means of expression, the justices also seemed troubled by whether a ruling for Phillips could open the door to more widespread discrimination in the name of religious freedom.

"We want to find some kind of distinction that won't undermine every civil rights law since the year two," said Justice Stephen Breyer.