"He went to three stores and couldn't find anything," a source tells PEOPLE. "He was frustrated that they don't make men's rings anywhere. He said it's impossible. The fourth store was Saks. He saw a David Yurman ring and he knew that was the one, so he bought it. He said it spoke to him."

With the hardest part finally out of the way, Bass, 34, was ready to pop the question in Jackson Square in New Orleans.

"They went, but it was closed," the source says. "The park and gates were closed, so he got down on one knee outside of the gates and proposed. [Michael] said yes. After, they went to a karaoke bar called the Cats Meow, where all their friends gathered to celebrate."

Bass sold his place in New York, and now the couple is "officially back in L.A. and ready to start a family together," the insider says, adding that they will probably tie the knot in a year.

Michael Turchin (left) and Lance Bass

Courtesy of Lance Bass

The singer, who recently reunited briefly with his former 'NSYNC band mates, isn't the only one over the moon about his engagement to the artist.

"Lance and Michael share the same passions, dreams, and vision for the future," their good pal Sarah Smart tells PEOPLE. "They complement each other perfectly. I am overjoyed that two of my closest friends have found each other and true happiness."

Bass announced his big news on Sunday with a photo of the pair showing off an engagement ring on Turchin – a self-described "nice Jewish boy" – on Instagram.

"He said YES!!" Bass wrote as a caption to accompany the shot. "Love this man."