Mary-Kate and Ashley Olsen Turn 21

Mary-Kate and Ashley Olsen turned 21 on Wednesday, and unlike many young stars hitting the milestone, they kept their celebration low-key.

The sisters hosted a cozy dinner party at Hollywood's legendary Chateau Marmont, where guests included their sister Lizzie, 19, and brother Trent, 23.

"It looked like a family dinner," an onlooker tells PEOPLE, "You could tell they were all really close. The girls couldn't stop smiling."

A little after 11 p.m., two red velvet cakes – one covered in white frosting and the other in black – were brought out as the table burst into "Happy Birthday." Their friends cheered as the girls closed their eyes and blew out the candles.

Soon after the cake was cut, another treat arrived: To the delight of the guests, boxes of cupcakes were served between the tea lights and petite vases of cut pink and white flowers dotting the table.

While the champagne and wine continued to flow, each guest took a turn raising a glass and saying a few words about the girls. Between cheers and laughter at inside jokes, the sisters posed for pictures taken with a vintage Polaroid camera, then anxiously crowded over the developing photos before laughing at the results.

Though there were plenty of gifts on the table, the true present for Mary-Kate was the guy sitting next to her: boyfriend Max Snow. After giving the last toast, he kissed her lovingly before she jumped into his lap and gave him a peck on his chin. As she looked through her birthday cards, he brushed her cheek with one hand and caressed the back of her sheer black dress.

And even the A-list diners in the restaurant were star-struck: Director Brett Ratner strained to peek over his high-backed chair hoping to catch a glimpse of the famous birthday girls.

What did the sisters do before the party? As they wrote to fans on their Web site: "Today, for the first time in a looooong time, we're taking a day off – doing absolutely nothing and looking forward to every second of it!"