With her wholesome beauty, big-toothed grin and powerful voice, Sutton Foster has become Broadway’s go-to musical sweetheart — albeit one with a wicked comic sensibility. These attributes are on ample display in her Café Carlyle debut, in which the star of “Thoroughly Modern Millie,” “The Drowsy Chaperone,” “Shrek: The Musical” and more is clearly relishing the opportunity to show some new sides.

After a gushing introduction Tuesday by Donna Murphy, her co-star in the Encores! revival of “Anyone Can Whistle,” Foster began her show, predictably enough, with songs from her shows, including “Little Women” and “Annie.” (She played the maid in the ill-fated Nell Carter revival.) This was followed by the Drifters’ classic “Up on the Roof,” delivered in a sweet but generic fashion that didn’t bode well for the rest of the night.

Happily, this was just a warm-up. The novelty number “Air Conditioner” showcased Foster’s flair for antic humor, while its follow-up, “Warm All Over,” revealed her sultry side.

Much of the show consisted of such dichotomies: “My Heart Was Set on You” was delivered with an aching vulnerability that contrasted well with the cynical “Down With Love.”

The hilarious “Show Off,” from “The Drowsy Chaperone,” was virtually a one-woman show in itself — one that let Foster change clothes onstage, show off her Tony, yodel and even touch the tip of her nose with her tongue.

Another highlight was when she let an audience member — in this case Zach Braff, her co-star in the upcoming off-Broadway play “Trust” — randomly select a number from what she called “The Big Book of High Belt Songs.” It was “Defying Gravity” from “Wicked,” which she delivered full- force, prefaced by her warning, “Everyone has to back up!”

But for all her comic bravado, Foster’s essential sweetness shines through. Few performers can make John Denver’s “Sunshine on My Shoulders” seem anything less than hokey, especially in this room. Yet with that and her touching encore of The Beatles’ “Here, There and Everywhere,” Foster showed how she makes every character she plays believable — even the one in love with a big, green ogre.