Black Eyed Susan's for sure. I've had several spectacularly good meals there. It is BYOB (there is a liquor store a short walk down Main St where you can pick up something if you want), and doesn't take credit cards. Unless they've changed thing since last year, they do 3 seatings a night (6 pm, 7:30 pm, 9 pm). The 6 pm seating you can make reservations for on the phone; if you want to eat later (or can't get a 6 pm reservation), show up around 5:30 and get in line to put your name down for the 7:30 pm or 9 pm seating. It sounds complicated, but it is definitely worth it.

If you're looking for something more upscale, I had a great dinner once at the Straight Wharf restaurant -- I still think about the bluefish pate they served with the bread basket 5 years later.

Down Cliff Road is Something Natural Bakery (easy 10 minute walk from town, or even shorter bike ride). Great sandwiches and bread, perfect for lunch. The herb bread in particular is great -- an avocado, cheddar, and chutney sandwich on herb bread is one of my favorite things for lunch on the island. If you do have bikes, get sandwiches at Something Natural and bike down to Tuppency Links (further along Cliff Road) or Dionis (town beach, follow signs on the bike path) for a picnic. Or take sandwiches back to Brant Point for a picnic there.

You should also get ice cream at the Juice Bar. I love their blackberry ice cream in a waffle cone, but everything is good.

Boarding House/ Pearl same chef/owner is nice ,,Ships Inn as well ..Todd English opening this weekend downtown,,,,,,,,, American Seasons has a nice menu .. Drive to Wauwinet for a drink,,or lunch..As a former Islander, no trip is complete w/out stopping by the Chicken Box for a cold one Let me know how the White Elephant is I worked there years ago. Enjoy the Island it is a special place,,Chow P.S. make sure to drive out to Madaket for sunset...

I think Brotherhood of Thieves has gone downhill considerably since new owners took it over several years ago (after the fire that closed it down for a while). Definitely overpriced for what it is, even for Nantucket.

A better low key, causal option might either Centre St. Bistro or Nantucket Lobster Trap. But it's been a few years since I've eaten at either of those places.