Clothes may not necessarily make the man, but they sure make memories. In her new book, Worn Stories, Emily Spivack compiles reflections from Rosanne Cash, Piper Kerman, Marcus Samuelsson and others about the meaningful articles of clothing stored in their closets.

"I asked them to look for something that they couldn't part with," she tells NPR's Scott Simon. "Something that held some memory, whether it was something spectacular, momentous, wonderful, unusual that happened to them while they were wearing that piece of clothing."

Simon Doonan, creative ambassador-at-large for Barney's, and one of the contributors to the book, also joined the conversation.

Simon Doonan used to wear these bike shorts to Jane Fonda's aerobics classes in the early 1980s.

Ally Lindsay/Courtesy of Princeton Architectural Press

Doonan: When Emily called me it wasn't hard for me to figure out which garment I might choose because I'd actually only kept one thing: these strange, mirror-printed, fluorescent Lycra bicycle shorts from the early '80s. And I've been unable to part with them for a variety of reasons, which I spelled out to Emily. ... These shorts, I used to wear them to Jane Fonda's aerobics classes during a time when the AIDS epidemic was just really gathering terrifying momentum, where there was no solutions, no cure. And so this piece of clothing holds both frivolity — superficiality, aerobics — and also an incredible reminder of this horrible dark period in the '80s before there were any kind of options for people that were HIV positive.

On deciding what to keep when you're feeling sentimental about a loved one's clothes

Doonan: I always say, 'Well, pick one thing that somehow epitomizes that person.' And I think that's — the spirit of what Emily's done is really great, 'cause one thing can really epitomize a person, you don't need to keep their entire inventory of stuff.

Spivack: And that's been, actually, one of the interesting things about this project, having all the garments sent to me and then having them photographed. They all have some sense of wear: the rips, the tears, the love that's gone into wearing them. You could really just get the sense that there was a human touch.

This is the suit that Orange Is the New Black author Piper Kerman wore to court on the day she was sentenced.

Ally Lindsay/Courtesy of Princeton Architectural Press

On Spivack's flip-flops

Spivack: That story is really about the mundane in our wardrobe that actually comes to mean something over time. So, they're a pair of completely nondescript black, rubber flip-flops that I've had for almost 17 years at this point and so there's just something about the comfort that comes along with having a few things that just have kind of been with you for a long time.

On how she'd feel if something were to happen to her favorite garments

Spivack: What I've found is that by actually documenting it, by writing the story, by collecting the story, I think I would feel like, 'OK, at least I've got this story. At least I've documented this garment and the anecdotes that went along with it.' And I think that that's a lot of what people experienced when they were telling me their stories, that at least the story has been documented.

On the suit Piper Kerman, author of the book Orange is the New Black, wore in court the day she was sentenced.

Spivack: Her attorney said, "You want to wear something that a judge is going to look at you and he can see his daughter, his niece, his cousin; he can relate to the person who's wearing that."

Emily Spivack is the creator of Threaded, Smithsonian's fashion history blog.

Ally Lindsay/Courtesy of Princeton Architectural Press

On how we use clothes to express ourselves

Doonan: I think clothing is nonverbal communication, so you can use that in any way, you can use it to repel people, attract them, seduce them, appall them or delight them. ... Today I'm wearing a very, very flamboyant flowery shirt and a pair of groovy jeans, a little narrow '60s tie and fluorescent, cerise pink New Balance sneakers. ... But this book is just a reminder that as superficial and noisy and crazy as this huge landscape of fashion has become, people's actual relationship to clothes is really nuanced and subtle and poetic and strange and I think it's a really good lesson.

Spivack: And I think also when you flip through the book you see these just very basic garments — a T-shirt, a pair of shorts — and yet they have such rich stories attached to them.