The physical manifestation of Third Man Records—on 7th Avenue South in Nashville—has existed only since 2009, though it seems like the realization of a lifelong dream. Jack White had been using the Third Man name since he was an unknown upholsterer in Detroit, and it became the name of his record label in the early 2000s, then finally a warehouse/store/performance space/novelty emporium/magic factory just a few years ago.

Third Man is divided into a bunch of discrete spaces: The record store, which sells Third Man releases and novelties almost exclusively, is tiny. Staffed by a crew that wears the required black and yellow, it stocks limited-edition vinyl, Third Man-branded turntables, and various White Stripes gear. Adjacent to the store in an even smaller room is the novelty emporium, which offers a Mold-a-rama (it spits out a wax reproduction of White’s signature guitar, but only after you feed it a Third Man coin), a photo booth, and a Scopitones machine that plays only Third Man films. The day we were there, the Record Booth was down for repairs: Normally it allows visitors to record their own poem or song onto vinyl, which is delivered in just a few minutes.

If this all sounds like an expensive playground, it kind of is. Clearly a ton of money and creative energy went into planning and designing this Wonka-like factory. Beyond the store, the company’s offices are gorgeous but not open for filming: There’s red and white subway tile, an old office door that now reads “John A. White III, D.D.S.” and a piece of ornate woodwork atop a door frame that looks like an old turntable. It’s incredible.

Beyond that, Third Man has a performance space that not only hosts concerts, but also serves as a photo studio—the walls are blue—a darkroom, and a recording studio. The night we arrived in Nashville, Aziz Ansari did an impromptu late-night stand-up set to a couple hundred people—it’s that kind of place. Performances can be lathe-cut live by a team of engineers who wear Third Man lab coats, and many of those performances are released on limited-edition vinyl. (I nabbed an LP by The Racontwoers, a slightly re-configured version of The Raconteurs.)

And then there’s the massive warehouse, again staffed by nattily dress-coded workers stuffing boxes full of Third Man releases. The space is made to look like an old motel, complete with false doors on a snaking balcony. It also houses the Third Man Rolling Record Store, a big yellow truck capable of selling records and serving as a mobile stage, complete with a P.A. The truck had just returned from an outing when we visited, and was stocked with limited-edition tchotchkes like Third Man rattlesnake eggs and Third Man popguns.

In other words, it’s a hell of a place. Thanks to Ben Swank—whose title is Third Man Consigliere—for talking to us and for showing us around, though he was very busy trying to figure out how to laser-cut wooden boxes and gold-plate vinyl for a super-deluxe Great Gatsby soundtrack.