Brown's Diner in Nashville is a squat grey building with flaking paintwork and an enormous ugly satellite dish extruding from its roof. Parked beside it, on concrete slowly ceding to weeds, is a trio of Chevy pick-up trucks. Of course, this is the place the Black Keys suggested for lunch. Where else to meet a rock band whose career has been one long and dogged slowburn founded on a lack of pretension? Singer-guitarist Dan Auerbach and drummer Pat Carney are sitting side by side in a booth, scoffing cheeseburgers while appreciating the gathering of gnarly boozers at the jukebox in the corner. "Love this crew getting day-drunk in their tank tops," smiles Auerbach.

"That guy looks like he just got off tour in 1987," says Carney, gesturing at a particularly grizzled rocker, before quickly adding, "You have to be careful in Nashville about how loud you observe." Carney reveals that, once, he commented rather too loudly on a guy wearing bedazzled jeans at an American football game and ended up being chased into the arms of the nearest cop. The story has Auerbach shaking and wheezing with laughter.

Watching these two crack each other up in a scruffy roadside diner, I have to remind myself just how huge the Black Keys are these days. Now with eight albums and seven Grammys to their name, the ubiquity of their songs is such that they've probably soundtracked some part of your life even if you don't realise it. In 2012, tickets for their Madison Square Garden show sold out in 15 minutes. That's faster than Jay Z. The pair have lived in Nashville for the last four years, a period of their lives that comprises their best of times professionally and their worst of times personally. Both 34 years old, both now divorced, their upcoming album Turn Blue is freighted with the pain and the weariness of the past few years. As a result, it's the most sophisticated and affecting thing they've done.

In August, while they were recording Turn Blue in LA, Auerbach finalised his divorce from Stephanie Gonis, gaining temporary custody of their six-year-old daughter. It's been a painfully public break-up: Gonis made accusations of abuse, Auerbach claimed she made suicide attempts in front of their daughter. The press gorged on the lurid details, including a lock of Bob Dylan's hair that Gonis won in the settlement. Today, his grief and exhaustion are palpable. "I've never relied on music to get me through anything like I did on this one," he says, quietly. "This past year, I realised how fortunate I was to have this thing I can do: making music, and lyrically, saying things that are personal. I do know that, subconsciously, sadness can have a huge effect on an album. With this record, I didn't try to push it in any direction but it came out pretty…" he trails off.

The word he eventually settles on is "moody", but I'd offer "melancholy", not an emotion that the Black Keys' rootsy, brawny grooves previously gave much credence to. Turn Blue begins with their longest-ever track, the seven-minute Weight Of Love, but most startling is the world-weary In Our Prime, which opens with minor piano chords and the words, "Pour me down the drain, I disappear/ Like every honest thing I used to hear".

"It's not an instantly gratifying record of quick, fast pop songs like El Camino was," emphasises Auerbach. "But over the course of years and decades hopefully it's going to be as pleasing as a record like this can be."

"I want people to like our music," adds Carney, "but I don't want to make music we know they're gonna like just to make music we know they're gonna like."

Auerbach and Carney have known each other since attending the same school in Akron, Ohio. Auerbach was captain of the American football team, a jock, albeit a long-haired, stoner jock with a penchant for bluesman Junior Kimbrough; Carney, by contrast, was a lanky nerd who'd geek out over Television, the Stooges and the Stones. They only connected when their older brothers suggested they get together to make music. The first time they jammed in Carney's basement, they improvised a six-track demo, a copy of which found its way to LA label Alive Naturalsound, which signed them without having seen them play. Their debut, The Big Come Up, was released in 2002.

For the next few years, the Black Keys slugged it out on the toilet circuit, but even as the Strokes-fuelled garage rock revival gathered momentum, significant radio play and media attention escaped them. It took a car commercial to jump-start their career: in 2006 they decided to license early song Set You Free to Nissan and, since then, their songs have helped sell more bras, cars and TV dramas than any band around. Their cheerful pragmatism over licensing music to adverts extends to appearing on The Colbert Report in 2011 for a "sell-out off" against Vampire Weekend's Ezra Koenig (Colbert's adjudication: "Clearly you have equally whored out your music").

The band at the Bottle Rock Music festival last year. Photograph: Tim Mosenfelder/Getty

Brothers, their sixth and tightest album, lifted them to a new level of commercial success, partly thanks to producer Brian Burton AKA Danger Mouse, who's become the Black Keys' honorary third member. That was followed by 2011's hugely successful El Camino. "We went to arena graduate school for that record," jokes Carney. "It was figuring out how to play 11 songs in front of a crowd who aren't familiar with our previous records." The Black Keys are not underdogs any more, but they maintain a scrappiness of ethos. They're hilariously frank, for example, about not being "proper musicians". "We don't know how to play our album," says Carney cheerfully. "We never do."

"The last album," adds Auerbach, "we hired our friends to come play with us on tour and the rehearsals were basically them teaching us how to play our songs." To this day, neither of them can read music. "I always loved how Pat did absolutely nothing by the book," he continues. "He's never taken a drum lesson in his life. Technically he's terrible, but, you know, I'm not trying to hear a good technical drummer, that would put me to sleep."

By rights, they should be living it up in their Bel-Air mansions by now, but they chose to settle in Nashville because it reminded them of home. "If we lived in New York we'd have a lot of people kissing our asses," says Auerbach. "If we lived in LA it would be like a constant scenester thing. Nashville feels to me a lot like Akron where we grew up; it's a small town."

Of their enduring relationship, Carney says: "I'm the goofball younger brother and that's the way it's always been. I feel like I'm 16, I act like I'm 14."

I turn to Auerbach: what about him? "I feel old," he says, forcing out a pained laugh. "Today I do."It's around this point that Auerbach politely, wearily takes his leave. Once he's gone, Carney becomes sincere and voluble in considering his bandmate's recent travails. "It's hard to break up with somebody you love," he says. "It just sucks. It was really affecting, and this is where the record comes from."

Carney knows how he feels, having divorced from writer Denise Grollmus in 2007. "The only thing I had left was my family and Dan. I put more effort into the band than I did my marriage; that's a straight-up fact and that's why it failed. I wanted the band to work. But I never thought it would actually work work." When success came, it overwhelmed him. It began at Lollapalooza in 2010 with paralysing stagefright. "I really felt like we couldn't deliver the goods to 50,000 people in broad daylight. And you know what the answer to that is? You can't. I went to this hypnotist in LA called Kerry Gaynor – I would never drop someone's name but he fucking saved my fucking career – and he was like: 'Dude, rock'n'roll is supposed to be sloppy.'"

Dan Auerbach doing his thing. Photograph: Tim Mosenfelder/Getty

Speaking of rock'n'roll, it's hard to talk about the Black Keys without also mentioning that other famous blues-rock duo, the White Stripes. Auerbach and Carney have never met Jack White but a decade of comparison to their fellow Nashville resident has become a kind of spectre for them. White has always been vocal about his dislike, telling Rolling Stone in 2010, "I'm a lot more to do with Jay Z than… with the Black Keys", but last summer saw another chapter added to the feud. Court documents surrounding White's own divorce leaked, including an email in which he referred to Auerbach as a copycat "asshole".

Auerbach has disappeared before I can broach the subject, but Carney is equable. "Those emails were obviously private communications not meant for us to see," he says. "When you're a musician you're in a state of perpetual adolescence. I just feel bad for people who are really stuck in that state. I'm not saying Jack White is, but I know that a lot of times it leads to this inflated ego thing. And honestly, I feel bad for him, I really do, because I've said a lot of dumb, fucked-up shit and if it ever came out I'd be mortified."

That "dumb shit" includes last year's Twitter spat with Justin Bieber which began when Carney remarked on Bieber's lack of Grammy nominations ("He's making a lot of money. He should be happy, I guess"). Bieber parried with, "the black keys drummer should be slapped around haha" and the Beliebers piled in. "It freaked me out," says Carney. "I don't want Justin Bieber sticking 40 million 12-year-olds on me!"

Apart from the vague threat of being set upon by an army of enraged Beliebers, Carney is happy that he and Auerbach have been able overcome their woes, channelling them productively into a compelling new album. "At this point, after five years and the extreme emotional things that have happened, our career going from the bottom to this point where we never imagined it being, I have no expectations for this record. I hope Dan and I are sitting at this table with you 10 years from now. But if it all ended tomorrow, honestly, I'd feel at peace with the world." Which may well be one of those slightly disingenuous musician cliches, only in this instance, I truly believe him.Turn Blue is out on 12 May; the Black Keys headline the Latitude festival, Southwold on 20 July