Behind most great performances is a skilled director—but in the case of “The Invisible Woman,” Ralph Fiennes directed himself as Charles Dickens having a final fling with the much-younger Nelly Ternan (Felicity Jones).

“What I love as an actor is when a director encourages me to explore stuff, even if it ends up on the cutting room floor,” Fiennes says. But as the captain of the ship, he couldn’t afford the luxury of giving himself too much time. “With the clock ticking and the adrenaline and making sure the other actors are properly nurtured, it’s hard,” he admits. Luckily, because the producers originally approached him about both directing and starring as Dickens, Fiennes had months of preparation under his belt by the time cameras started rolling.

“I had weeks and weeks and weeks of speaking the lines,” he says, before adding that the hardest aspect of the process was the risk of losing his objectivity in a scene. Fiennes recalls a few moments in the middle of a scene when he realized that the moment wasn’t right. He points to his first scene as Dickens in the midst of rehearsals for the play he co-wrote with Wilkie Collins, “The Frozen Deep,” as a particularly difficult moment.

“Dickens’ first appearance in the film, that was quite complicated with the Steadicam and the coverage and trying to find the right sense of Dickens’ social, hospitable energy and for it not to be too much or too over the top,” Fiennes says. “He’s definitely very vital and energetic, and to get that right under pressure, I remember that being quite tricky.”

None of those moments of uncertainty reveal themselves in the finished product. Fiennes is charming, almost boyish as Dickens, whose desire for the beautiful, young Nell prompts him to perform some outlandishly boorish behavior: When an expensive gift meant for Nell is accidentally delivered to his wife, Dickens insists she personally deliver the box to its rightful owner. But though we may recoil from Dickens’ callous treatment of his longtime wife, we never hate him for it. The depth of his love for Nell is so consuming that not only do we understand his actions, we root for Nell to accept what little he can offer her in the way of respectability.

Though Fiennes is loath to try to explain his process too much—“There’s a point where words to describe the process are redundant,” he says—he does explain that in those moments of falseness he just kept going. “It’s all to do with emotion and interior search, and often it’s just having another go,” he says. Joan Washington, the film’s dialect coach, often told Fiennes to try the scene again faster, which he found helpful. “Sometimes it can be something quite banal that unlocks you, like entering the room faster. But it’s the mystery of all acting: What is the thing that releases an actor into being present and inside the moment? It’s a quest all your life, probably.”

As the director, Fiennes was able to watch his cast find that release in their own different ways, a process he found to be a relief. “You see all the levels of insecurity, anxieties, and then the actor suddenly finds the zone,” he says. “In a way it makes me kind of calmer. This is the challenge. Everyone goes through it, and you’ve just got to be kind of patient with yourself. So it’s just made me think, It’s OK.”