Def Jam founder Russell Simmons put out a statement on Thursday, soon after screenwriter Jenny Lumet penned a column accusing him of a nonconsensual sexual encounter in 1991. Simmons said that Lumet’s memory is “very different from mine,” but did not deny that the incident took place.

Lumet’s column and Simmons’ response were published by The Hollywood Reporter Thursday morning. She described how Simmons offered her a ride as they left a restaurant where a group had been hanging out, but when she gave the driver her address, Simmons said, “No,” before the driver took them to Simmons’ place. Lumet wrote that she was scared, as Simmons made physical advances, ignoring her when she said, “Wait.” The two allegedly had sex, and Lumet said she was frightened but did not say no.

Lumet says the incident took place in New York. Under the sex crimes laws in that state, the alleged behavior would probably not have been considered rape, as the statute requires forcible compulsion. It could, however, be considered sexual abuse, a misdemeanor that includes sexual contact where the victim “does not expressly or impliedly acquiesce” to the contact.

Simmons reacted to Lumet’s account by saying:

I have been informed with great anguish of Jenny Lumet’s recollection about our night together in 1991. I know Jenny and her family and have seen her several times over the years since the evening she described. While her memory of that evening is very different from mine, it is now clear to me that her feelings of fear and intimidation are real. While I have never been violent, I have been thoughtless and insensitive in some of my relationships over many decades and I sincerely apologize.

Simmons went on to say that he is stepping down from Def Jam and his other businesses, saying, “I don’t want to be a distraction.” He added, “I will step aside and commit myself to continuing my personal growth, spiritual learning and above all to listening.”

This is not the first accusation that has been made against Simmons. A recent Los Angeles Times article included allegations from model Keri Claussen Khalighi that he took off her clothes and forced her into oral sex while filmmaker Brett Ratner was present. Both Simmons and Ratner have denied these allegations.