The decision comes after several months of back-and-forth negotiations and court visits following an emergency restraining order filed by Mueller in March.

Mueller's mother, Moira Fiore, and Sheen's ex-wife, Denise Richards, with whom he also has two kids, have both offered to care for the twins as it seemed drugs were getting in the way of their parenting abilities.

Sheen most recently took Mueller to court in April, arguing he deserved sole custody because of her drug use. But the judge was reportedly not convinced, in part because of Sheen's own substance abuse issues.

The former "Two and Half Men" star and Mueller officially became divorced on May 2, one week before he switched gears and tried to change his custody setup with Richards.

Sheen, who hadn't been spending time with 7-year-old Sam and 6-year-old Lola due to his increasingly outlandish behavior, met with Richards and "neutral third parties" to argue for weekend visits, TMZ reported at the time. He was again denied.