A source tells PEOPLE that West, who was hospitalized last Monday for exhaustion and sleep deprivation, has a history of episodes in which he would act erratically. However, his wife’s absence while he was touring due to her terrifying robbery at gunpoint in Paris caused him to have trouble rebounding.

You'll get the latest updates on this topic in your browser notifications.

“In the past, he snapped back quickly. This has been going on for years,” the insider says of the rapper’s history of personal struggles and erratic behavior. “In the past when Kanye seemed to struggle, no one took it too seriously. The attitude was more like it was just Kanye being Kanye.” After going through a few rough days, the source says “things would go back to normal. This time is much different.”

The source continues, “Without having Kim by his side during the tour as planned, Kanye wasn’t able to snap back. Kim being absent because of her own struggles seems to be why things are different with Kanye this time.”

The source says the 39-year-old rapper is still being treated at UCLA, and Kardashian West, 36, is spending as much time as possible with him. The reality star spent several hours with him in the hospital on Thanksgiving morning before heading to dinner at Kylie Jenner‘s Calabasas home.

“Kim is focused on Kanye right now and making sure her family is getting all the care and attention that’s needed,” a source close to the Kardashians tells PEOPLE. “They’re her priority and main concern right now.”

According to the insider, the couple’s two kids — daughter North, 3, and son Saint, 11 months — haven’t visited their father.

Another source previously told PEOPLE that West, whose hospitalization came shortly after he canceled the remaining 21 dates of his Saint Pablo tour, was “extremely sleep deprived and had not been taking care of himself.”

On Wednesday the insider said that West, 39, is “starting to feel more like himself.”

“He understands that getting help at the hospital was necessary,” said the source. “He is resting and getting help. They are taking it day by day.”