Prince William and Kate Middleton are frequently praised for their parenting skills. However, other British royal family members receive less than stellar report cards in the parenting department. William’s own father, Prince Charles, got mixed reviews as a father figure over the years. But that doesn’t compare to Prince Charles’ relationship with his parents, Queen Elizabeth and Prince Philip.

What was it like having the heads of the monarchy as parents? Royal family secrets reveal the rocky relationship Prince Charles had with them growing up. (Page 10 in particular may shock you.)

1. Mostly smiles now

Everyone seems to get along these days. | Chris Jackson/Getty Images

Recent photos have captured warm, even silly, interactions between Prince Charles and his mother, the reigning queen of England. There have been suggestions that their relationship has gotten better over the years, and that Queen Elizabeth has become more easy-going in her old age.

Next: Things weren’t always so rosy and wonderful.

2. The parents who weren’t there

As Town & Country summarizes, Queen Elizabeth and the Duke of Edinburgh were not around much in Charles’ early life. Charles was only a couple years old when his mother became queen. She quickly became very busy with royal duties and traveling. The royal family even took a six-month tour of the Commonwealth countries in 1954. Charles about five years old at the time.

Next: Even when his parents were there, they didn’t exactly raise him.

3. Who really raised Prince Charles?

Town & Country explains that Prince Charles and his younger siblings were primarily raised by nannies, whether they were at home or traveling with their parents. It is thought that Charles had an especially close relationship with his grandmother, the Queen Mother.

Next: Speaking of…

4. The Queen Mother’s role

It’s no secret that the Queen Mother was close with her grandchildren. But it has been widely reported that she stepped in as a mother figure for Charles while Queen Elizabeth was reportedly not available. At the Queen Mother’s funeral, Prince Charles said that “she meant the world” to him and that he had “dreaded” her passing.

Next: Back to his actual mother…

5. Was the queen really that distant?

Prince Charles might not have had the same opportunities to get close to his mother. | Roger Jackson/Central Press/Getty Images

We have to ask this question because Prince Charles’ take on the queen as a mother is very different from that of his siblings. Princess Anne was quoted as saying: “I simply don’t believe there is any evidence what so ever to suggest that she wasn’t caring as much as it was needed. It just beggars belief.”

Next: So we must ask next…

6. Why was their relationship so strained?

He also spent time away in boarding school. | Keystone/Hulton Archive/Getty Images

It’s possible that Prince Charles has such a different view of how Queen Elizabeth was as a mother because she became queen so soon after he was born. She reportedly became closer to Anne when the Princess was a teenager. And since Charles’ younger brothers, Princes Andrew and Edward, are much younger, it’s believed their relationship with Elizabeth is better.

Next: Which leaves us to wonder…

7. What about his relationship with his father?

Prince Phillip might have had his own reasons for being a distant father. | Central Press/Hulton Archive/Getty Images

If there’s anything that all parties can agree on, it’s Prince Charles’ fractured relationship with his father, Prince Philip. The Duke of Edinburgh has even said himself that he and his oldest son are very different people, and their polarizing characters made them distant. It’s also believed that Philip’s own troubled childhood affected how he was as a father.

Next: The most shocking aspect of their relationship?

8. Prince Philip only communicated by letter

Possibly the most shocking revelation about this father-son dynamic is that Prince Philip didn’t have heart-to-heart talks with his eldest son. Heck, they didn’t even have many face-to-face discussions. Instead, Prince Philip allegedly sent Prince Charles hand-written letters to get a point across.

Next: One area where he felt pressure from both parents was…

9. When it came to marriage

They strongly encouraged him to tie the knot. | Patrick Riviere/Getty Images

Regardless of what the relationship was like with his parents, it’s widely believed that they both pushed Prince Charles into getting married. Both Queen Elizabeth and Prince Philip were supposedly growing weary of Charles’ bachelor ways and were more insistent that he commit to his girlfriend, Lady Diana Spencer. Prince Philip reportedly wrote a letter to Charles telling him to marry Diana or let her go.

Next: Philip’s role in Charles’ first marriage.

10. The infamous ‘mistress’ quote

It’s no secret that Prince Charles and Princess Diana had a tumultuous marriage. One of the most obvious problems was that Charles was still in love with his old flame, Camilla Parker-Bowles, and had an affair with her. It was later reported that Prince Philip gave him permission to have a mistress, saying that if things didn’t work out with Diana that he could start seeing Camilla on the side.

Next: How the parental problems trickled down to the wives.

11. Did the familial problems affect Diana?

It’s possible that Charles’ fractured relationship with his parents had an effect on their relationship with Princess Diana. There have certainly been plenty of reports that Princess Diana didn’t have a good relationship with Queen Elizabeth. The reports of who Diana got along with Prince Philip vary a great deal, from reports of epic feuds to talk that Philip tried to help Diana after the divorce.

Next: When it came to Prince Charles’ second wife…

12. How it affected Camilla

There was reportedly an undeniable rift between Prince Charles and his mother when it came to Diana’s death, and his subsequent relationship with Camilla. Queen Elizabeth reportedly didn’t want Charles to pursue a relationship with Camilla following his divorce from Diana. However, when he was mourning Diana’s tragic death, the queen was reportedly cold towards him and left the comforting to Camilla.

Next: Since we mentioned the other heirs earlier…

13. How it affected Charles’ siblings

His experience with his parents could very well have been different from his other siblings’. | AFP/Getty Images

Prince Charles’ comments about his relationship with his parents have no doubt drawn the ire of his three younger siblings. However, it’s more likely just sibling rivalry than anything else. We already pointed to Princess Anne’s rebuttal to the comments that Elizabeth was a cold mother — but, keep in mind, she and Prince Charles don’t agree on most things.

Next: And what about the siblings that Charles raised himself?

14. Did it shape Prince Charles as a father?

After a cold childhood, he might have wanted to be a nurturing father. | Joel Robine/AFP/Getty Images

Sure there was speculation that the Prince of Wales himself was a bit emotionally distant as a father. However, Princes William and Harry have always said good things about their relationship with their dad. Charles was reportedly a very hands-on father, even when he was away for royal duties. He also tried his best to fill a duel parent roll after Diana past away.

Next: Speaking of playing parent for William and Harry…

15. Elizabeth and Philip as grandparents

Whatever has been said about Prince Charles’ relationship with his parents, there’s no denying that they stepped up when it came to being grandparents. Queen Elizabeth was reportedly very present for Princes William and Harry as they were growing up, especially since they no longer had a mother. Even Prince Philip has noticeably filled in the grandparent role well. (Which certainly must make their relationship with Prince Charles a little better, right?)

Next: The next generation

16. Adjusting to the latest additions

Royal family | Kensington Palace via Twitter

As the royal family continues to expand, we’ve been getting even more details on the new family dynamic. Both Queen Elizabeth and Prince Philip have excellent relationships with their granddaughters-in-law, Kate Middleton and Meghan Markle. The heads of the royal family also reportedly have a very sweet relationship with their great-grandchildren, too.