Now open IIS management and recycle the MSExchangeAutodiscoverAppPool application pool.

If that doesn't help, next guess would be TMG is trying to reverse proxy Exchange, in which case I would change all the external EWS details to internal, because that's what people will be being served (same procedure as above but set everything to internal URLs).

-Or- Don't use TMG to proxy EWS, because I am struggling to see a point to that, anyway?

Managing Active Directory does not always have to be complicated. If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

Not to familiar with split-brain DNS.
Both internal and external have contoso.com
We have DNS servers in our LAN, and a public DNS.

What you are describing here is split-brain DNS. That your external and internal URLs all match. Furthermore than your Public DNS points to the public IP of your CAS. Then you have recreated the contoso.com domain internally and the same records points to the internal IPs of the server, as opposed to the public IPs.

All URL's are set up for external services, with exception of Outlook Anywhere.
We don't have Outlook Anywhere set up, should that make any difference for EWS ?

Outlook Anywhere isn't a factor. But the addresses you have for EWS does.

Run this command that HostOne mentioned to make the internals and externals match for EWS.
Set-WebServicesVirtualDirectory -Identity "CAS_Server_Name\EWS (Default Web Site)" -InternalUrl https://mail.contoso.com/ews/exchange.asmx