Hosted Tile Map Service is not automatically building cache tiles

Even though I have the setting checked to automatically cache tiles at the time you publish within an ArcGIS online organization account, it never does this. Every time I have to go to arcgis.com and open up the map service and manually create the cache tiles from there. Is there an additional setting or step I'm missing?