Does the Off-The-Grid Maps (SD card) always need to be in my Android phone?

Yes. In order to see maps in the field, you need to keep the Off-The-Grid Maps (SD card) installed in your Android phone. The Trimble GPS Hunt or Fish app directly accesses the memory card to bring in topo maps, private lands, public lands, and other helpful map overlays.