Why you should not remove a USB drive without unmounting it

Normal case some read/write operation might be in progress and you simply disconnect the drive. This will corrupt your data.

Also most of the operating systems use caching feature which means OS holds on to the operations to be performed on external drive till a minimum threshold of operations is reached post which it execute all those operation. This is basically for performance. When you eject your drive it basically instructs OS to flush the cache and complete any pending operations. Now if you eject the disk and operations are not flushed your data is lost.

So always eject/unmount your drive before removing it. Same for Android as well. You can go to

Settings -> Storage -> Unmount SD card

Don't just remove it while your phone is on. There might be apps on your SD card, images or many other thing that might be in use. This also means you don't have to turn off your phone to remove SD card. Simply unmount it. Adding screenshots below for better understanding -

It may take some time to unmount your SD card. Once it is unmounted you can plug it out. Also no restarts needed. Hope this helps!