Despite the name, the Goblin Thief does not actually steal anything from the player's chests, piggy banks, etc. One of the Goblin Tinkerer's quotes explains this as a direct reference to Goblin Thieves.

Notes

Goblin Thieves (and Goblin Peons) have a hitbox that's only 2 blocks high and 1 block wide, meaning that they can squeeze through smaller holes than Zombies or the player can. (which are both 3 blocks tall and 1.5 blocks wide)

The Goblin Thief and Goblin Sorcerer have the same head textures, except the Goblin Sorcerer's head color is a bit brighter. (Pre 1.4)