The wallet file may be encrypted with a passphrase however the wallet file contains the private keys associated to addresses in that wallet. Without the private keys you can never spend the bitcoin in those addresses.

It is all the magic of asymetric cryptography and digital signatures. A private key is used to sign data, and a public key is used to verify the data was signed by the appropriate person.

Otherwise they might be saving all wallet login information into the hardware itself and don't have another copy of it :) even there's some people create special email address for the wallet and save it in text inside the hardware.

It sounds like maybe you've been using a wallet like Electrum, which allows the recovery of the wallet from a passphrase if the wallet itself gets lost/destroyed. This works by using the randomly generated passphrase as the seed to create the keys used for 'storing' bitcoin. Not all wallets support this functionality, instead generating the seed completely randomly instead of generating it from a passphrase. Back when these people had a large number of bitcoins there may have been no wallets with this feature.