It is my understanding that the government has to hand over all evidence it has if they are charging you with a crime.

Your understanding is incorrect. The government is constitutionally required to disclose information (not just "evidence") which is the hands of the prosecution team which is exculpatory (favorable to the defendant). It gets a bit complicated as to what constitutes the "prosecution team." It would include, for example, all exculpatory information in the hands of the police officer investigating the crime even if the prosecutor never learns of it. There are other cases where knowledge is not imputed to the prosecutor when the connection between the prosecutor and the person/agency holding the information becomes more attenuated. A discovery motion directing attention to specific agencies and/or specific types of information is advisable when possible.

The federal government and most (all?) states have additional rules which require some sorts of information be turned over to defendants. See Federal Rule of Criminal Procedure 16 for what the federal government must disclose.