You neglect to mention which SDcard socket you are using to boot from.
That is a detail that has to be configured in the builds of both AT91Bootstrap and U-Boot.

Apparently the default U-Boot environment does not match the boot medium that you are actually using.
Either
(a) rebuild U-Boot with a properly defined CONFIG_ENV_FAT_DEVICE_AND_PART that reflects the intended boot device, or
(b) edit the bootcmd variable in the U-Boot environment.