What happens is that when you build in an x86_64 environment, some files are put in lib and lib/mysql/plugin, instead of lib64 and lib64/mysql/plugin. The %files section then fails, because it's looking for the lib64 location, using the %{_libdir} variable.