Windows Setup from x64 boot.wim can't access install.wim

I have created a USB multi Windows Installation disk which allows me to boot either x86 or x64 UEFI computers to install windows.

I used an x86 Windows 10 Installation CD and copied the entire contents of the CD to the USB drive. I renamed the boot.wim in the sources folder to boot32.wim. I then used an x64 Windows 10 Installation CD and copied the boot.wim file to the sources folder on the USB drive and named it boot64.bin. I then removed the install.wim file under sources and added my own install.wim file with merged Windows Installations. I modified my BCD boot entries to give me options to boot either the 64 or 32 bit boot.wim files.

In testing, I can boot to the 32bit boot.wim where setup starts and I can see the windows versions in my install.wim file. However, if I boot to the 64bit boot.wim setup will start but it cannot access the install.wim file and says that I need a CD/DVD driver.

I figured out that the reason it does not work is because I'm booted into the 64bit boot.wim but all the additional files in the sources folder are from the x86 Windows 10 setup disk. If I replace all the files on the USB under the sources folder with the files from the sources folder of the x64 installation CD, the install.wim file is able to be accessed under the 64bit boot.wim and installation can continue.

It seems the solution would be to get the files from the x86 and x64 sources folders of the installation CD merged into the single sources folder on the USB but I can't seem to get that to work.

Another option would be to have two different folders for each architecture and then point the setup to my install.wim but I'm not sure if that is possible.