How in the world does this setup work, I cannot for the life of me figure out how this stays on the motor shaft securely.
Theres no place to tighten down the adapter, I know this is probably really simple to do.

And Im guessing these 9x6 props from the brushed motor need to be drilled to fit? Or just buy the props that fit correctly?

Yeah, some props must be drilled to fit. The APC-E Electric props come with a set of interchangeable "Bushings" that adapt to the size of your motor's shaft. Or, if you have a Harbor Freight store in your area, check out their supply of "Number" drills, or their complete set of fractional, letter and number drills. These are of fair quality, but entirely sufficient to handle this type of project. If you have one of those cheap drill presses, this will make the drilling out the hub far easier.

As for that prop adapter, the double clam shell type of bushing is several orders of magnitude better than those cheap, simple set screw type of prop adapter.