what type of paint do you use for 5 inch wide aluminum transition strip from carpet to vinyl plank floor next to carpet.

A paint designated for metal. You can spray it or brush it on. I'd cover it with a poly to give it a little more resistance to foot traffic.
If you have access to a powder coating facility, that would probably be the most wear resistant.
Ron