Eventually pure QML ends up invoking code written in C++, perhaps with GLSL and assembly language thrown in. You'll never beat going directly to the source with added layers of indirection.

Whether the added cost of convenience is worth the performance penalty is something for you to decide. If you have access to target devices, benchmark a few implementations and decide which works for the use case. If they're close enough to equivalent, consider which is easier to work with for the likely set of maintainers. Another angle is to examine which is easier to adapt for reuse.