A few months ago, Veeam released a free, full featured utility to quickly and securely transfer data from on-premises to Azure Virtual Machines without the need for a VPN, or a remote desktop connection.

Some of the features include:

Secure file copy over HTTPS with built-in encryption (no VPN needed)

Manual file copy to/from Azure VMs without the need to keep the UI open until the file copy completes