Q: Does a virtual machine (VM) enabled with RemoteFX only use graphical memory, or does it also use system memory from my Hyper-V host?

A: When a VM is started, memory is allocated to the VM equivalent to the memory configuration of the VM. However, additional memory is also used for each VM to handle other resources of the VM, such as processes that look after the VM.

When a VM is RemoteFX enabled, a certain portion of the GPU memory is reserved for the VM, based on the number of displays configured and the resolution specified. Additionally, a certain amount of system memory is also allocated to the VM. So the answer is that a RemoteFX-enabled VM gets allocated GPU memory and additional system memory to function. The amount of system memory allocated varies between around 85MB and 130MB. Microsoft provides a capacity planning guide that gives some additional information.