Workaround:
The vsdiffmerge.exe merge tool in Visual Studio 2017 is usually located here: C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer. You will need to create one additional folder named Ide inside the Team Explorer folder Then you can copy the vsdiffmerge.exe file from the Team Explorer folder to the newly created Ide folder. You can check the attached vsdiffmerge-path.png screenshot.
Then you can create the environment variable like this:
Variable name: VS120COMNTOOLS
Path: C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer\IDE