I have this kind of problem:
I have a testcase, where SUT license data will be modifed -> button in SUT is pressed and license modification query is shown. This will stop the SUT process and new process (License_update) will be started. After new license is entered and continued, (License_update) process is closed SUT will be started by this (License_update).

According to software developer:
Altiris\Wise\Virtual Package Editor
Check in use File - method

is used to check if the SUT is stopped when License_update can be executed.

If I now execute my Ranorex testcase (Project name is R_SUT.exe)
(- could there be a problem when I am using almoust the same name in my project than the SUT?)

-> SUT will inform, that SUT is still running. When checked from task manager, SUT.exe is closed.

So does the Ranorex (Project name R_SUT.exe) somehow keep the SUT running and prevents me to execute the test?

If I modify my testcase so, that testrun is stopped immediately after License update is selected (-> so R_SUT.exe is closed and SUT is closed), everything work fine. (only that I would like to continue the test run from here )

With a .Net Application Ranorex loads the DLLs of your Application (because we hook into the application) and the DLLs stay locked until the Ranorex executable stops. So, the application is not "hold open" by Ranorex, but the DLLs are used and can therefore not be deleted by your package manager. A possible solution for your problem would be to start another test at the end of your first test.

Thank you for the answer.
Sure there is some very simple solution , but how can I solve this:

I am running my automated testset with NUnit (started with automation.bat). When this testcase is under execution (the one where SUT should be closed by License query).
Should the nunit-testrun be stopped and someway modify this automation.bat that the rest of the cases can also be executed after that?

Very lost here , maybe easiest to just drop that case from automation set

Just split your project to two assemblies and execute in the first assembly the steps including the step where you exit you Application. Place the rest of your tests to the second assembly.Then change the NUnit Settings to "Run tests in a separate process per Assembly" and now you should be able to run your project without any errors.