At work, I have remote access to any computer in the company, because I deployed a system to do so. If you are at work, or a public place (Library, College campus, etc) on a PC you don't own, that's probably what happened. If it's your personal device, regardless of what network you are on (as long as you are not joined to a domain), then you should be concerned.

Hackers usually try to hide their presence so that they don't get detected and removed, so no, obvious computer takeover is not very common. And yes, I would immediately kill the Internet and start looking in my light-weight task manager that shows hidden processes. I would probably use Processes Hacker, too.