Please check following:
- Firewall is turned off
- WMI service is running on the target server
- RPC service is running on target server

Also please perform the following test:
- Run Services snap-in on Service monitor host
- Select Actions
- Connect to another computer
- Connect to target server with account that is using by Service monitor
Does it connect successfully?

You can check the Tracing folder in the Service Monitor installation directory for logs.