According to the documentation, the Net::SSH.start method accepts a :non_interactive => true option which should make your test (code) fail instead of asking for a password interactively. Other, non-interactive authentication options, such as the private key, will of course still work.