We have a custom page for nonexistent content - a styled page saying "Content not found, if you believe you got here by error, please contact us", with a few internal links, served (naturally) with a 200 OK. The URL is served directly (no redirection to a single url).

I am afraid this may discriminate the site at google - they may not interpret the user friendly page as a 404 - not found and may think we are trying to fake something and provide duplicate content.

How should I proceed to ensure that google will not think the site is bogus while providing user friendly message to users in case they click on dead links by accident?

You can still send a 404 status and provide user-friendly messages for dead links in the same response. Even "normal users" should get the 404 status even if the page doesn't look like your typical failure page. How you intercept the request depends on your webserver. That's going to be a lot easier than detecting the user-agent and doing something different for Googlebot.