I have enabled X forwarding on remote machine where SSH server is running:

# grep -i forward /etc/ssh/sshd_config
X11Forwarding yes
#

On local machine, I have started SSH client with -X flag which instructs the SSH server, running on remote machine, to set up a X-server proxy. In addition, it creates the $DISPLAY variable which points to this proxy and calls the xauth to install a proxy key which authenticates to this X-server proxy on remote machine:

# wireshark
(wireshark:10083): Gtk-WARNING **: cannot open display: localhost:11.0
# xterm
Warning: This program is an suid-root program or is being run by the root user.
The full text of the error or warning message cannot be safely formatted
in this environment. You may get a more descriptive message by running the
program as a non-root user or by removing the suid bit on the executable.
xterm: Xt error: Can't open display: %s
#

X forwarding doesn't use xhost so at least this can be excluded. I tried to find some useful log entries both on machine where SSH server is running and machine where SSH client is running with find /var/log/ -mmin -5 -type f command, but this did not give any hints. SSH server version is OpenSSH_5.9p1 and SSH client version is OpenSSH_5.2p1. Output of /tmp/.X11-unix/ directory on remote machine can be seen below:

The # suggests you are logging in as root. If that is true, why?
– Faheem MithaJan 10 '14 at 9:36

5

Sometimes root is treated differently than other users. If possible, test with a normal user. In any case, doing things as root if you don't have to is generally a bad idea.
– Faheem MithaJan 10 '14 at 11:47

4

Also try using ssh -Y instead of ssh -X, that might help you get around authentication issues.
– terdon♦Jan 10 '14 at 12:18

1

@FaheemMitha I created a non-root user to a machine where SSH server is running and logged in to SSH server with this non-root user, but unfortunately this did not change anything. @terdon I tried with ssh -Y, but this did not help. I still receive the cannot open display error-message.
– MartinJan 10 '14 at 15:45

2

Adding the -v flag to ssh is often helpful in troubleshooting as well.
– alancJan 13 '14 at 21:27

As far as I know, xhost ACL is not even checked if X forwarding is done over SSH.
– MartinJan 15 '14 at 10:20

You are right, but for some reasons sometimes doesn't work as expected. you can also try this: - X11Forwarding yes X11DisplayOffset 10 X11UseLocalhost yes more on: X-Forwarding
– nbariJan 15 '14 at 10:27

First a point of debugging methodology, when you run xterm as root, it does not print any useful diagnostic messages, so when troubleshooting xterm do it as an ordinary user.

Now as to your problem, There Are two types of X11 forwarding, secure and privileged, and you probably did not enable ForwardX11Trusted yes. You don't need it on Debian and derivatives because they always enable it.

So some background on how we got here. X11 is not the most secure protocol. It was designed in friendlier times, and there have been a number of efforts to make it more secure over the years including running it through a secure shell tunnel. Tunneling X11 reduces almost all of X11's vulnerabilities. One that remains is the risk of displaying windows from untrusted systems. An effort was made to declare a safe subset of the protocol that would impede deployment of key-sniffers and similar. The project, although technically successful, has some real world challenges like the odd fact that finding programs that only use the safe subset is really hard because a lot of the disabled functionality is very handy, and since most people write for the local machine which has permission to use the full protocol there is little drive to create programs that work with the restricted subset.

The other possibility is that the X11 forwarding is only enabled on a specific host. The easiest way to check if this is the problem is to use the -Y flag with ssh to enable trusted X11 forwarding. If that solves it add both forward allow lines to the relevant host sections of your ssh config file.

On my (standard) Ubuntu environment I got the xterm over ssh failure message "... suid-root program..." (see above), even with all the proper forwarding settings.
This behavior went away a soon sshd is configured to use only IPv4, because of an X11 forwarding bug in SSH if IPv6 on the system is disabled.

I am using Debian "testing" and due to some updates my "hosts" file was changed. The line with the localhost entry was commented out. So your solution gave me the hint I needed. Thanks a lot.
– Winnie TiggerMay 18 '18 at 8:57

Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).