The goal:

I want to stream video (i.e., to Twitch) of the graphical output of a Java program I'm running on a headless Linux box. I also want to be able to start this up remotely and without a monitor attached, e.g. so I can do a sudo reboot over SSH and then start it up again using SSH after the reboot.

I've made substantial progress, and the first part essentially works. At this point, however, it's getting too deep into Linux for me to have a useful mental model of all the pieces, so I'd appreciate some advice on what to try next. It seems to me the issue is somewhere around the login process and/or login manager and how it relates to the X server, and maybe distinctions between the console user and virtual users.

The problem:

If I attach the machine to a monitor and keyboard and log in, streaming works as expected. I can start the application and the stream over SSH by running (in different screen sessions):

DISPLAY=:0 ./gradlew viz
DISPLAY=:0 obs --startstreaming

However, if I don't log in, those applications will fail, along with glxinfo.

If I run DISPLAY=:0 glxinfo:

No protocol specified
Error: unable to open display :0

(The first line doesn't appear when trying to use a display that doesn't exist, so :0 is there but not working for some reason. After logging in, glxinfo works as expected.)

In the Java program, I get:

Exception in thread "main" java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
(etc.)

And obs gives me:

No protocol specified
QXcbConnection: Could not connect to display :0
Aborted (core dumped)

As I've mentioned before, I'd really like to avoid any routine steps where I have to be at the physical location of the computer, like typing into a keyboard attached to it.

Details about my current setup:

I'm working with a physical machine, not a VM. It's running Ubuntu Server 16.04.

The box's primary purpose is to run this program and some other programs I've written. I'm willing to significantly change the installed software to get this working, if necessary. Reinstalling or replacing the OS is an option, albeit annoying.

I'm using OBS Studio for Linux to do the streaming. It recently added a --startstreaming command-line argument, so after configuring the stream once with a monitor attached, I can start the stream remotely whenever I want.

OBS Studio on Linux requires support for OpenGL 3.2, so I repurposed a GeForce 9500 GT for the computer. This supports the requisite version of OpenGL as long as I'm using the NVidia proprietary drivers ('nvidia-current').

I bought a "fake monitor" HDMI dongle, and it seems to work, so don't worry about software solutions for making the computer think a monitor is attached. (When I start up the computer with either it or an actual monitor attached, ps aux shows an Xorg process with display :0; when it starts up with neither attached, the process is absent.)

EDIT: Outcome

This has been hacked into a more usable state. I'm sure there are better solutions out there, but I no longer need physical access to the streaming box.

First I removed xdm. Then I added a script that runs "startx" as part of startup. The script apparently runs as root, which causes Xauthority issues and error messages about MIT-MAGIC-COOKIEs; so I have the script run startx with XAUTHORITY set to /home/myUsername/.Xauthority. This requires one more workaround; I have to sudo chown the .Xauthority file back to my user after startup (its ownership gets set to root). After this, I can run programs in DISPLAY :0 as expected.

1 Answer
1

I can recommend reading the ArchLinux wiki pages on this subject, as it will have much in common with a recent Ubuntu now that both use systemd. There are a lot of new problems introduced by having session permissions, and being able to run the X server without it being setuid root.

Start by setting up a auto login to a VT, then run startx or xinit from there. Other pages to read are on the Xorg server and the display manager and systemd user. Also look for the keyword kiosk mode as it is similar to what you want. I've not yet done any of this so cannot provide more details.