It was forced on because upstream had it automagic. Patch submitted and
accepted upstream. Also add X USE flag and make it synchronized with
clutter since there is conditional code being built depending on whether
clutter has X11 support or not ending up in clutter-gst linking to
libX11.