TL:DR: For now I'm going to consider this a driver issue and move on. If anyone plans on using Mint w/Xen in Dom0 I'd advise against buying an AMD 7000 series card as you may experience issues like mine with the proprietary drivers (fglrx). Of course if you have one already then maybe you'll succeed where I failed

I really hesitate to post anything about my problem because I figure I'll get dismissive answers due to my unusual setup. However, I'm at my wits end after searching for a solution for a week so I'd appreciate any help/input/direction.

Disclaimer: Sorry if I sound mad or my tone is bad. It's merely frustration and it's not meant to be directed at anyone/thing. I just wish this worked.

Summary:My goal, as the title suggests, is to run the proprietary fglrx driver on Mint 14 in a Xen Dom0. Everything boots fine loading Mint 14 w/o Xen. Booting w/Xen only works if I pass the 'text' option to the kernel via grub (meaning no X). When attempting to load into X windows from xen I get a black screen (no cursor, no login sound, etc) and when comparing xorg.0.log's it seems the xen version just ends at the point where it would normally print details about the video card.

The first response to this sort of problem, as I've observed elsewhere, is "load the open source driver". I know that's an option and I've actually confirmed that it works (only with Mate). So what's the problem you ask? Support for my video cards seems to be a work in progress according to the xorg radeon feature matrix. 3D support is sketchy and there is no support for HDMI audio if I decide to use it. There is power management, my main concern, but it's said to introduce visual defects (dynPM). In an ideal world I'd be using the open source driver. Now that I got that out of the way some details.

Software:Mint 14 Cinnamon - I've since installed the Linux-meta-mate package due to another issue and have switched my default session to Mate. I'll probably back everything up and do a fresh install of Mate soon.Xen 4.1.3 (xen-system-amd64)fglrx (Amd's official version 13.1 at the moment. I first tried the packages available to mint.)Custom Kernel built from ubuntu git (branch 3.5.0-17.28)- Now before you point fingers I can confirm that this issue is the same regardless of which kernel I load - stock Mint or my custom. I built the custom because I needed kernel options not turned on in the Mint version. I'm happy to test things on the stock kernel. Just throwing that out there because you can see as much in my Xorg.0.log below.

and look for the "Kernel driver in use: fglrx_pci" entry for your HD 7700, and pciback for the HD 7800.

Here my reasoning:

You may have installed the fglrx driver while running the regular Linux Mint, not the Xen hypervisor. In that case the driver would see 2 graphics cards which may affect the configuration. So when you start Xen it won't work.

Another problem may be the blacklisting of the radeon driver. But I doubt this since you said it works under normal boot (non-Xen).

Finally there could be a problem with the proprietary driver attaching itself to the second graphics card while booting, before the pciback script is able to detach the card. If that is the case, you may need to include the xen-pciback module in the initramfs and run update-initramfs.

# List of modules that you want to include in your initramfs.# They will be loaded at boot time in the order below.## Syntax: module_name [args ...]## You must run update-initramfs(8) to effect this change.## Examples:## raid1# sd_modxen-pciback passthrough=1

and run sudo update-initramfs. It will run for a while and produce a new initrd.img... under /boot.

I'm not sure the following will work, but it may be worth a try. Edit the /etc/default/grub file to include:

Create the pciback.conf file. See how-to for all the steps up until part 3, point 9 - do everything except reboot until you are done with the Xen host configuration!

As you can see I've done much the same as you in my xen cmdline. As far as the pciback.conf I've already told you I chose the statically compiled route. If anything I should be removing any trace of pciback to see if it's causing a problem here. But I've already tried that. Sorry, but I think we should concentrate on one problem at a time. Right now I'm focused on getting a solution for fglrx in Dom0. Besides I've already confirmed that pciback works just fine as I've configured it.

3. Reboot into Xen and install the proprietary fglrx driver (I used the default one, not the latest). You can use:

You can see in my original post of my xorg.0.log that I had the radeon blacklisted on my Xen boot.

Note: Usually the fglrx driver should blacklist the radeon driver by placing a file (actually a link) named fglrx.conf into /etc/modprobe.d with the following content:

Yeah, I noticed that when I went to blacklist another module that was giving me problems. I did notice that after doing the grub blacklist that two errors I had been getting went away. One about /dev/fb0 not being found and the other about a DRI device. If I'm not mistaken that refers to a framebuffer and a direct rendering interface. Sadly they weren't the source of my problem as there is no mention of them in my Xen log.

However, you may have seen the posts in the how-to that this doesn't always work, so use the grub method. Don't forget to run update-grub after you edit the /etc/default/grub file!

Many a times I curse at myself for forgetting to do so, but out of habit I hit 'e' before selecting a grub menu item to make sure my changes took place.

and look for the "Kernel driver in use: fglrx_pci" entry for your HD 7700, and pciback for the HD 7800.

I'm not in Xen right now (don't have X with this driver installed after all) so I can't post my output directly. But I have verified this is the case.

Here my reasoning:You may have installed the fglrx driver while running the regular Linux Mint, not the Xen hypervisor. In that case the driver would see 2 graphics cards which may affect the configuration. So when you start Xen it won't work.Another problem may be the blacklisting of the radeon driver. But I doubt this since you said it works under normal boot (non-Xen).

Hmmm. I do believe I tried reinstalling fgrlx while in a Xen terminal and the first time I installed it I was in Dom0. But I'll try uninstalling and reinstalling everything one more time to be sure I'm not mistaken. Is there actually a config file besides xorg.conf that dictates which cards to use? I had assumed this wasn't the case because of the necessity of blacklisting drivers. I just assumed xorg was detecting all available hardware and matching them with all available drivers (well in a first come first serve manner).

Finally there could be a problem with the proprietary driver attaching itself to the second graphics card while booting, before the pciback script is able to detach the card. If that is the case, you may need to include the xen-pciback module in the initramfs and run update-initramfs.To do so, just edit the /etc/initramfs-tools/modules file as follows:

Well the thought of xorg trying to use the second card had crossed my mind. That's why I disabled pciback at one point to see if it was conflicting with the driver process.

I'm not sure the following will work, but it may be worth a try. Edit the /etc/default/grub file to include:

This is how I'm doing pciback. I've compiled the the module into the kernel so that I don't have to worry about the timing of the reservation. It happens as soon as the kernel loads.

Last not least: Read the ATI graphics driver how-tos and forum posts to see if there are some driver-related issues and solutions.

Believe me I've looked. The unnofficial AMD driver wiki seems to have the best howto's. Such as - http://wiki.cchtml.com/index.php/Ubuntu ... st.2Ffglrx Which I'll be using once again to do my reinstall. There are a lot of posts that come close to my situation but the solution still evades me. I'm thinking that the solution is either going to be very very simple and staring me in the face or I've stumbled on to a really unique issue.

What I did -0) Removed pci-back from grub to prove that it's not part of the boot issue.1) Boot into Xen with the 'text' option to disable X temporarily.2) Uninstalled the fglrx driver.3) Reinstalled the Radeon driver (open source).4) Removed 'text' and 'blacklist' from grub.5) Rebooted into Xen again.6) Grabbed a copy of my xorg.0.log just in case and confirmed Open source and X are still happy.7) Reinstalled Fglrx.8 ) Blacklisted Radeon again. Btw I've been wondering why everyone is so focused on blacklisting the xorg radeon driver? It would be a simple matter to just remove it. So I'm assuming that there is a reason nobody does? Some shared libraries?9) Rebooted in to Xen once again and confirmed the issue still exists. Black screen.

Indeed it is... I was wondering last night if there's a way to make the Xorg.0.log log debug level info. Maybe then I'd get a clue about why it's hanging just before announcing the card I have? I'll look into that. Oh, btw that portion where it's scanning my PCI devices, I think that only happens when a new driver is installed. If you look at my original Xen Xorg.0.log it doesn't do that. Not sure if that was the difference you were trying to point out. You'll also see that I've been using nomodeset from the beginning. I haven't had the splash disabled though.

Doesn't seem like it did much though. It appears that the blacklist doesn't work if X isn't started as part of the boot process (must get passed through the bootup scripts) as I did get some of the old Radeon errors again. Log ends the same way -

1. Purge the fglrx driver and remove the radeon.blacklist to boot the OS driver next time, and remove the pciback entry from grub. Update-grub. Also remove the xorg.conf file (copy it to a safe location just in case). Reinstate the pciback script.2. Boot the standard Xen-kernel, not the one you compiled.3. When in Xen, check sudo xm pci-list-assignable-devices - your second graphic card 02:00.0 and 02:00.1 should be listed. If not, try to run the pciback script manually. If that doesn't help either, remove or disable the pciback script and add the pciback entry to grub, next reboot into your compiled kernel.4. Once you got your 2nd graphics card listed as assignable, run the AMD fglrx installer. I would try the driver from the repos, that can be installed via synaptic. Then blacklist the radeon driver!5. Reboot and cross your fingers.

When you install fglrx, the second graphics adapter should have the pciback driver. Check with lspci -v to make sure.

After you installed the fglrx driver and blacklisted radeon, the boot process should first attach the pciback driver to the 2nd graphics card (so that it appears in xm pci-list-assignable-devices). Then it should prevent the radeon driver from loading, and start the fglrx driver instead.

Again, try with the standard kernel and Xen, not the kernel you compiled. I didn't have to compile any kernel to make it work. Only if you can't get pciback to grab your second graphics card, remove the pciback script and use your compiled kernel with pciback grub option. Make sure to install fglrx while booted into Xen.

1. Purge the fglrx driver and remove the radeon.blacklist to boot the OS driver next time, and remove the pciback entry from grub. Update-grub. Also remove the xorg.conf file (copy it to a safe location just in case). Reinstate the pciback script.

I've already done this step above.

2. Boot the standard Xen-kernel, not the one you compiled.

I've never used a custom Xen hypervisor. I'm assuming you meant use the standard Mint kernel. I have done so, but I can try again.

3. When in Xen, check sudo xm pci-list-assignable-devices - your second graphic card 02:00.0 and 02:00.1 should be listed. If not, try to run the pciback script manually. If that doesn't help either, remove or disable the pciback script and add the pciback entry to grub, next reboot into your compiled kernel.

I know you're trying to help me but I'm puzzled why you keep trying to get me to use the pciback script when it's not necessary for the task at hand. My problem isn't doing pci passthrough after all. It's getting fglrx to work in Dom0 which doesn't require immou at all.

4. Once you got your 2nd graphics card listed as assignable, run the AMD fglrx installer. I would try the driver from the repos, that can be installed via synaptic. Then blacklist the radeon driver!

I've tried this once already per your request to install the fglrx driver from within Xen (at which point I already had my second card assignable and listed in xm pci-list-assignable-devices).

5. Reboot and cross your fingers.

I do appreciate the responses, but I feel like I'm beating a dead horse at this point. Not your fault. I've just been over these permutations a lot over the last week and half. I was hoping someone might know of something I haven't tried.

When you install fglrx, the second graphics adapter should have the pciback driver. Check with lspci -v to make sure.

After you installed the fglrx driver and blacklisted radeon, the boot process should first attach the pciback driver to the 2nd graphics card (so that it appears in xm pci-list-assignable-devices). Then it should prevent the radeon driver from loading, and start the fglrx driver instead.

I thought I had mentioned that I already had this working in this thread or the vga passthrough thread. I apologize if I hadn't but again my problem is not with vga passthrough. That works.

Again, try with the standard kernel and Xen, not the kernel you compiled. I didn't have to compile any kernel to make it work. Only if you can't get pciback to grab your second graphics card, remove the pciback script and use your compiled kernel with pciback grub option. Make sure to install fglrx while booted into Xen.

Never had a problem with fglrx attaching to the correct video card in lspci. Just that X windows won't boot.

Today I found two links to problems that resemble mine (w/o Xen though), but unfortunately no solution is forthcoming. http://ati.cchtml.com/show_bug.cgi?id=743 Notice that this is a 7000 series GPU. This bug report is pretty detailed. The xorg.0.log looks like mine and they've even tried the beta driver.http://forums.gentoo.org/viewtopic-t-95 ... art-0.html 7000 series as well.I'm beginning to think this is an issue with these newer boards. Iirc, I haven't seen anyone report success with a 7000 series in Dom0 with fglrx. The first link even says they can get it to work with the open source driver like me.

Thanks for trying to help but I think I'm going to call it. I might just go back to dual booting until these drivers and/or Xen has baked a little more.

I agree with you - it does look like a driver issue, or perhaps an issue with your specific card. If you have any option to try another AMD card for dom0, that would help locate the problem (or even solve it).

The reason I was so adamant about going through the steps again was that I too had a black screen at first, and only the steps I described helped to solve the issue.

I agree with you - it does look like a driver issue, or perhaps an issue with your specific card. If you have any option to try another AMD card for dom0, that would help locate the problem (or even solve it).

The reason I was so adamant about going through the steps again was that I too had a black screen at first, and only the steps I described helped to solve the issue.

I guess my hangup is that, vga passthrough or not, the Radeon driver works fine under Dom0. I even made sure it wasn't the issue with Fglrx by turning passthrough off and on (or rather pci-back). In the end I'm not sure if we are getting no video for the same reasons. /shrug. Who knows at this point.

I might have to check it out some. I never really saw their posts show up in my searches so maybe I missed something.

I'm afraid I'm no expert on graphics cards and drivers. Good luck!

Nor I. I'm just stubbornly trying to troubleshoot this. I've tried 3 different distro's now. The other two I couldn't even get fglrx to load properly. Opensuse comes with a Xen kernel and Xen 4.2 so I thought it might have a better chance of working (Suse, like AMD, made contributions to Xen 4.2). At least insomuch that it was significantly different from my Mint install (the definition of crazy is to try the same thing expecting a different result). But alas Suse's driver support is lackluster and I couldn't get either the community provided fglrx package or the AMD version to load atop my Xen kernel (the former always loading on the standard kernel and the later complaining about missing header files that were loaded). Started to load Arch Linux and then realized that they are on Xorg Server 1.14 and Fglrx only supports 1.13. /sigh. Guess that's what you get with bleeding edge rolling releases. I could have backported to get fglrx loaded on it but then it just felt like I was missing the point of Arch Linux. So I've come full circle and will probably reload Mint 14 Mate, get vga passthrough working, and use the open source Radeon in my Dom0. Though, I'll probably still dabble with Arch on the side.

If only Nvidia cards at least worked under Dom0. Their proprietary drivers supported Xorg Server 1.14 before it's release and their release schedule is more aggressive.

(oh, and it turns out the "Xen" kernel on opensuse doesn't have pci-back compiled into the kernel so it does you a whole lot of good. )

2 weeks is a lot of time spent to troubleshoot one problem. But at least it looks like you learned a bit on the way. Thanks for your input on Arch and Suse - I'm quite surprised that Suse doesn't have pciback in the kernel.

Yes, it would be great if Nvidia drivers would work with dom0. I totally agree with you that when Nvidia proprietary drivers work, they are great. Also, Nvidia has a much longer support cycle for their graphics cards.

It also explains why the pciback stuff and the video card on dom0 is related. With pciback grabbing the second VGA, the fglrx driver should only see the first card and configure it. This is why I insisted on getting pciback working and then installing the fglrx driver in dom0. Anyway, you tried it and it didn't work.

The thread above also explains how to edit the xorg.conf file to make X work with only one graphics card out of two.

You'd think Nvidia would be more willing to contribute to virtualization projects to sell more video cards, but if their latest trade show is any indication they're looking to get into grid/cloud computing with their Arm+GPU combo's. I guess that makes sense considering the Desktop PC market is in a slump right now. Will they get more involved now that valve is talking about the steambox? Hard to say. Nvidia did bow out of the bidding for the next console generations so maybe they've decided to go all in on hybrid server computing. Time will tell who made the better bet between AMD and Nvidia. Hey, if Nvidia stops focusing on the desktop GPU market maybe they can lend some of their programmers to AMD for driver development?

Finally, have a look at this thread to see if it helps: http://askubuntu.com/questions/251045/how-can-i-run-x-on-1-out-of-2-gpus-connected.It also explains why the pciback stuff and the video card on dom0 is related. With pciback grabbing the second VGA, the fglrx driver should only see the first card and configure it. This is why I insisted on getting pciback working and then installing the fglrx driver in dom0. Anyway, you tried it and it didn't work.

Right, but my point was that I wasn't worried about passthrough or disabling any cards at this point. I just wanted normal video support under Xen. If that worked then I could go on to the next step. /shrug. It's water under the bridge now since I've already done all that.

The thread above also explains how to edit the xorg.conf file to make X work with only one graphics card out of two.

I don't know if I posted it in this thread or the other one, but I've already tried the Quantal version of that guide (what Mint 14 is based on). Thats what I was reinstalling in the posts above.

Again, in my case I used the driver package from the Linux Mint repository, not this manual install method. Just an option you may not have explored.

I don't really think it matters the version at this point, but again I've tried both of the packages in the repos.

Well, I got LM14 Mate loaded last night. Going to mess around with Xen again later (sticking, for better or worse, with the open source driver). I was tempted to give KVM another whirl after reading some of the last thread you posted, but then I noticed what kernel versions they were using and I changed my mind. Back in 2009 I tried to do vga passthrough with KVM (before it was common knowledge that iommu support was spotty). I even wrote a Howto while I was doing it but I don't think I ever posted it. The strange thing was that I actually got my windows guest to see my nvidia card (device manager), but I wasn't able to install drivers for it. Of course to get even that far I had to compile new kernels, qemu, etc. Suffice to say I'm not so quick to travel down that path of bleeding edge again.

I am also having som trouble with fglrx drivers. But I am seeing some very odd behavior.

Linux Mint 13AMD Redeon HD5450 graphics cardLinux 3.2.0-23-generic (x86-64) kernel under XEN, right off reposcurrently using fglrx-updates driver. Have also tried the fglrx and radeon drivers.

At first I tried changing between Radeon, fglrs, and fglrx-updates with various success and failure. At first tonightI could only get into gnome-fallback under Xen, then a lot of fails. But here is the very odd behavior I am seeing: with fglrx-updates driver-

# Uncomment to enable BadRAM filtering, modify to suit your needs# This works with Linux (no patch required) and with any kernel that obtains# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

### BEGIN /etc/grub.d/40_custom #### This file provides an easy way to add custom menu entries. Simply type the# menu entries you want to add after this comment. Be careful not to change# the 'exec tail' line above.### END /etc/grub.d/40_custom ###

and some system info: Yes, I have a third graphics card- an old (c.1996) pci video card Ive been trying to get working but have not worked on yet. Could that be at issue? But without Xen it is not an issue.

Whoa. That is strange. It alternates between working and failing between boots with no user modification? On HD 5000 cards too. Maybe my theory was incorrect. What exactly do you mean X fails to load? Do you get a black screen like me? Or are you getting some sort of error when X attempts to load?

gfxmode: What I meant is the last successful graphics mode or fallback mode. Not sure how it's called, but grub / xorg has the option to boot the last graphics mode that was successfully loaded, in other words, at some point it saves the graphics mode that it uses for the next time you boot. This could potentially toggle the mode.

radeon.blacklist=1: Yes, I saw that, but it's the last option in the line. I think it should come as the first option, before all the other options. Because if the kernel boot parameters are executed in the order they appear, that would make a difference.

In other news it seems my onboard audio doesn't work passed through (win7 driver issues). I should have tried it sooner. So I canceled my USB soundcard order before it shipped so I can try some things. I might end up having to get a proper internal soundcard if I want decent audio for my Win7 DomU.

This - http://www.gossamer-threads.com/lists/xen/users/256539 suggests that an older kernel might actually work. So I'll probably load Mint 13 to try it. I'm also going to give KVM a try in another distro that has a newer kernel. Not to mention KVM will probably bypass this issue with Fglrx. Here's hoping I can iron out everything this week.

Locuust wrote:In other news it seems my onboard audio doesn't work passed through (win7 driver issues). I should have tried it sooner. So I canceled my USB soundcard order before it shipped so I can try some things. I might end up having to get a proper internal soundcard if I want decent audio for my Win7 DomU.

This - http://www.gossamer-threads.com/lists/xen/users/256539 suggests that an older kernel might actually work. So I'll probably load Mint 13 to try it. I'm also going to give KVM a try in another distro that has a newer kernel. Not to mention KVM will probably bypass this issue with Fglrx. Here's hoping I can iron out everything this week.

1. To get sound in Windows domU, you plug a USB sound card into a passed-through USB port and connect the headphone output to the line input on the motherboard. You don't pass through the onboard sound. If you were to pass through the onboard sound, you'd be loosing sound under Linux / dom0.

powerhouse wrote:1. To get sound in Windows domU, you plug a USB sound card into a passed-through USB port and connect the headphone output to the line input on the motherboard. You don't pass through the onboard sound. If you were to pass through the onboard sound, you'd be loosing sound under Linux / dom0.

Lol. I know how you did it, but I want to have the option of having a surround sound setup while playing games in DomU (or 3D audio with headphones). My plan all along as to use the USB soundcard for Dom0.

Will check it out once I've tested things under Mint 13. It looks pretty detailed, but man using an RC kernel... I hope it doesn't come to that. The VFIO stuff sounds cool, but it's essentially a rewrite of what KVM had done with x86 specific VGA passthrough. Unless I misunderstood and it makes it easier to implement in some way I'm hoping the old x86 approach will be enough to get me by. We'll see.

Perhaps one of you will be able to post a kvm how-to for Linux Mint?

Lol, I dunno. I wrote a KVM howto in vain last time. I might go back and do one after the fact if it does indeed work.