The keyboard on the X11 session is working fine in the VirtualBox windows (VirtualBox Manager and even the Settings page from the VirtualBox Console). But the console itself doesn't get much keyboard events.

stefans, Tonin, it looks like the XKB extension on your system is not working the way I expect it to. Would you be able to build and run a small C programme which I will attach and attach the output to the ticket? That may give a clue as to whether the problem is in VirtualBox or your systems. Thanks.

Good work getting the test programme working. For (useless) information, "error(3)" isn't quite the same as "perror(3)", in particular because "error(1, ...)" terminates the programme when it is called, but that really doesn't matter here of course.

Before calling setxkbmap the data is clearly invalid. The documentation for the calls we are making to the X server does not give any indication that this may happen, so I assumed it would not. Obviously I have to rethink that assumption, but since the call succeeds and simply returns empty data it may be worth thinking about what constitutes success. If it can return empty data it may also be able to return partial data which isn't enough to work with. Then again, I could define that to be the user's problem, since it is clearly possible to fix the data returned.

After your call to setxkbmap the data returned looks correct, so I assume that my code is not handling it correctly, and I will have to take a a look to work out why not.

I assume that ssh is not important in the equation here, since it should just be proxying the connection to the X server without changing the data going through it. The output from your second run of xkbtest should provide me with enough information to support your XQuartz layout even without XKB working correctly, assuming that the data will be the same on different XQuartz systems. (I can probably check that in the X.Org source code.)

ssh X11 forwarding is clearly working fine for the QT4 windows of VirtualBox (Virtual Box Manager for example, the keyboard works perfectly when typing in the description fields), so yes it seems it's not the issue here. And this is with or without a call to setxkbmap, but I guess this is a different story and that VirtualBox needs the raw keyboard events where QT might be ok with only the chars coming in.

If there is anything else you want me to test on my side, let me know.

On closer inspection the output after the setxbkmap command looks reasonable. You say that the console still doesn't handle keyboard input correctly at that point. Could you describe more precisely what does and doesn't work?

On closer inspection the output after the setxbkmap command looks reasonable. You say that the console still doesn't handle keyboard input correctly at that point. Could you describe more precisely what does and doesn't work?

The keys are all messed up. There's not a single key that is in its correct position, even space and return produce other chars. I cannot input a correct username and password at the login prompt.

I'm not sure where the issue lies... Some people are reporting the same issue to the XQuartz developers although it is not clear if the issue is with XQuartz or not. Some KVM users are reporting a similar issue when trying to use the virt-manager in the same way.

I tried different settings with setxkbmap and had different results in VirtualBox, but still not a single one with some correct keys... I'm a bit clueless now.

On the setxkbmap manual page I see the following (though I assume you have read it too):

"If you have an Xserver and a client shell running on different comput‐
ers and XKB configuration files on those machines are different you can
get problems specifying a keyboard map by model, layout, options names.
This is because setxkbcomp converts these names to names of XKB config‐
uration files according to files that are on the client side computer,
then it sends the file names to the server where the xkbcomp has to
compose a complete keyboard map using files which the server has. Thus
if the sets of files differ significantly the names that the setxkbmap
generates can be unacceptable on the server side. You can solve this
problem by running the xkbcomp on the client side too. With the -print
option setxkbmap just prints the file names in an appropriate format to
its stdout and this output can be piped directly to the xkbcomp input.
For example, the command

setxkbmap us -print | xkbcomp - $DISPLAY

makes both steps run on the same (client) machine and loads a keyboard
map into the server."

By the way you should run the patched version without calling setxkbmap. It should notice the empty mapping and fall back to a different method. If it works then attaching the log file should provide me with useful information to support XQuartz better. If not then tell me the result anyway.

Result is slightly the same as before (without any call to setxkbmap, like in my very first try). In this situation the console doesn't receive much key events, only return, some function keys and keypad keys are actually printing something.

Could you try starting a virtual machine with the environment variable "LOG_KB_SECONDARY" set to something (e.g. to 1) and attach what it prints to standard output? Thanks. (I hope that this code is still working as it should, given how long it is since some one has needed it.)

It would be great if you could try the following patch in addition to the first one, again without running setxkbmap. By the way, I am very confused that our mapping detection code fails to find your "Q" key although it finds every other one.

That is actually expected (if I think about it). What interests me though is the virtual machine log and (even more) whether the keyboard works any better now. And whether you have any idea what my code might have against your "Q" key!

I wasn't sure at first as I had been playing with different settings in both the host, the guest and XQuartz to try make something work. But now I can happily say it: IT WORKS!

Your last patch indeed made it works. The VBox.log still says that the keyboard doesn't seem to be known to VirtualBox. But the console was working like an AZERTY keyboard, and after changing the guest console keyboard configuration to 'fr' and 'mac' variant (instead of plain 'be' config, you're right, belgian and french Apple keyboards are actually the same, thank you Apple marketing for making us believe otherwise) I now got a fully working keyboard in the VirtualBox console! Yeah!

To be complete, I must say that it works when my Debian guest is fully started. If I brake in GRUB while booting, then I have a plain QWERTY keyboard (US I guess) under my hands, which is acceptable.

I really must thank you for this correction and for your support: Thank you!

Summary
changed from Keyboard not known to VirtualBox (X11 from MAC, host Ubuntu srv 12.04) to Keyboard not known to VirtualBox (X11 from MAC, host Ubuntu srv 12.04) -> fixed as of 21 Feb 2013; not fixed in 4.2.x

Glad I was able to solve that. And just to sum everything up, in case it is of interest...

VirtualBox has three different methods of detecting the X11 keyboard mapping (that is, how the "key code" numbers which the X server uses to describe physical key positions are best mapped to PC scan codes; this may be approximate if e.g. the X server has a keyboard which doesn't look much like a PC one). We can do the detection by looking at the keyboard mapping (i.e. a French keyboard has a ".?" key but a US one doesn't) and working backwards from there; or by looking at the key codes for well known keys like Control, Alt and Shift and seeing if they match a mapping we know; or just asking XKB for the position on the keyboard of each key code.

On your system only the first method was working, but due to a bug in the code we were still trying to use the third method. That method works nearly everywhere, which is why no one ever noticed that the other two were broken. And the output in the log file is the information I need to make the second method work too, so that we can work right with XQuartz servers even if the first method fails. I was able to check the information here: http://xquartz.macosforge.org/trac/ticket/511

And I have also solved the problem of the "Q" key - it is detected, but simply not logged due to another minor bug.

I will after all add one more patch - the change I mentioned to detect XQuartz by the second method. This should make the log message go away and more importantly the keyboard should still work after applying it. Since this patch will go into VirtualBox it would be nice if you could test that it doesn't break your set-up again.

I'm joining the conversation a bit late here, but it looks to me like there's still something wrong here since you're committing some hard coded values for an "XQuartz" keyboard type.

"XQuartz" does not correspond to a particular keyboard. The keyboard layout is generated at runtime based on the user's OS X keyboard layout (we query the entire mapping and generate an XKB layout at runtime). It is also changed when the user changes their layout.

You should check out the code in Xnest and xf86-video-nested for how to query the keymap and follow it when it changes on the hosting X11 server.

Jeremy, the "XQuartz" keyboard "type" actually corresponds to the way the XQuartz server maps key codes to physical keyboard keys, which we represent as PC AT scan codes. We detect XQuartz by comparing the key codes for some key symbols which are rarely re-mapped and usually exist, namely left control, left shift, caps lock, the function keys and a few others. In deference to old traditions we also check for swapped control and caps lock. This is not meant to be bullet proof (some people will re-map their shift key), merely a way to improve our chances of getting the right mapping, where it is one of two fall-backs if (as is the case with XQuartz) we can't use the XKB key names for the purpose. And of course we are not actually interested in the key map, as virtual guest systems expect to see scan codes, not symbols.

I will after all add one more patch - the change I mentioned to detect XQuartz by the second method. This should make the log message go away and more importantly the keyboard should still work after applying it. Since this patch will go into VirtualBox it would be nice if you could test that it doesn't break your set-up again.

Hi Michael,

I just integrated your latest patch and it is running fine. Keyboard works as well as yesterday and I don't see the warning message in the log anymore. Thanks!

Thank you for testing and for your good work on debugging this problem! I must point out that you used the VBoxSDL front-end, which does not print any keyboard-related warnings, even if there are some; it does use the same code though, so if the keyboard works there it should work in the VirtualBox front-end. When you start a machine using the VirtualBox front-end you shouldn't see any warnings either, and you should see a message saying "Using known keycode mapping for keycode to scan code conversion". If you don't add any more replies to this ticket then I will assume that that is the case.

I will keep this ticket open for a few days in case Jeremy has anything to add. Otherwise I will treat the issue as sorted.

Summary
changed from Keyboard not known to VirtualBox (X11 from MAC, host Ubuntu srv 12.04) -> fixed as of 21 Feb 2013; not fixed in 4.2.x to Keyboard not known to VirtualBox (X11 from MAC, host Ubuntu srv 12.04) -> fixed in releases after 4 Mar 2013 (4.2 series and later)