I spent some time this week trying to help clean up the behaviour of Fedora 20 in regard to keyboard layouts, so I thought I’d write it up here. And write a (Fairly) Short History of Fedora Keyboard Input, while I’m at it.

In Linux, keyboard input at the console and X levels is actually handled differently. Console keyboard input basically happens in the kernel, with some very simple userspace utilities available to configure things in the ‘kbd’ package. The ‘loadkeys’ utility loads keyboard layout maps of a given format from a given location. There is no standard daemon or anything for switching between different layout map files, in this system: it’s expected you pretty much load a single layout file and stick with it. (The way ‘configuration’ works for kbd is really bone simple: somewhere during init there’s a very trivial function which reads a config file, gets a layout name from it, and runs ‘loadkeys (layout)’. That’s it.)

At the X level, keyboard input is handled by xkb, about the complexity of which I have written before. But for the purposes of this, the key point is that it uses layout maps of a different format, stored in a different location, and in the xkb world, switching between layouts is normal and expected behaviour.

Prior to Fedora 20, we had two entirely separate sets of keyboard layout maps: one for kbd, one for xkb. They had separate upstreams and separate histories; even maps which happened to have the same name in both schemes were not necessarily the same. Prior to Fedora 18, you were expected to configure your keyboard via the ‘system-config-keyboard’ utility, which had a table of a limited number of maps for which it knew about roughly corresponding kbd and xkb configurations; you hoped your layout or one like it was in s-c-keyboard’s list, you picked it, and s-c-k handled setting the kbd and xkb configurations. During installation, we just ran s-c-keyboard for you to configure your keyboard layout. That kinda worked, but it meant maintaining a tool no-one was particularly excited about maintaining, and we were basically stuck with a fairly arbitrary subset of all possible keyboard layouts which had got written quite a long time ago and more or less bit-rotted since then.

So in the new installer UI introduced in F18, we changed things up: we didn’t run s-c-keyboard in the installer any more. Instead, we gave anaconda the ability to offer every available xkb layout as a possible option, and wrote some fairly trivial heuristics for guessing what layout you might want to use based on your choice of language at the start of the installation process, so we could pick a sane default layout. Around the same time, systemd’s localed sub-system gained the ability to sort of do what system-config-keyboard used to do: localed contains a copy of s-c-k’s xkb/kbd layout matching data. So the design was that you’d pick an xkb layout in anaconda, and systemd would transparently take care of figuring out a ‘matching’ kbd layout and configuring it at first boot.

Well, we had various teething issues with that design, but it’s not worth going into here. But the new design did bring some urgency to something we’d had in the works for a long time. Instead of having systemd do contortions to try and figure out a ‘matching’ kbd layout for your chosen xkb layout, it’d seem a lot simpler if we could magically make all the xkb layouts available to kbd, right? Turns out we were actually planning that all along.

And so early in Fedora 20, we introduced a change to the kbd package. It now used a couple of neat tools to generate kbd-format layout maps based on all the available xkb maps. For every xkb map present in Fedora, we now had a matching kbd map.

So things are really simple now, right? We should just drop the localed clever code for ‘translating’ xkb layouts into kbd layouts and simply tell it to use whatever xkb layout you got from the installation process for kbd as well.

Well…turns out, not so fast. I actually had bug reports in asking for these changes to be put in place for a while, without it happening. By the time I got around to requesting those changes with a bit more urgency last month, I was starting to suspect we’d missed a rather large problem, and it turned out we had. It goes back to the issue of layout switching which I mentioned earlier.

I said that there’s no mechanism for switching between multiple kbd layout maps. This is true, but what you can do with a kbd-style layout is switch between different layouts within a single map file. In both kbd-style and xkb-style layouts, each key can produce up to four characters: one when you press it alone, one when you press it with shift, one when you press it with alt-gr, and one when you press it with shift and alt-gr. What you can also do in a kbd-style layout is define a key or key combination that effectively does for alt-gr what caps lock does for shift: toggles it. So if you write a kbd layout which defines alt-gr mappings for a lot of the keys, and has a key combo for toggling altr-gr, you’ve effectively got a switched layout. The layout I usually use to illustrate it is Russian: if you load the ‘ru’ kbd layout and start typing with the letter keys, you’ll notice it seems just like a US layout. If you then hit the left ctrl and shift keys together, the letter keys ‘switch’: they now output Cyrillic characters. That’s the alt-gr toggle. In a Russian layout, the letter keys output Latin characters when pressed alone, Cyrillic when pressed with alt-gr. This is how Russians expect their keyboards to behave.

Now in xkb, you can achieve basically the same result, but by a different method. A typical Russian xkb configuration would be to have one of the Russian layouts and US English both enabled, and define a key combination to switch between them. xkb can have as many layouts enabled as you like, and can designate a key combination that switches between layouts. So the user experience is much the same: you can type Latin characters, then hit a key combination and switch to typing Cyrillic. But the implementation mechanism is different.

Russian’s one example of this, but there are actually dozens of layouts of this type, where users expect them to be switched, so they can input both Latin and ‘native’ character sets. It’s not like a UK English, or French, or German, or whatever, layout, which is an alternative arrangement of keys but mostly inputs the same set of characters.

So the big problem we hadn’t really accounted for was simply this ‘switching problem’. No kbd layout which has been tool-generated from an xkb layout will have a set of alt-gr characters and a switch combo defined, because that’s just not how xkb layouts are designed to work: you’re supposed to switch between xkb layouts, not within them. So any kbd layout produced from an xkb layout whose users would expect to load it in combination with a Latin-capable layout and switch between them becomes a booby trap, because if you load it, you cannot input Latin characters at all. This is obviously not good.

Originally, the expectation was that everyone would use an xkb-converted layout, and so the ‘old’ kbd layouts were moved to a ‘legacy’ subdirectory and taken out of the loadkeys path: you could only load them with an explicit path. Really, we were just keeping them around in case we’d screwed something up. Which, of course, we had. Obviously that couldn’t really work, though. (We also set up symlinks for all the old kbd names which pointed to a similar xkb layout; this was intended to handle people upgrading from older releases. Of course, this meant anyone with a ‘switched’ layout who upgraded to F20 suddenly lost their switching, which I imagine they weren’t happy about…)

So in the last week, after thinking this through, we’ve come up with a set of changes to kbd and systemd. As I’d originally intended, we changed localed’s logic, so instead of always trying to find a ‘matching’ kbd layout for the selected xkb layout by using its table, if a kbd layout of the same name as the selected xkb layout exists, it will use that. But we kept the table of ‘corresponding’ layouts around, and the logic: it’s called only if a layout of the exact same name isn’t found. We changed up the kbd package so the legacy layouts are back in the loadkeys path, but after the xkb ones, and dropped the symlinks (the bug report is about a problem I didn’t actually mention in this post – console layout loading frequently didn’t seem to work at all. We think these changes incidentally fix that bug too). And I came up with a kinda-clever, kinda-gross hack: now, after it generates the full set of xkb-derived kbd layouts, the kbd package finds all the layouts which have no mapping for the character ‘A’ (an upper-case Latin ‘a’) and deletes them (this is a trivial zgrep | xargs rm pipeline). This is acting as a proxy for ‘cannot input Latin characters’ – there are 400+ xkb layouts and 150+ of them cannot input Latin characters, so trying to keep track of them in a ‘curated’ fashion seems inappropriate. Doing it this way at first felt like a hack, but to be honest, now it seems like the most sensible approach. I did do some manual sanity checks with layouts we know to be either capable or not capable of Latin input.

Now if you followed all that, award yourself a gold star, and you should be able to figure out the eventual result. If not, here’s how it works:

If you pick an xkb layout which is capable of inputting Latin characters, then the kbd package will contain a kbd layout generated from that xkb layout with the same name. localed will use that layout as your console layout, and you’ll have precisely the same layout in X and at the console.

If you pick an xkb layout which is not capable of inputting Latin characters, the kbd package will not contain a kbd layout generated from it, because you wouldn’t want to use that (you wouldn’t be able to type any Latin characters). localed will notice this, and use its table to try and find a matching ‘legacy’ kbd layout. If it finds one – for instance, if you picked Russian – it will use that, and you’ll have a proper, native, switchable layout. If it can’t, you’ll just wind up with the default kernel layout, which is, inevitably, US English. And at least you’ll be able to type Latin characters, which is the most important thing to be able to do at a console.

It’s a saner result than a ‘native’ layout which is functionally useless. People who upgrade should also wind up with something sane – they may wind up with a converted or a legacy layout (depending on the names), but they shouldn’t wind up with anything that doesn’t work usably.

One other change we put into place between Fedora 18 and Fedora 20 was to use langtable in the installer to try and be better about picking an ‘appropriate’ X layout for your language by default. langtable is a fairly new project aiming to store this kind of ‘if X, then Y’ information for i18n stuff: it’s intended to be a source of information like ‘what keyboard layouts are people who pick Language X, Variant Y most likely to want to use?’ It has a list of keyboard layouts, languages, territories, and timezones, and all sorts of mappings between them. It’s a work in progress, but it already contains a lot of the knowledge that system-config-keyboard had and localed inherited.

So the ultimate upshot of this: if we lined everything up right, after various teething issues in Fedora 18 and Fedora 19, Fedora 20 should be getting pretty good at this keyboard layout stuff. Most users should have a sensible default layout (or pair of layouts – when we know your ‘native’ xkb layout can’t input Latin characters, we automatically configure it alongside US English, with a layout switch combo) selected in the installer after they pick their language and locale. If you want to pick a different one in the installer, you can – you can set any number of layouts you like, and customize the layout switch key combo. And we should do a decent job of selecting the best available console layout to ‘match’ whatever xkb layout you set to be at the top of your list in the installer.

Around the same timeframe, GNOME has been similarly improving and refining its input handling. While there’ve been various transition pains and teething issues at all kinds of levels over the last few releases, I feel like we’re kind of reaching a point where we’re getting back to the level of correctness we had prior to Fedora 18, but with much more flexibility: you’re now not limited to a fairly arbitrary and bit-rotting subset of keyboard layouts and tied into the Fedora-specific system-config-keyboard (and even older system-setup-keyboard, which systemd killed). You have the whole set of xkb layouts to choose from, and at the GNOME level, with 3.8 and 3.10, we have really pretty awesome support for more complex input methods via ibus. There is much less Fedora-specific infrastructure around, and we’re at least more converged between xkb and kbd.

I’m now very much looking forward to the introduction of a new thing called kmscon to Fedora. kmscon is a user-space replacement for the kernel console, intended to replace the kernel console plus all the userspace bits that go with it (gettys, kbd and all the rest of it). kmscon actually uses xkb – via libxkbcommon, with no X dependency, of course! – for keyboard input. So in the Glorious Future when we can switch to kmscon for our consoles, we really will be able to forget all about this kbd vs. xkb mess, and have a single system-wide keyboard input framework, data and configuration. How I look forward to it…

16 Responses

I have a problem with Fedora 20 on my laptop. It defaults to US English. I need it to be UK English. As far as I remember, the installer did not pause long enough for me to find the UK English option during install, and defaulted to US English. Now, when I type in my email address, it comes out as william.berry3″ntlworld.com. The UK currency symbol is NOT # , which is what comes out when I press that symbol on my keyboard.

The only way I can change to the UK layout is by using system-config-keyboard. @loadkeys uk@ or @loadkeys gb@ does not work. (The @ symbol is the result of pressing shift-2, which is ” on a UK keyboard). However, s-c-k does not preserve the configuration between reboots. In my view, this is a bug.

Thank you for explaining your efforts to co-ordinate kbd and xkbd. That seems to me the sane thing to do. However, it would be helpful if a switch in keyboard layout was preserved between reboots.

This is probably not the place to ask for help. However, I believe it would be helpful if you were to post on the main Fedora site something to help ordinary users like me to deal with this problem. I had to a lot of searching around to find out that system-config-keybord even existed! I had previously used loadkeys, which, up to now (but not with F20) worked.

“As far as I remember, the installer did not pause long enough for me to find the UK English option during install”

Not to be rude, but I’m fairly sure you’re incorrect. The language selection screen is the very first shown during installation, and there is no ‘auto-continue’ – that screen won’t go away until you click the button to indicate that you’re done.

“However, s-c-k does not preserve the configuration between reboots. In my view, this is a bug.”

I can’t be entirely sure from the information you’ve provided here, but I’m fairly sure what’s actually happening is not what you think is happening.

s-c-k does save the configuration you specify, and ‘loadkeys uk’ almost certainly does work: but if you’re using GNOME as your desktop, when you first booted the installed system there was a first time wizard, and setting a keyboard layout was a part of that. GNOME overrides system-wide settings with any settings a user configures for GNOME.

If you want to check this, see if the file /etc/X11/xorg.conf.d/00-keyboard.conf says “Option “XkbLayout” “us” or “Option “XkbLayout” “gb” (or “uk”, I always forget which is being used when). If it’s gb or uk, s-c-k did its job fine; your setting is just being overridden at a higher level.

To change the GNOME keyboard configuration, go to the Settings panel, click Region and Language, and then use the + button under Input Sources to add a new keyboard layout, and remove the US layout if you never want to use it (GNOME will provide a switcher icon if you configure multiple layouts).

Thank you so much for all your help. However, something odd is happening. etc/X11/xorg.conf.d/00-keyboard.conf tells me that keyboard is gb. Gnome settings tells me that language and region is gb. (I have removed us). In a terminal found by ctrl-alt f2 the keymap is gb. In LibreOffice the keymap is gb. In Gnome-terminal the layout is us. In Gnome-terminal running “loadkeys gb” and “loadkeys uk” both return “Couldn’t get a file descriptor referring to the console”. system-config-keyboard is the only thing that works, but the setting is not saved – I get the same problem every reboot.

I had not originally intended to ask for help here. However, your original post was about unifying everything, to make one mechanism for regional settings. The fact that I am getting these problems may highlight something that is yet to be resolved.

After that, when I go back to gnome-terminal, I get a gb keyboard layout! (also in gedit – gb layout). I am confused – “man setxkbmap” indicates that -print only tells you what the current situation is – it doesn’t set anything.

Shut down – restart. US keymap in gnome-terminal, gb keymap in vt. Run “setxkbmap” in vt -> “Cannot open display “Default Display”. (This is not altogether unexpected). In gnome-terminal “setxkbmap” (just that – no parameters) gives no output in response, but switches to gb keymap in gnome-terminal and gedit!

I know very little about what happens at boot, but I suspect that something should be happening that isn’t. I suppose I could just put setxkbmap in rc.local. That would solve the problem for me, but it wouldn’t tell you why I need it. I am assuming that the fact this is a laptop is not relevant?

Is this blog the right place for this conversation? If it helps you to get closer to your goal, I suppose it is.

I rather suspect that what’s happening in your case is that Cinnamon is configured, somewhere/somehow or other, to use a US layout. The desktop can override the X settings when it starts up. In GNOME, there’s actually rather a bit of hidden complexity here: if you go to GNOME control center and set a given keyboard layout, it’ll certainly always use that layout for your user account when running GNOME (overriding whatever the X settings say). However, if your system only has one user account (yours) and that account is an administrator, it will also go out and change the X configuration too. If you have more than one user account and/or your user account is not an admin, though, it won’t.

(If anyone’s wondering what KDE does, why would you wonder? The answer’s obvious: it has a button for everything! It can use the X configuration, or you can set a per-user configuration.)

So, I suspect Cinnamon does something similar (as it’s a GNOME fork, after all) and that’s where your problem lies. I’m not sufficiently familiar with Cinnamon to tell you where to look, but I’d go poke around the Cinnamon control center, and also have a look through dconf-editor.

I’ve also seen cases where Cinnamon and GNOME have not been properly decoupled, and bits of one or the other will run in the other’s session, and/or they’ll read each other’s configuration. So you might want to check through the GNOME settings in dconf-editor too…

I didn’t have dconf-editor, so I installed it. I created a new user “test” which behaves in exactly the same way. I have tried KDE, Gnome, LXDE desktop, but not Sugar, MATE or e17. (What a wonderful array of choices!). sudo grouplist tells me that ‘GNOME Desktop’ is available, but not installed (!). Actually, it is. I thought that perhaps some vital part of Cinnamon was not installed (I didn’t have dconf-editor). “sudo yum groupinstall Cinnamon” returned “No packages in any requested group available to install or update”.

Gnome is the only DE I have tried that uses gb keyboard – the others default to us.

I do not know what I am looking for in dconf-editor. org.freedesktop.ibus.general.xkblayoutconfig shows default as mi (Australia)! I have no idea why. In this key, there is no gb layout. “uk” is in Eastern Europe (Ukraine?). There is an “en” in the “Western Europe” list. So now we have different two-letter country codes in different places – uk (Ukraine, or United Kingdom?), gb (Great Britain), and en (England? The Scots and the Welsh will not like that!). Scrub that – further search shows that whatever I select here shows as “default”, but the “Set as default” button is greyed out.

sudo dconf-editor shows different values: org.cinnamon.desktop.input-sources is now completely blank. org.gnome.libgnomekbd.keyboard is also blank.

I run system-config-language. It shows “English (Great Britain) highlighted. If I press on the button “System Defaults” it returns a box that says “Do you really want to change system language to default [en_US]?”

/etc/locale.conf has LANG=”en_GB.UTF-8″. Should it also have something else? With this is place, why is the default according to system-config-language en_US? Is there another setting somewhere else?

You can completely ignore the ibus stuff. ibus is input methods – it’s what we use for languages like Chinese, Japanese and Korean that need something more sophisticated than simply one keystroke = one letter. It’s not applicable to your situation. (For the record, ‘en’ is ‘English’, not ‘England’).

But aside from that, er, honestly, no? The config is clearly saying one thing and the system’s behaving otherwise. Um. Do you have any *other* files in /etc/X11/xorg.conf.d ?

No – just 00-keyboard.conf.
I found this thread: https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1028207 but didn’t understand most of it.
/etc/locale.conf had LANG=”en_GB-UTF8″. That seems to be an error, so I changed it to LANG=”en_GB.utf8″. It didn’t make any difference.
/etc/vconsole.conf had KEYMAP=uk. I hope that means United Kingdom, but as it might mean Ukraine, I changed it to gb. No difference.

The LANG / locale stuff doesn’t matter to you either, it is not about keyboard layouts.

The /lib/kbd stuff does not matter to you, that is what’s used at the console, not what’s used in X.

.UTF-8 and .utf8 are, basically, both valid. You can Google around and read a whole bunch of history and theories as to why both exist and what to use when and blah blah blah, but anything that deals with locales will deal with both. And it’s not relevant to the keyboard layout anyhow.

I found the answer – it IS ibus. There is a small icon on the right of the lower bar,which looks a bit like a keyboard with a tail (connecting cable). Click on it, and it gave me 2 options: US or US.

Right-click for a menu, and choose Preferences. The window that comes up is titled “IBus Preferences”. Select “Input Methods” tab, then “Select an input method”. I found a number of English options, including “English – English (UK, Extended WinKeys)”. Select that, click “Add”. I also removed US, but it was still there (as second and third choices) on reboot. However, the UK layout has priority, and is the one I have when I reboot.

Problem solved – but I still don’t know why I have IBus if I don’t need it. I have other issues to sort out, but they have nothing to do with the keyboard. Now that I have a solution (even if not ideal), I shall stick here.

Thank you for what you are doing to try to simplify keyboard configuration. I hope that this additional complication may help you in troubleshooting other’s problems, and possibly simplifying further.

ahhhh, that would *certainly* explain it. Good catch! If I were you I’d just find whatever you have to do to disable ibus, since it’s no use to you. Just removing all ibus-related packages you can remove without causing anything you care about to be removed may do it.

some bits of ibus or other are usually present by default, but it’s not usually active. if you install with a language that needs it – Chinese or Japanese or Korean or something else along those lines – it’ll be active automatically (in GNOME at least).

Hi, my apologiese to chime here. I have a similar problem but needing ibus. I do use a Japanese keyboard in both romanji mode and kana/ kanji mode. The keyboard selected in Region & Language settings ins “Japanese (Kana/ Kanji)”.. which probably should be be the kkc ibus method. Anyway, after selecting this everything works fine until I reboot. After reboot, I get a US keyboard layout and hence can’t use the japanese keyboard options (i.e. Kana/ Zen..) to, among others, switch between Romanji and Japanese input anymore. I tried playing with the “Use custom keymap” option in the IM preference. It offers US & Japanese. But selecting either or neither, still will default me to the US keyboard layout after reboot. How can I make it stick?

Hello, I’ve been having similar issues with Cinnamon and after reading the above have found a fix (at least it works for me).

I didn’t have the “small icon on the right of the lower bar,which looks a bit like a keyboard with a tail” as mentioned by Bill Berry so by using a terminal command ‘ibus-setup’ (which brings up a simple GUI) I could deselect the ‘use system keyboard layout’ in the advanced tab and also select ‘show icon on system tray’ under the general tab.

Now I have the above mentioned ‘small blue icon’ and by right-clicking it I can ‘restart’ the keybord back to my profile settings (in my case Spanish) – life is much easier with Cinnamon now, I had almost given it up!