Why you can't use xmodmap to change how Synergy handles modifier keys
As previously mentioned I am doing a major overhaul of my workstation setup, the biggest since 1999. This has turned out to be a huge can of worms.
I will try to explain the tentacular complications caused by some software called Synergy...
I use an Apple trackpad and keyboard, and 1Password
I made a less-big change several years ago - big in terms of the physical devices I use, but not the software configuration: I fell in love with Apple, especially the Magic Trackpad. On the downside, getting an Apple trackpad to talk directly to a FreeBSD or Debian box is (sadly) a laughable proposition, and because of that my setup at work is a bit weird.
(Digression 1: I also like Apple keyboards. I previously used a Happy Hacking Keyboard (lite 2), which I liked a lot, but I like the low profile and short travel of the Apple keyboards more. Both my HHKB and Apple keyboards use rubber membranes, so despite appearances I have not in fact given up on proper keyswitches - I just never used them much.)
(Digression 2: More recently I realised that I could not continue without a password manager, so I started using 1Password, which for my purposes basically requires having a Mac. If you aren't using a password manager, get one ASAP. It will massively reduce your password security worries.)
At work I have an old-ish cast-off Mac, which is responsible for input devices, 1Password, web browser shit, and disractions like unread mail notifications, iCalendar, IRC, Twitter. The idea (futile and unsuccessful though it frequently is) was that I could place the Mac to one side and do Real Work on the (other) Unix workstation.
Synergy is magic
The key software that ties my workstations together is Synergy.
Synergy allows you to have one computer which owns your input devices (in my case, my Mac) which can seamlessly control your other computers. Scoot your mouse cursor from one screen to another and the keyboard input focus follows seamlessly. It looks like you have multiple monitors plugged into one computer, with VMs running different operating systems displayed on different monitors. But they aren't VMs, they are different bare metal computers, and Synergy is forwarding the input events from one to the others.
Synergy is great in many ways. It is also a bit too magical.
How I want my keyboard to work
My main criterion is that the Meta key should be easy to press.
For reference look at this picture of an Apple keyboard on Wikimedia Commons.
(Digression 3: Actually, Control is more important than Meta, and
Control absolutely has to be the key to the left of A. I also
sometimes use the key labelled Control as part of special key chord
operations, for which I move my fingers down from the home row. In X11
terminology I need
ctrl:swapcaps. If I need caps
it's easier for me to type with my little finger holding Shift than to
waste a key on Caps Lock confusion . Making Caps Lock into another
Control is so common that it's a simple configuration feature in Mac
I use Emacs, which is why the Meta key is also important. Meta is a somewhat abstract key modifier which might be produced by various different keys (Alt, Windows, Option, Command, Escape, Ctrl+[, ...) depending on the user's preferences and the vagaries of the software they use.
I press most modifier keys (Ctrl, Shift, Fn) with my left little finger; the exception is Meta, for which I use my thumb. For comfort I do not want to have to curl my thumb under my palm very much. So Meta has to come from the Command key.
For the same reasons, on a PC keyboard Meta has to come from the Alt key. Note that on a Mac keyboard, the Option key is also labelled Alt.
So if you have a keyboard mapping designed for a PC keyboard, you have Meta <- Alt <- non-curly thumb, but when applied to a Mac keyboard you get Meta <- Alt <- Option <- too-curly thumb.
This is an awkward disagreement which I have to work around.
X11 keyboard modifiers
OK, sorry, we aren't quite done with the tedious background material yet.
The X11 keyboard model (at least the simpler pre-XKB model) basically has four layers:
keycodes: numbers that represent physical keys, e.g. 64
keysyms: symbols that represent key labels, e.g. Alt_L
modifiers: a few kesyms can be configured as one of 8 modifiers (shift, ctrl, etc.)
keymap: how keysyms plus modifiers translate to characters
I can reset all these tables so my keyboard has a reasonably sane layout with:
$ setxkbmap -layout us -option ctrl:nocaps
After I do that I get a fairly enormous variety of modifier-ish keysyms:
$ xmodmap -pke | egrep 'Shift|Control|Alt|Meta|Super|Hyper|switch' keycode 37 = Control_L NoSymbol Control_L keycode 50 = Shift_L NoSymbol Shift_L keycode 62 = Shift_R NoSymbol Shift_R keycode 64 = Alt_L Meta_L Alt_L Meta_L keycode 66 = Control_L Control_L Control_L Control_L keycode 92 = ISO_Level3_Shift NoSymbol ISO_Level3_Shift keycode 105 = Control_R NoSymbol Control_R keycode 108 = Alt_R Meta_R Alt_R Meta_R keycode 133 = Super_L NoSymbol Super_L keycode 134 = Super_R NoSymbol Super_R keycode 203 = Mode_switch NoSymbol Mode_switch keycode 204 = NoSymbol Alt_L NoSymbol Alt_L keycode 205 = NoSymbol Meta_L NoSymbol Meta_L keycode 206 = NoSymbol Super_L NoSymbol Super_L keycode 207 = NoSymbol Hyper_L NoSymbol Hyper_L
These map to modifiers as follows. (The higher modifers have unhelpfully vague names.)
$ xmodmap -pm xmodmap: up to 4 keys per modifier, (keycodes in parentheses): shift Shift_L (0x32), Shift_R (0x3e) lock control Control_L (0x25), Control_L (0x42), Control_R (0x69) mod1 Alt_L (0x40), Alt_R (0x6c), Meta_L (0xcd) mod2 Num_Lock (0x4d) mod3 mod4 Super_L (0x85), Super_R (0x86), Super_L (0xce), Hyper_L (0xcf) mod5 ISO_Level3_Shift (0x5c), Mode_switch (0xcb)
How Mac OS -> Synergy -> X11 works by default
If I don't change things, I get
Command -> 133 -> Super_L -> Mod4 -> good for controlling window manager Option -> 64 -> Alt_L -> Mod1 -> meta in emacs
which is not unexpected, given the differences between PC and Mac layouts, but I want to swap the effects of Command and Option
Note that I get (roughly) the same effect when I plug the Apple keyboard directly into the PC and when I use it via Synergy. I want the swap to work in both cases.
xmodmap to the rescue! not!
Eliding some lengthy and frustrating debugging, the important insight
came when I found I could reset the keymap using
setxkbmap (as I
mentioned above) and test the effect of
xmodmap on top of that in a
reproducible way. (
xmodmap doesn't have a reset-to-default option.)
What I want should be relatively simple:
Command -> Alt -> Mod1 -> Meta in emacs Option -> Super -> Mod4 -> good for controlling window manager
xmodmap I should be able to just swap the mappings of keycodes
64 and 133.
xmodmap script is a very thorough version of this
idea. First it completely strips the higher-order modifier keys, then
it rebuilds just the config I want.
clear Mod1 clear Mod2 clear Mod3 clear Mod4 clear Mod5 keycode 64 = NoSymbol keycode 92 = NoSymbol keycode 108 = NoSymbol keycode 133 = NoSymbol keycode 134 = NoSymbol keycode 203 = NoSymbol keycode 204 = NoSymbol keycode 205 = NoSymbol keycode 206 = NoSymbol keycode 207 = NoSymbol ! command -> alt keycode 133 = Alt_L keycode 134 = Alt_R add Mod1 = Alt_L Alt_R ! option -> super keycode 64 = Super_L keycode 108 = Super_R add Mod4 = Super_L Super_R
WAT?! That does not work
The other ingredient of the debugging was to look carefully at the
xev and Synergy's debug logs.
When I fed that script into
xmodmap, I saw,
Command -> 64 -> Super_L -> Mod4 -> good for controlling window manager Option -> 133 -> Alt_L -> Mod1 -> Meta in emacs
So Command was STILL being Super, and Option was STILL being Alt.
I had not swapped the effect of the keys! But I had swapped their key codes!
An explanation for Synergy's modifier key handling
Synergy's debug logs revealed that, given a keypress on the Mac, Synergy thought it should have a particular effect; it looked at the X11 key maps to work out what key codes it should generate to produce that effect.
So, when I pressed Command, Synergy thought, OK, I need to make a Mod4 on X11, so I have to artificially press a keycode 113 (or 64) to have this effect.
This also explains some other weird effects.
Synergy produces one keycode for both left and right Command, and one for both left and right Option. Its logic squashes a keypress to a desired modifier, which it then has to map back to a keysym - and it loses the left/right distinction in the process.
If I use my scorched-earth technique but tell
xmodmapto map keysyms to modifiers that Synergy isn't expecting, the Command or Option keys will mysteriously have no effect at all. Synergy's log complains that it can't find a mapping for them.
Synergy has its own modifier key mapping feature. If you tell it to map a key to Meta, and the X11 target has a default keymap, it will try to create Meta from a crazy combination of Shift+Alt. The reason why is clear if you work backwards from these lines of
keycode 64 = Alt_L Meta_L Alt_L Meta_L mod1 Alt_L (0x40), Alt_R (0x6c), Meta_L (0xcd)
My erroneous mental model
This took a long time to debug because I thought Synergy was mapping
keypresses on the Mac to keycodes or keysyms on X11. If that had been
the case then
xmodmap would have swapped the keys as I expected. I
would also have seen different keysyms in
xev for left and right.
And I would not have seen the mysterious disappearing keypresses nor
the crazy multiple keypresses.
It took me a lot of fumbling to find some reproducible behaviour from which I could work out a plausible explanation :-(
The right way to swap modifier keys with Synergy
The right way to swap modifier keys with Synergy is to tell the
Synergy server (which runs on the computer to which the keyboard is
attached) how they keyboard modifiers should map to modifiers on each
client computer. For example, I run
synergys on a Mac called
synergyc on a PC called
section: screens grey: super = alt alt = super white: # no options end
You can do more complicated mapping, but a simple swap avoids craziness.
I also swap the keys with
xmodmap. This has no effect for Synergy,
because it spots the swap and unswaps them, but it means that if I
plug a keyboard directly into the PC, it has the same layout as it
does when used via Synergy.
I'm feeling a bit more sad about Synergy than I was before. It isn't just because of this crazy behaviour.
The developers have been turning Synergy into a business. Great! Good for them! Their business model is paid support for open source software, which is fine and good.
However, they have been hiding away their documentation, so I can't find anything in the current packages or on the developer website which explains this key mapping behaviour. Code without documentation doesn't make me want to give them any money.