The paper outlines how a laptop camera can be repurposed to watch over the user’s hands as they operate the keyboard. This camera would determine whether the hand was open or closed, recognizing particular postures in place of a shortcut activated via a combination of keys.
The study suggests 10 simple postures per hand: each finger outstretched individually to press a key, with the hand being either open or closed. These postures were then filtered into three categories based on comfort: recommended, intermediate and avoided.
The postures were then put head to head with the most common forms of keyboard shortcut mappings. These range from individual keys used in applications like Photoshop that require minimal text entry, to shortcuts that use a single modifier key like Ctrl or Shift combined with several different keys, and the inverse.
An experiment demonstrated that posture-based shortcuts were about as effective as traditional shortcuts that use one key and three modifiers. Shortcuts that use three keys and a single modifier and those that just use single keys both fared a little better.
The paper describes these results as “encouraging,” because the participants were completely unfamiliar with the posture-based shortcuts, but had some experience with the other methods from their own computer usage. It’s expected that users could become even more proficient with a little more time to acclimate.
While computer hardware is in a constant state of evolution, the methods we use to interact with it have remained much the same for years. It might seem unlikely that users could be convinced to relearn input methods to such a dramatic degree, but given the rise of the touchscreen over the past decade, there’s evidence that a majority are willing to learn something new.
