Paper Reading #3: Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input

Comments
Stuart Jones
Alyssa Nabors

Reference information
Authors: Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, Bill Buxton
Where: CHI 2010

Summary
In some prior related work, Guiard observed that both hands work together and noted that it's not a question of which hand is better, but in what way do the hands cooperate, or,  "what is the logic of division of labor between the hands?" Thus, Manual Deskterity, a scrapbooking application prototype, studies the division of labor between pen and touch, where the pen writes and touch manipulates. An early study was conducted to see how people use pens, tools and pieces of paper by having either participants brainstorm ideas for a short film by pasting and annotating clippings in a physical notebook. Several observations were made about their behaviors, particular how pens were tucked in their preferred hands and clippings were "framed" using thumb and index fingers while writing about them.

Manual Deskterity is primarily a prototype for research purposes into pen + touch input rather than as a practical or public application. Information from the above study resulted in features in the digital system including moving and selecting objects, placing items into a stack, cutting and tearing items, and copying items. Creating new objects or bringing in new materials was observed to come from above. Thus, this behavior was facilitated with a bezel menu that is activated with a swipe from the top bezel. Radial context menus are implemented upon selection of objects. Switching between modes is in the hand, so the user can work one-handed, tucking the pen between fingers to go from pen to touch, or two-handed by writing with one hand and using touch manipulation with the other--this is simply the user's preference.

This initial evaluation was supplemented by another with seven professional designers, who praised the bezel menu and commented that, while the gestures were not immediately intuitive, they became natural once learned.

An example of the radial context menu. Source: publication

Discussion
This was a very fun and interesting read and believe this is an important course of study and could be expanded upon to be applicable outside of scrapbooking. As a drafting or scrapbooking tool, Manual Deskterity looks to be a very intuitive extension and expansion upon simple pen and paper drafting. I also thought it was smart, efficient and useful that the designers conducted the user study with the physical paper notebook in order to drive their design, as this yielded some important observations that may of otherwise not been made. I would really like to try this out to experience it for myself. Implementing pen + touch should be a very useful feature.