Touch Computing Hits Its Stride

Microsoft's Surface and other touch-sensitive products can simplify complicated interactions, but are software makers and consumers game?

by Aaron Ricadela


It's the computer that almost wasn't. Microsoft's Surface—a touch-sensitive table that could redefine the way people interact with machines—got its start in company research labs five years ago, though backers considered putting the project on ice several times.

Engineers wanted to build technology that would let users tell a computer what to do by moving everyday objects, such as a digital camera or a game piece, around the screen's surface. Yet some managers viewed the system as an unmanufacturable toy. "Probably every year I thought about killing it," said Robbie Bach, president of Microsoft's entertainment and devices group, picking over dinner at a San Francisco restaurant recently as he recalled budget meetings. "We struggled with the business model."


View Slide Show

Microsoft (MSFT) has overcome many of those initial challenges. And the company intends to bring Surface, initially intended for niche markets including stores, casinos, and hotels, to consumers. Research into tactile, or tangible, computing is one of the most fertile areas of electronic-product design. The systems incorporate familiar objects such as toys, game tokens, cell phones, or wine glasses—and even substances like sand and clay—into the computing experience. By taking advantage of people's natural sense of touch and spatial orientation, the systems can offer more precise control over what's happening on the screen than pointing and clicking with a mouse.

Done right, tactile computing could help users design products, play games, and complete business tasks. Microsoft is working on a giant, 6-ft. by 4-ft. version of Surface that lets groups of four or six people gather around it to collaborate. Hiroshi Ishii, a professor at the Massachusetts Institute of Technology's Media Lab, has designed systems that let architects shape landscapes and buildings using sand and clay, then see the results of their models on a computer screen. His Tangible Bits group has also devised a system that lets users move magnetic pucks to design cell-phone networks, and computerized animals that remember and replay the shapes they're twisted into. "Tangible Bits is an attempt to defy the gravity of the pixels," Ishii says.
Touch Pioneers

Apple (AAPL) has brought "multitouch" technology from its music-playing iPhone into its ultra-slim MacBook Air, and could in the future adapt it to specialized desktop computers. Hewlett-Packard (HPQ), Mitsubishi Electric, and IBM (IBM) have also done work in the field of tangible user interfaces. Anchors on CNN (TWX) manipulate maps, charts, and photos with their hands on the network's computerized "Magic Wall." And touchscreen technology for PCs is already showing up in products including Hewlett-Packard's TouchSmart PC (BusinessWeek, 6/25/08), which can recognize gestures like the flick of a finger for choosing albums to play or selecting photos to view.

A confluence of greater processing power, the spread of supersized displays, and the success of the iPhone at raising users' awareness of the power of touch computing has sparked interest in the field. Microsoft's Surface efforts survived in no small part because of advocacy from Chairman Bill Gates, long a champion of "natural" methods of interacting with a computer, including touch and speech. In fact, tactile computing is one of just a handful of areas (BusinessWeek, 6/26/08) that Gates will continue to help navigate after his retirement from full-time work at the company. "Bill's a half-step ahead of us, and two steps ahead of the market in his thinking," Bach said.
Industry Implications

As forward-thinking as they may be, Gates and other surface-computing proponents need to ensure the technology doesn't leave the rest of the industry behind. The systems are expensive, the vast majority of software doesn't work this way, and there's little agreement over the best way for users to interact with tactile computers.

There's also a nagging question over how useful surface computing can be in a work setting, since applications remain somewhat limited. Today, the systems handle such tasks as helping salespeople explain the features of cell phones in AT&T Wireless (T) stores; assisting patrons at a Harrah's Entertainment casino in ordering drinks; or letting users download photos from a wireless-equipped digital camera placed on top of Surface.

Bruce Tognazzini, a principal at usability consultant Nielsen Norman Group, who's also worked at Apple and Sun Microsystems (JAVA), says the technology has big implications for the design industry, for starters. Tognazzini, who worked for Apple from 1978 to 1992, says it's "very realistic" for Apple to design a multitouch Mac for graphics designers with a horizontal screen. "The payoff is going to be high enough that people are going to go for it," he says. Apple spokesman Steve Dowling declined to comment on what he called speculation about future products. Yet Apple is already broadening its use of touch computing. In addition to the iPhone and iPod Touch, Apple's MacBook Air laptop features a large track pad that lets users make pinching, swiping, or rotating gestures on it to manipulate text or images.
Will Consumers at Large Adapt?

Microsoft is looking into business uses of Surface, such as manipulating photos for a magazine layout, or letting two people compose a PowerPoint slide show together, says August de los Reyes, a user experience architect at the company. Gates showed a group of CEOs in Redmond (Wash.) a prototype system called TouchWall that lets users stand in front of a giant vertical screen and use finger flicks to flip through document pages or slide decks. And Microsoft recently demonstrated Windows 7, due in 2010 or so, running on a touchscreen laptop and responding to gesture commands.

Andy Wilson, the researcher at Microsoft whose work led to Surface, says Windows 7 could simplify what today are complicated interactions, like the multistep process of clicking, grabbing, and rotating an object in a drawing program by using a mouse. "If you're going to have these kinds of complicated conventions, sometimes it might be easier to put your hand on the thing and do what you want," he says.

As with every new technology though, the success of tactile computing may depend in large part on users' willingness to adapt. "People still have a resistance to changing their work style," says MIT professor Ishii. In design shops and other creative milieus, people draw on whiteboards, talk about ideas, and point to things—interactions that tactile systems can mirror. "That's an exception," he adds. "Most meetings are one guy talking, and many people listening."

See BusinessWeek.com's slide show for more on surface computing.

Ricadela is a writer for BusinessWeek.com in Silicon Valley.

-----------------------------------------------------------------------------------

The reason I put this article up here is to highlight my belief that touch computing is the next big thing in personal computing. I remember how enamoured we were with the touch computing kind of technology in Iron Man.

--vj