The way we think about user interaction – actually the user interfaces themselves – is changing. The iPhone seems to be the protagonist teaching us new ways to interact with phones and iPad even coins a new form factor driving this trend further. Touch and multi touch are becoming main stream because vendors have begun to create operating systems, UI metaphors, and backing services around these interaction principles – rather than slightly adjusting OSes/UIs build for conventional PCs with keyboard and mouse.
This is actually a defining feature of the next evolutionary step of UI, namely Natural User Interfaces (NUI). As wikipedia states…
A NUI relies on a user being able to carry out relatively natural motions, movements or gestures that they quickly discover control the computer application or manipulate the on-screen content. The most descriptive identifier of a NUI is the lack of a physical keyboard and/or mouse. (wikipedia)
While Apple seems to take the lead in public perception, Microsoft has a rather mixed lineup: With smart phones Windows Mobile 7 seems a bit like “taking the last chance”, even if the move to Silverlight as a platform is a bold one and (IMO) a good one. On the other hand they just managed to drop the very promising – by itself as well as positioned against the iPad – Courier project. As a colleague stated in your internal company blog: “I’m frustrated. Period.” And lastly Microsoft has Surface which has no competition I’m aware of at all (unless you want to build one yourself).
Surface is not only commercially available, it also adds the capability to detect objects placed on the table and thus goes beyond plain multi touch. And it is subject to further research, as this excerpt from PDC09 shows: (better quality here, at 83:00)
Looking Ahead
Well, this is kind of what we have today. If you would like to see where this might be heading, have a look at the Microsoft Gives Glimpse Into the Future talk Stephen Elop held early ‘09. It’s a 36 minutes video, but you may jump to 14:00 and watch the presentation of "Glimpse in the future". What’s presented there is impressive: Live translations enabling people talking with each other in different languages. Surface like tables interacting directly with iPad-like multi touch tablets placed on it. Minority Report like control. Augmented reality. …. It’s even more impressive since everything is backed afterwards by actually existing (if in early stages) technology. There’s a shortened and also an extended version available on youtube:
Speaking of Minority Report. Another great video comes from John Underkoffler; John has been the science adviser for that movie and he does the whole presentation with exactly that technology!
This talk is certainly worth watching, as he makes some very interesting observations (in fact, watching this video triggered this post; thanks Daniel). His final prediction is … ambitious: “I think in 5 years time, when you buy a computer, you’ll get this.”
Is that cool or what?
Second Thoughts
Well, as they say:
“Prediction is very difficult, especially about the future.” (various)
There’s one thing I don’t like about those predictions. They are (deliberately?) incomplete. They certainly shine in new fields of applications for computers, new degrees of collaboration, new ways of interaction. Like home integration, meeting areas with huge collaboration screens, geo services and augmented reality, or simply navigating and reshaping existing data. But in their aim to show new ways of doing things, they neglect the “old”, conventional demands, demands that won’t go away.
The very fact that these NUI approaches – touch, gestures, even voice – are defined by “the lack of a physical keyboard and/or mouse” (and in case you didn’t notice – NONE of the above videos hat a keyboard in it!) renders them inappropriate for a whole bunch of scenarios. Can you imagine a secretary typing on a virtual keyboard? A call center clerk waving at his screen while he talks to a customer? A banker shooing stock rates up and down? A programmer snipping his code into place? Cool as all that Minority Report and other stuff may seem, I have a hard time imagining anyone whose daily job today requires a keyboard to a substantial degree using some other “device” instead.
In the end we’ll probably see both. NUI approaches are going to spread, new devices targeted at different scenarios simply require different notions of user interaction. But they are not going to replace today’s conventional computers, they are going to be a complement, actually even a necessary one. Another necessary complement is the mutual integration with each other, the internet/cloud, and social platforms, but that’s a different story.
For us developers this will be the actual challenge: developing on conventional machines for devices and environments that have totally different ideas of how an application should look like and interact with its surroundings. Testing is going to be a bitch.
That’s all for now folks,
AJ.NET
AJ's blog 
Leave a Reply