Sunday, 31 May 2015

Apple's Force Touch And Taptic Engine

Apple Force Touch

David Briddock investigates the touchy-feely experience on Apple's latest MacBooks

On 9th March, at one of Apple's Special Events, CEO Tim Cook announced changes to its MacBook product line-up. Included in this was a brand-new, ultra-thin and light 12" MacBook plus an enhanced 13" MacBook Pro. Both models had Retina displays, a better keyboard, USB-C and other welcome updates.

Confusingly, the 12" MacBook is lighter than both the 11" and 13" MacBook Air products, which may well disappear altogether at some point. However, what caught many people's attention was a completely re-engineered touchpad. It's a design that embodies both Force Touch, first mentioned for the Apple Watch, and the all new Taptic Engine technology.


Previously


Before these models appeared, the state of play for trackpads on MacBooks (and any other ultra-format laptop for that matter) was a metal plate hinged at one end and usually balanced on springs.

Apple's design concept had two elements. Firstly, to remove any physical motion from the trackpad and replace it with a more subtle tactile feedback through the fingers. And secondly, to offer a range of pressure sensitive click events. This redesign would mean the trackpad becomes slimmer, lighter and more elegant - goals which always rank highly on Apple's product enhancement agenda.

As this new trackpad doesn't have any physical movement, click actions are triggered by finger pressure. Captured by four sensitive pressure sensors, this push is translated into a click event, but the sensors can also determine the level of pressure exerted, so a 'deeper' kind of push can be translated into a Force Touch event.

Force Touch


Internally, a Force Touch event is interpreted as something akin to a third mouse button press, a feature never before seen on any previous Mac mouse or trackpad, even though it's quite a familiar option to Windows and Linux users.

This new kind of event opens up many possibilities for an enhanced user experience and, importantly, this experience will vary between individual applications and even while performing different tasks in the same application.

An Apple support document (see Links boxout) provides examples of some of the shortcuts possible from a Force Touch event. These include:
• Safari and Mail links (web page preview).
• Finder file icons (document preview).
• Finder file name (edit file name).
• iMovie (timeline animation style options).

But what about providing some sort of click feedback? Because there are no moving parts, this was a key aspect of the design challenge.

It was achieved by employing some clever vibrational technology, which fools the brain into thinking the trackpad has actually moved during a click, so let's examine how it actually works.

Taptic Engine


Many users will still expect some kind of feedback, as a substitute for the lack of physical movement. This is achieved through high-frequency vibrations and something Apple call the Taptic Engine.

At its heart the Taptic Engine hardware is pretty rudimentary. In essence, it's a series of electromagnets directly connected to a metal rail. Mounted underneath the trackpad's metal surface, this rail induces buzz-like vibrations every time a click event is detected. For a Force Click event, there's a second buzz.

It might seem a little counterintuitive, but a precise horizontal jolt underneath a trackpad feels, to the finger tip, just like a downward click. In theory, the Taptic Engine could imitate a wealth of virtual feedback - for example, the edges of buttons and other user interface components, and even animated button movement.

High Quality


There's nothing particularly revolutionary here. Similar feedback motors appear in game controllers and quite a few other devices. Yet, once again, it's all about Apple's quality of design, engineering and attention to detail.

The four pressure sensors are able to determine a wide range of pressure values, rather than just a click and force click. As they're located in the four corners of the trackpad, the software behind these sensors can determine the exact location of the finger (or stylus) applying the force.

Therefore, it's possible to capture both movement and pressure data at the same time, and this data opens up a multitude of smart interface opportunities, such as advanced signature verification, handwriting recognition and pressure sensitive drawing/painting tools.

Using the operating system 'Settings' app, virtual click pressure can be adjusted to become more or less dicky depending on your personal preference, and haptic feedback can be switched off completely if so desired, although many will consider this vibrational feedback an asset.

In addition, there's an audio feedback option, with three different settings. The lowest audio setting is most applicable for those who already prefer the 'tap-to-click' option, rather than a physical hinged press type of click.

Haptic History


As always at an Apple event, the speakers like to present new features as something that could only have been envisioned inside its highly secretive labs. However, this isn't a new concept. In fact, haptic texture research can be traced back to work done by Margaret Minsky at the Massachusetts Institute of Technology (MIT) and Vincent Hayward at the McGill University over 20 years ago.

Completed in 1995, Minksy's doctoral thesis centred on applying a lateral force to simulate the feeling of texture. A custom software environment called Sandpaper subjected specific patterns of horizontal force to a joystick, which allowed users to 'feel' various textures.

At around the same time, haptics pioneer Vincent Hayward, who has written dozens of papers on the topic, was producing phantom clicks with horizontal forces in his McGill University lab. Of course, 1990s technology was far less advanced than today, and Hayward's prototypes weighed as much as a modern MacBook. What Apple's lab team have done is to translate historical research into desirable technology for everyday computing products.

Force Touch

Electrovibration


But some will argue that Apple's Taptic Engine technology shouldn't be seen as the latest and greatest take on haptic feedback. Other research groups are already starting to release details of their own intriguing tactile investigations.

One such group can be found inside Disney's Research labs in Pittsburgh. With help from colleagues at Carnegie Mellon University, this team is investigating a very different technique that dispenses with electromagnets altogether.

Named by Disney researchers as electrostatic vibration (and previously called TeslaTouch), it's based on a phenomenon known as electrovibration.

The idea to use electrovibration as feedback came from an accidental discovery by Edward Mallinckrodt in 1954. Dragging his finger over a conductive surface covered with a thin insulating layer and excited with a 110-volt signal, he felt a characteristic rubbery sensation.

What had happened is that an attractive capacitive force was generated between his finger and the metal surface. Although very weak, this force managed to simulate a rubbery surface touch sensation.

Varying the voltage, Mallinckrodt generated a range of tactile sensations that could simulate the act of touching other objects, such as glass, stone and types of textiles.

Electrovibration Benefits


Since it doesn't need any motors or moving parts, the electrovibration surface can be any size or shape - as big as a room wall or curved like a car windscreen. And electrovibration is easily combined with other tactile technologies, including capacitive and resistive touch screens. Most importantly, electrovibration feedback technology isn't exclusive to Disney. Companies like Senseg (senseg.com) also exploit this phenomenon and have demonstrated its distraction-free capabilities to automotive manufacturers.

And at least one Microsoft Research group also seems to be exploring electrovibration, so a future Microsoft Surface type cover may well offer a quite different kind of typing and trackpad experience.

App Updates


Apple has already started to update a selection of OS X apps to cater for context-sensitive Force Touch events and Taptic Engine feedback.

So what sort of changes does this new trackpad usher in? Let's say you have a map on the screen. Using a Force Touch, you can zoom in and out multiple levels, without clicking the same button repeatedly, and you can drop a pin anywhere you'd like with a single Force Touch. It's fast and intuitive.

In iMovie, a Force Touch action allows you to quickly advance the timeline's frame rate. One single 'deep' press moves cycles through 2x, 4x, 8x and so on - much faster than clicking four times to raise the frame rate to 16 times normal speed.

As far as haptic feedback is concerned, an iMovie support document says that you'll feel a snapping sensation when you move movie clips around during editing. And you're also supposed to feel the beginning and end of each separate clip.

Many more apps already support a Force Click event, and the list keeps on growing.

To try it out for yourself, visit an Apple store or other Apple seller. After all, other manufacturers are bound to follow Apple's lead, which means your next laptop may well include similar technology.

More To Come


As always, the Apple rumour mill suggests it has bigger plans for Taptic Engine technology.

Apple could, for instance, allow users to feel the display interface under their fingertips by simulating smooth, bumpy or sticky sensations. In this way, text fields, buttons and scrolling lists could all feel very different to the touch.

This kind of feature would be extremely useful for tablets and smartphones, where certain tasks could be performed without having to look at the screen at all.

Take the virtual keyboard, for example. By running your fingers over it you'd be able to sense the different keys, giving a realistic touch typing experience.

Obviously, the Taptic Engine mechanism itself must be miniaturised for these kind of devicese, and this could be a problem. In reality, the electrovibration approach may be far more applicable for small-screen products, which means in future it may be Windows and Android handheld devices that offer the most flexible and cost-effective haptic feedback experience.