David Briddock uncovers the latest technology from Google’s skunkworks-like group
The Advanced Technology and Projects group (ATAP) is but a small division in the huge Google empire. Yet its impact on the direction of the company and the technology industry as a whole is enormous. The members of this skunkworks-like group are a pretty radical set of individuals – even for a free-thinking organisation like Google. In fact they define themselves as a “small band of pirates. Believers. Makers.”
The ATAP group only has around 100 staff, but the key to pushing its radical, groundbreaking ideas into real products is its fast-growing list of external commercial partners, which already number in the hundreds. Yet, while there’s plenty of activity going on, in true skunkworks tradition, details of the group’s projects are hard to come by. However, at its recent I_O 2015 developer conference Google unveiled some fascinating technology – and a number of surprises.
Project Soli
Project Soli was debuted at the Google I_O 2015 event by project founder Ivan Poupyrev (ivanpoupyrev.com) and lead developer Jaime Lien. As the presentation gathered pace the knowledgable audience migrated from hushed amazement to wild enthusiasm about Soli’s potential; by the end, they clearly showed their delight in what this team had been able to achieve. Soli technology has been a hot topic of conversation ever since. The key question that’s being asked is: ‘Has Google just made current motion detector/controller obsolete?’
It’s too early to answer this just now, but there’s no doubt Project Soli appears to be a game changer in terms of user interaction. It captures tiny finger movements and full hand gestures in three dimensions. Yet there’s no need to touch a physical surface – and it doesn’t involve any kind of camera technology.
Soli’s Miniature Radar
So how does it work? It’s all down to an innovative new sensor coupled with some very smart software.
Soli sensors broadcast high frequency radar waves and capture any reflections. This arrangement allows it to distinguish movements in the sub-millimetre range and ascertain the distances of objects from the sensor. It can also capture motion at up to 10,000 frames per second, which is far faster than any camera-based system could manage.
Highly specialised software algorithms then make sense of these complex wave patterns. These algorithms themselves were devised through the application of advanced machine learning technology – which of course is one of Google’s key strengths.
Most importantly the laptop-sized sensor development rig has been shrunk down to a single thumbnail-sized chip. With help from certain manufacturing partnerships, all this was done within an amazing 10 month timescale; there’s already talk of a Soli developer kit release later this year.
Soli Applications
It’s clear the Soli sensor chip and accompanying software could be integrated into just about any device, from PCs and laptops to tablets and smartphones. However, Soli really excels when the screen shrinks to watch-sized dimensions. The ability to precisely control a smartwatchbased app through natural, real-time, in-the-air finger movements and gestures makes existing interfaces, including the Apple Watch, seem clumsy and dated.
In fact, because no physical touch display or camera capture system is needed Soli is applicable to a wide cross section of home appliances and products – like portable bluetooth speakers. Soli would also be perfect for in-car control, navigation and entertainment systems.
Wearables are another obvious area, from Google Glass-like applications and wrist bands, to technology that could be embedded into our everyday clothing.
Which brings us neatly to the next ATAP project...
Project Jacquard
Immediately after his Project Soli presentation, Ivan Poupyrev announced another initiative close to his heart: Project Jacquard (g.co/projectjacquard). While it continues the theme of improving user interaction with small-screen or no-screen devices Project Jacquard is all about expanding the user interaction area. It does this through some clever smart wearable technology that goes far beyond the rather crude integration we typically see today. As Poupyrev mentioned in his presentation the problem with smart watches, fitness bands another similar types of devices is that not everybody wants to wear one. However, we all wear clothes, so why not make them smart?
Jacquard Yarn
The key to Jacquard’s considerable potential is that it’s based on a new type of yarn. It’s a blend of existing fibres (cotton or synthetic) and highly conductive threads, braided together in the usual way. This approach solves a number of tough challenges. The yarn looks and feels like any other textile yarn and can be used on existing industry weaving machines to manufacture garments at scale. It can be made in any colour so textile fashion, design and innovation is unimpeded. And the finished article can be pulled, stretched, folded and washed.
The Jacquard team also worked with industry partners to design special processes that bring the woven conducted fibres together into small, discrete connectivity points using 3D meshing techniques. The end result is a garment with one or more regions that act like multi-touch sensing trackpads. These regions recognise swipe, tap or pinch gestures and even identify the number of fingers involved.
Jacquard Potential
On stage, Poupyrev himself wore a Jacquard-enhanced ivory jacket, hand-made by a Saville Row tailor, that could interface with smartphones, tablets and watches.
Google also managed to recruit the Levi Strauss Company as a partner. In fact, Levi’s head of global product innovation, Paul Dillinger, walked onto the stage to extol the virtues of Jacquard technology and state his company’s desire to make interactive clothing a reality.
Jacquard technology opens up a brand new vista in the clothing industry. One where fashion designers and software developers can work together to create an endless variety of smart clothes and garment apps that seamlessly integrate into our everyday lives.
Project Tango
One of the most intriguing ATAP initiatives presented back at I_O 2014 was Project Tango. In essence Tango empowers a mobile device with the same kind of visual and topology sensing we ourselves use to navigate the physical world. It delivers this fine-grained local-area spatial perception through the application of advanced computer vision, special vision sensors and image processing software.
Tango Technology
In case you’re wondering, Tango isn’t just another AR/VR headset scenario. Instead these immersive experiences are generated in a real-time fashion on the screen of a modified Android tablet. This tablet has an extended array of sensors and cameras that capture up to 250,000 measurements a second. The captured 3D spacial data targets motion tracking, depth perception and analysis of the surrounding area.
Motion tracking enables the device to understand its position and orientation in real-time. Depth sensors evaluate the shape of the immediate locality, and the device uses visual cues to help recognise nearby objects. In practice this means Tango can build a 3D model of an object or complete room. Something that would obviously be a boon to 3D modellers and 3D printing projects.
Tango Gaming
To showcase how Tango technology can enhance gameplay a number of companies have produced Tango-centric games such as Zombie Gunship Reality, Cowpocalypse, Bullseye’s Playground and Project Tango Explorer.
Taking Zombie Gunship Reality from Limbic Software as an example, it’s a fresh spin on Limbic’s hit mobile game Zombie Gunship, which already has a following of over 10 million players worldwide. Here, the simple gameplay sees the player installed in the gunner seat of a heavily-armed AC-130 ground attack aircraft (goo.gl/w3KoBN) and tasked with eliminating a zombie threat while defending the remaining survivors of the ongoing apocalypse. By taking advantage of the Tango hardware, the player is immersed in a world where the device itself becomes the gunship.
Tango Future
Project Tango has attracted a large number of hardware and software partners, including ABSquare, Infineon, NVidia, Open Source Robotics Foundation, NASA’s JPL and the University of Minnesota. And the second incarnation of the Tango-modified 7” tablet-based dev kit is now available for developers (goo.gl/na5kpX).
However, this dev kit is for US developers only, and costs a hefty $1000. There’s no sign of a product launch programme in the near future, so what’s next for Project Tango? Will it remain a separate project? Will it become entangled with another ATAP project? Or will it be quietly dropped in a Google Glass-like fashion? Only time will tell.
Project Abacus
Nobody likes using passwords; more importantly the security risks they engender are well known and frequently publicised. Just like Apple, Microsoft and others, Google wants to provide a better solution to the security authentication problem. An ATAP group has been working on a system that captures user interaction and other distinctive patterns – everything from how they type to their walking gait – to create a unique fingerprint for each and every user.
Project Abacus was set up to gather this kind of data from a large group of individuals, recruited from a total of 33 universities across the US. The result is 40TB (40,000GB) of sample data, which can be analysed and investigated by purpose-built machine learning algorithms. One of the goals is to build a solution that can be deployed as a software update to existing smartphones and tablets. One that also offers a range of authentication scenarios, from the low level access to the kind required for online banking and commerce services.
However, Google is also keen to establish a robust, hardwarelevel solution to securing personal data.
Project Vault
We’re all familiar with the SIM card in our mobile phones. It’s there to securely protect network data and services, and it does a pretty good job. Google, however, has been asking why there isn’t something similar to protect what’s important to us? The solution devised by the Project Vault team is a rather an innocuous looking SD card. Nevertheless, inside it holds is a complete implementation of a security computer running a specialised Real-Time Operating System (RTOS) on an ARM processor, plus a short-range NFC wireless antenna for authentication purposes.
Led by security guru and ex-hacker, Peter Zatko, the ATAP team have added a suite of cryptographic services, including hashing, signing, batch encryption and a hardware random number generator.
Intriguingly the team ensured RTOS has no implicit drivers. Instead it appears to have just two files, one read and one write. It stores encrypted data onto what the host device thinks is bad disk sectors. And this driverless card works on any platform – Windows, OS X, Linux, Android and so on. If it has an SD slot, it just works.
Data is transferred via fully encrypted cypher text, which only makes sense to the sender and receiver. Most importantly no keys or cryptospecific data are shared between the communicating devices.
When can we buy one? Unfortunately, at this point in time that question is unanswered.
Project Ara
Do you remember Project Ara? It’s a flexible, multi-module, snaptogether smartphone where the owner determines the module combination and so dictates what functionality their handset actually contains. Google has been working on Ara since 2013 and we’ve mentioned it in the Micro Mart pages a number of times since then. Yet, despite enormous media and public interest it doesn’t seem to be any closer to becoming a product you can buy.
On stage at its I_O 2015 conference Google demonstrated a working Ara handset that was plugged together and boot in around 60 seconds. A missing camera module was then added and instantly recognised by the device’s operating system. However, don’t get too excited just yet. We’re afraid these devices aren’t going to appear in your local phone shop anytime soon.
Google has admitted the challenges were much tougher than it originally envisioned; while it said a market pilot would take place in 2015, the disappointing punchline was that we’d have to wait for the next developer conference for a news update.
Which means it could be years before we have an Ara smartphone in our pocket.
Magic Leap
Finally we have the intriguing Magic Leap project. Unfortunately, this is still firmly classified as top secret. So what do we know so far? Well, it’s attracted interest and financial backing from a wide range of highly influential individuals and organisations, who have already pledged a total of over $500 million. Which all sounds very encouraging, as with this sort of funding it would be pretty much impossible for Google to quietly drop this particular project.
However, commercial funding appears to have forced Magic Leap out of the mainstream ATAP environment. This ensures it can remain highly secretive, even for Google employees who already work on other ATAP projects. Some suggest, though, that it may lead to the birth of a brand new company built around the technology.
Magic Leap Innovation
Judging from the carefully leaked details it appears to be a direct competitor to Microsoft’s HoloLens headset, which blends augmented reality and virtual reality. Interestingly both may appear as purchasable products sometime in 2016.
The key difference with HoloLens is that Magic Leap technology isn’t supposed to need a display screen. Instead is seems there’s a tiny projector that beams images directly into the retina via a transparent lens. The talk is that this arrangement fools the eye and brain into believing digital images actually exist in the real world.
All we have so far is a few tantalising pieces of information and a couple of short videos, the latest of which looks pretty impressive (see Links boxout). So, for now, the future of Magic Leap is still shrouded in mystery.
ATAP Future
What we’ve covered is a pretty impressive list of ideas and innovation. Yet there’s bound to be even more unannounced hardware and software initiatives going on right now.
And Google seems to have learnt a few lessons from Google Glass and is keen to bring commercial partners on board as soon as possible. That means many of these ideas, concept and prototypes are destined to become real products.
Links
ATAP videos: goo.gl/olnez3
Project Soli video: goo.gl/ciB9HP
Project Jacquard video: goo.gl/IPNjKR
Project Tango home: goo.gl/RuRcZq
Project Vault video: goo.gl/EbuEiV
Project Ara demo video: goo.gl/Xx3ESu
Magic Leap video: goo.gl/iCczCX