From left to right: evolution of EHF sensors with built-in radar, source: Google AI
Millimeter waves (MMW) or extremely high frequencies (EHF) is a range of radio waves with a wavelength of 10 mm to 1 mm, which corresponds to a frequency of 30 GHz to 300 GHz. They are used in military and police radars, security scanners, EHF therapy for the treatment of many diseases, astronomical instruments.
But now the era is approaching when millimeter-wave (mmWave, 60 GHz) radars are ripe for widespread commercial use. Soon they will be allowed to be built into regular smartphones. This opens up radically new possibilities for using portable gadgets: gesture recognition in cars, remote sleep monitoring, tracking the movement of all people in the office, and much more.
Previously, the use of mmWave radar in consumer electronics required special FCC approval. Google was the first to receive it in 2018, then Vayyar, Amazon and some other companies.
On July 13, 2021, the FCC proposed to open the 60 GHz band to commercial mmWave applications. The decision sparked the green light for dozens of start-ups and new commercial products that were allowed to freely use EHF radars without special FCC clearance.
The possibility of a wide application of EHF radars also appeared due to the progress in the manufacture of RF CMOS
RFCMOS
The RF CMOS chip (RF CMOS) or RFIC integrates radio frequency (RF), analog and digital electronics. Recently, it has become possible to place radar, antennas, and computational cores on a single chip, which makes it possible to produce mmWave sensors at a much lower price.
Radar works by transmitting electromagnetic waves and processing the response waves reflected from surrounding objects. The signal at 60 GHz has a wavelength of 5 mm, which allows mmWave to receive detailed information about us and our physical environment.
For example, this is how the result of processing the reflected EHF signal on a smartphone looks like if a person approaches (left), moves away (center) or makes a hand gesture (right).
EHF radar data on a smartphone, source: Google AI
4D
On EHF microcircuits, there are usually several antennas, so the sensor receives a picture, as it were, in four dimensions. In addition to three coordinates, the sensor also captures the speed of each reflected point. Learn more about how 4D radars work here.
The diagram below compares four types of common sensors: ultrasound, camera (2D), lidar (3D), and radar (4D). As you can see, the radar is the only sensor that receives information about the speed of an object.
Another advantage of the radar is that it can see through plastic and other materials. This makes it an excellent choice for consumer electronics as it can be placed under the body of any gadget.
The radar is immune to environmental factors such as light, temperature and dust, so it works great in environments where the photosensor is useless. At the same time, it is extremely reliable, there is literally nothing to break in the microcircuit: there are no moving parts or lenses.
The real benefit of mmWave radars comes when they are integrated into AI systems. For example, researchers at Carnegie Mellon University last year trained a neural network that accurately categorizes different types of activity from radar data: arm-waving, squatting, cycling, clapping, lunging, jumping.
EHF radars in household appliances
The first prototype of a commercial device with a built-in radar was shown by the Google ATAP (Advanced Technology & Projects) studio in 2015. It was a Google Soli smartwatch project developed by a team led by Ivan Popyrev (see article on e).
In short, Project Soli is a radar on a miniature chip that can be embedded in any surrounding objects: for example, a mirror, a stove, a TV. Any object that interacts with a person. Now they will recognize finger gestures with an accuracy of less than 1 mm.
There is no need to install bulky cameras and additional equipment, especially since no cameras can track movements at a frequency of 10,000 FPS, as Soli does. Touchless gestures can be used to extend the UI of various devices.
In October 2019, Soli radar was built into the Pixel 4 smartphone (marketing name (Motion Sense), where it accelerated the face unlock process along with support for the recognition of several basic gestures, such as music control. But since then, the chip has not been included in any smartphone Pixel Probably, the developers decided to work on the application of this technology in smartphones.
In addition, in 2021, Google launched its second-generation Nest Hub smart display with Sleep Sensing.
Smart Display Nest Hub
The Sleep Sensing feature uses Soli's radar to monitor the sleep of the person next to the display based on their movements and breathing - all without a camera or some kind of wearable sensor.A teardown of this $60 gadget revealed an Infineon radar chip that costs $3.65.
Google Nest Hub (2nd generation)
Amazon launched the Ring Video Doorbell Pro 2 smart doorbell with a built-in EHF radar in early 2021. The corresponding functionality for consumers is touted as '3D Motion Detection' and 'Bird's Eye View' functions. The radar has fewer false positives and a wider range than a standard IR camera. In addition, he sees through bushes, trees, and other obstacles.
Amazon has also followed Google's lead by using EHF radar to monitor sleep quality.
Apple has not yet released anything with a built-in radar, but in early 2021 it received a patent for a device with a circular radar-antenna array.
Based on mmWave technology, next-generation office monitoring devices work, which accurately track the location of employees in the office in real time.
Novelic, Arbe, Vayyar, Uhnder, and Oculii and others are working on automotive EHF radars.
Startup Miku has developed a baby monitor with a built-in radar Miku Pro Smart Baby Monitor, which not only captures the baby on video, but remotely monitors breathing rate and sleep quality.
The technology can also be used for wireless communication both indoors and outdoors. All major cellular operators are now working on the implementation of mmWave. There are also third-party projects such as Facebook Terragraph and startup Aervivo.
The WiGig standard (WiFi at a frequency of 60 GHz) was adopted back in 2009 and is gradually gaining momentum. For example, Intel created a wireless adapter for the Vive VR headset using WiGig, which has near-zero latency and a data rate of 4.6 Gbps.
The latest WiFi standard combines radar and communication techniques for wireless sensing with WiFi routers. That is, the WiFi router will be able to effectively track the movement of objects around the apartment and recognize gestures.
As you can see, progress in the production of EHF radars and the decision of the FCC to open the 60 GHz spectrum helped a number of gadgets with fundamentally new functionality enter the market. Apparently, mmWave is another breakthrough technology that will greatly affect the interfaces around us. Perhaps in a few decades, gesture recognition will be a natural feature of many gadgets and other surrounding objects.
To promote the new standard and unified APIs, the Ripple Technology Alliance was formed in January 2022.
bbabo.Net