Sensing solutions are the unsung heroes of consumer electronics

Originally published in Fierce Electronics. By Rosa Chow, VP software engineering at InvenSense, a TDK group company.

The smartphone in your pocket is far more powerful than the computer in the Apollo 11 moon lander, and the most advanced phone you buy 10 years from now should have more processing power than all phones today combined. Improvements in sensor technology similarly transform consumer electronics, expanding their capabilities and increasing their value.

Smartphones are empowered by sensors to perform navigation, step tracking, and screen orientation. Drones require sensors for altitude detection and more. Augmented reality and virtual reality (AR/VR) systems rely on precise motion sensors to enable realistic, immersive experiences. Cameras and other products with embedded optical sensors benefit from sensor-based electronic image stabilization (EIS). As such technology advances, engineers demand sensors with higher accuracy, lower power consumption, and faster recalibration.

Rapid advancements in chip design, miniaturization, artificial intelligence (AI), materials science, and sensor fusion allow for significantly increased power and expanding capabilities in increasingly smaller and more compact devices. As processing capabilities and sensor solutions evolve, so too will consumer electronics.

Motion and magnetic sensors in particular are paving the way to new and richer features. These sensors are fundamental in cutting-edge hearables, wearables and extended reality applications.

Aerial drones are a thriving business, with wide adoption in the military, commercial, and consumer markets. 

Every advance in MEMS and magnetic sensors leads to greater precision in navigation and positioning, including altitude.

An entirely different set of sensors including optical, radar, and ultrasonic time of flight (ToF) sensors, complement MEMS and magnetic sensors by adding object detection and collision avoidance.

Sensor manufacturers keep improving the fundamental design and their products, but also their use in conjunction with software, including sensor fusion software, and AI.

Motion and magnetic sensor solutions empower new consumer applications

There are three sensor types used in various combinations to provide motion sensing and position sensing. Gyroscopes measure orientation or angular velocity. Accelerometers measure acceleration. Magnetometers measure a variety of the characteristics of magnetic fields.

Any one of these will make measurements along three axes. Combine two and the result can be described as a 6-axis sensor; combine three and it is a 9-axis sensor. Add on-chip sensor fusion software to the 6-axis or 9-axis, and you create an easy-to-integrate, turnkey solution enabling absolute orientation detection with always-on calibration.

Today’s sensor solutions are more miniscule, powerful, precise and energy-efficient than ever, enabling motion tracking, heading and navigation that’s fast, accurate and ultra-low power – ideal for next-generation hearables, wearables, extended reality, and more.

Let’s imagine what’s possible with the latest sensor solutions with on-chip software.

Hearables, a rapidly evolving use case

One of the use cases re-energized by advances in motion and position sensors is hearables. Advanced sensors are making possible a new category of earbuds with true wireless stereo (TWS).

Earbuds have relied on microphones to detect when the user spoke to trigger a response (turning down the volume of audio content or suspending it, for instance). A problem with microphones, however, is that they are prone to responding to any voice, the user’s or someone else’s nearby. The accelerometers in TWS earbuds’ IMUs, used as vocal vibration sensors, are far more accurate than microphones at distinguishing the user’s voice from the voices of others. Once the IMU ascertains that it is definitely the user who is speaking, then the MEMS (micro-electro-mechanical system) microphone in the earbuds can take over.

Also, earbuds commonly have buttons or pads that users can tap, double tap, triple tap, or wide-area tap to invoke different functions (stop/start, fast forward, etc.). But when the earbuds are equipped with motion sensors, users can add head gestures, such as shaking and nodding (or use those gestures to replace physical tapping). Motion sensors of course can also detect activity and inactivity to activate and deactivate functions.

Wearables

While we now prize our fitness trackers for their ability to monitor vital signs, such as heart rate and blood oxygen level, those are add-ons. From the earliest generation, fitness wearables have been based on motion sensors to make it possible to count steps.

Motion sensors subsequently became the basis for a whole new category of fitness monitoring –- sleep quality. The integration of such sensors made it possible to indicate how much a person was tossing and turning in their sleep, versus how much time was spent still, sleeping soundly.

While steps are undeniably useful for gauging cumulative physical activity, the “step” is also an entirely imprecise measure. New motion and position sensing technology, buttressed by AI, can classify activity, fairly accurately distinguishing between activities such as walking, running, and biking.

As with earbuds, gesture recognition in fitness wearables is possible because of motion and position sensors. It is possible to allow a gesture, such as a vigorous shake, to activate functions. The example here is earbuds, but gesture recognition can be customized for other products.

Since we’re mentioning physical activity, we’ll also just note that it is now common to embed motion sensors in sports equipment. The use of sensors is being enthusiastically embraced by a range of sports leagues from European football to American football, which are embedding sensors in all sorts of equipment, ranging from balls to golf clubs to players’ jerseys.

Health-tech wearables

Health-tech wearables are a distinct category from fitness wearables. We have heightened, stricter expectations for performance and reliability, and so health-tech wearables are subject to an entirely different and more rigorous set of standards.

Health-tech wearables can include anything from something akin to fitness trackers to pacemakers to continuous glucose monitors (CGM). Not all these items are natural vehicles for motion sensors, but the health and medical profession is considering how integrating motion and position sensing can improve patient care. Here, the application of motion and position sensors is still a bit more speculative, but the potential value is clear.

Health-tech wearables with motion and position sensors can make it easier to monitor people with a wide variety of conditions for their own safety. That can include people with frailties resulting from mishaps, disease, or simply age. Remote monitoring could give the subject greater autonomy, and yet caregivers can still be alerted quickly if active support might be necessary at any given moment. Is the patient still, for how long have they been still, where are they, and are they vertical or horizontal? Individual patient circumstances will inform what type of response is appropriate.

Extended Reality – augmented, mixed, and virtual reality

VR is slowly but inexorably growing in popularity, especially for video games. Meanwhile, AR glasses hold great potential, though built-in displays and speakers could provide better imaging and sound, and physically they’d benefit from being lighter and smaller still. Those improvements are being made, and more will come.

AR and VR are at least as dependent on position and motion sensing as any other product, and maybe more so.

All AR and VR headgear depend on motion sensing to detect how and where users move their heads. In classical VR, where the user is fully immersed in a virtual environment, motion sensing is critical to make sure that the illusion of being in a virtual environment is fully supported – the scene must scroll with players’ movement.

Tracking for scrolling might be less of a concern in AR, but it is still important, especially when using an app that provides a virtual overlay in the real world. When virtual overlays must overlap real-world objects, the virtual and real images must be tracked with each other. The need for accurate position sensing in these instances is acute.

All of that is fairly obvious. What many users of AR and VR don’t realize is how important image stabilization – EIS – is to the experience. User complaints of disorientation were far more common before AR and VR systems began to integrate image stabilization technology, which of course is based on motion and position sensors. The more accurate the motion and position detection, the smoother the AR/VR experience is.

Motion and positional sensing are, of course, essential to handheld VR gear – wands, gloves, guns, and the like. Positional sensing coupled with object detection is also extremely important in VR – the data from the two are combined and used to warn users away from bumping their shins on furniture or tripping on the dog.

High-quality sensors are a necessity for high-quality experiences

As we’ve talked about applications, it is important to note that sensor performance is key –- not all motion and position sensors are equal. Motion and position sensors or IMUs can be vetted by two important performance parameters –- energy consumption and susceptibility to drift and noise.

An important performance parameter for IMUs is energy consumption; some are more energy-efficient than others. Wearables generally have power consumption restrictions that are even more stringent than those for smartphones. Product designers often look for the lowest-power motion and magnetic sensors that fit their applications’ needs.

Also, sensors can be subject to drift, a problem that tends to compound when indoors or underground for long periods of time. Susceptibility to drift varies by supplier. 

The most accurate motion and magnetic sensors have on-chip software enabling always-on calibration.

Always-on calibration makes indoor tracking far easier. Imagine being able to find the cat when it’s hiding, or knowing you can trust the location on your smartwatch even when on a subway train.

AR/VR applications also benefit from power efficiency as well as improvement in tracking precision. Lower noise and better sensitivity are welcome improvements, combined with a stable output in the presence of magnetic shocks.

Conclusion

The examples offered here include some of the most widely used categories of consumer electronics, but position and motion sensors are essential in a very wide variety of applications, from IoT to factory robotics to agricultural vehicles.

Motion and position sensing are essential to the evolution of electronics, improving existing capabilities when not creating the opportunity to add new features.

Every improvement in sensors that detect position and motion makes our navigation apps that much more precise, our wearables that much more useful, and our VR experiences that much more enjoyable.

Thanks to motion, position, and magnetic sensors, more objects can take flight than just birds or planes. While Superman may be a fictional character, the days when humanoids take flight may not be too far out, thanks to MEMS and magnetic sensors.

Rosa Chow is Vice President of Software Engineering at InvenSense, a TDK group company. She leads software, firmware, and algorithm product development to deliver sensor innovations.