Drones Get Smarter and Safer

You can read this Chinese article at the link : Smarter drone for better safety

Author: 安西亞

As more people "own" drones and fly them in public spaces, many public safety issues have emerged. Therefore, improving drone flight safety has become a top priority.

Abroad, drones have been used in the military sector for some time, primarily to maintain homeland security. In certain professional civilian fields, such as agriculture, they have also been applied to tasks like spraying pesticides to protect farmers' personal safety. However, after small, consumer-oriented drones began to catch the eye of consumers, they became the next target market for the Internet of Things (IoT) at many exhibitions and among various manufacturers. Smartphones, watches, or other devices are also starting to integrate with drones in hopes of creating better sales. But as more people "own" drones and fly them in public areas, numerous public safety issues follow, making the enhancement of flight safety a primary concern.

With rapid technological advancements and continuous progress in semiconductor components, making drones smarter is a shared goal among related industry players. A smarter drone not only provides a better user experience for consumers or professionals but also further elevates its safety through features like active obstacle avoidance.

The Sizzling Appeal of the Drone Market

In 2012, the U.S. Federal Aviation Administration (FAA) estimated that the number of drones in the U.S. would reach 15,000 by 2020. Clearly, the FAA underestimated the drone's appeal. Currently, monthly drone sales in the U.S. far exceed this figure, and the market has even attracted fierce competition among many semiconductor and electronics companies.

[Free E-book Download in Chinese: New Automotive-Grade Optical Isolation Sensing Technology]

Among all regions, mainland China's drone market is developing the most vigorously. It has not only "nurtured" DJI, the world's largest drone manufacturer with a 50% market share, but research institutes also predict that by 2025, the total scale of China's drone market will reach 75 billion RMB, with an average annual growth rate expected to exceed 50%. This enticing market size has attracted many related companies to aggressively enter the Chinese market.

According to the UAV Drones Market research report, the major global drone market is projected to reach approximately 5.59 billion USD by 2020. The drone craze was also evident at the 2016 Consumer Electronics Show (CES) and other major consumer industry exhibitions.

Shun-Fan Yang, Deputy Manager of the iVOT Product Division at Novatek, stated that drones have recently received high attention from the media and industry. They not only saw excellent growth from 2013 to 2015 but also achieved great results in commercial, professional civilian, and consumer markets. This has attracted numerous semiconductor or system manufacturers, which is expected to make the market even more competitive and heated in 2016. A wave of mergers and acquisitions in the future drone market cannot be ruled out.

Additionally, starting in 2015, well-known processor chip suppliers such as Qualcomm, Intel, and NVIDIA officially announced their entry into the drone market, launching highly integrated smart chips specifically for drones. On one hand, the comprehensive solutions provided by chipmakers help lower the technical barriers to entry for manufacturers; on the other hand, these new processors allow drones to become further miniaturized and smarter. More importantly, the reduction in cost makes it easy for more consumers to purchase them.

Max Lai, Chairman of iSentek, believes that the aerial photography feature of drones, coupled with prices dropping from "exorbitant" to a universally affordable range, has caused this long-standing market to become incredibly hot.

Figure: 2014-2021 Commercial Drone Market Size (Source: Topology Research Institute)

Providing Different "Perspectives" to Attract Consumers

In the past, aside from dropping bombs, the most important task for military drones was environmental reconnaissance. Therefore, almost all drones were equipped with "eyes"—camera lenses. Whether in military or professional civilian fields, having "vision" is an indispensable condition. If consumer drones are equipped with photography features, offering consumers a different "perspective," they will be highly attractive new commodities.

Max Lai pointed out that although civilian drones have developed faster than commercial or consumer applications—replacing human labor for dangerous tasks like spraying pesticides or fertilizers, or even transforming into delivery drones in the commercial sector—consumer drones offer something unique to the general public. They can integrate with daily life, providing novel selfie angles and aerial views, creating new and appealing value for consumers. Therefore, consumer drones should hold a solid place in the future market.

On the other hand, whether viewed as a flying smartphone or a flying robot, the drone itself is a highly topical and novel product. Shun-Fan Yang believes that over the years, drones have continuously improved upon heavily criticized issues. The realization of technologies like manual hand-launching, binocular vision, and auto-hovering has made drone operation increasingly "foolproof." Today, drones are no longer devices only experts or hobbyists can master; they are gradually becoming entertainment devices anyone can easily control. Offering both room for imagination and endless fun, they are bound to remain in the spotlight of the consumer market. Most importantly, modern people are enthusiastic about taking selfies or live-streaming on social media, making consumer drones an excellent new medium.

Moreover, in the IoT world, drones can act as connected devices communicating with other IoT equipment. For example, when NXP envisions future autonomous vehicles, drones are integrated to transmit real-time traffic conditions or serve as tools for surprise deliveries.

Yang believes that drones require electromechanical integration technology, similar to the robotics/robotic arm electromechanical integration in IoT applications. This is the main reason chipmakers unanimously entered the drone sector—to invest early, train their teams, and accumulate development capabilities for connected electromechanical IoT devices.

Safety is Paramount for Both Professional and Entertainment Use

Whether for professional or consumer drones, safety is a crucial issue for the future. Given the frequent reports of drone aviation accidents—such as a drone crashing and injuring tourists at the famous Sun Moon Lake in Taiwan, or a drone crashing directly into Taipei 101 due to improper operation, not to mention a wide variety of accidents abroad—making drones smarter is another vital approach to enhancing safety, alongside regulatory constraints.

Yang stated that regulations should distinguish between commercial and consumer drones, as their usage scenarios differ. However, relevant standards and regulations are highly necessary for improving safety and will also aid the healthy development of the industry.

Max Lai pointed out that one of the reasons for frequent drone aviation accidents is poor mechanical design. This stems from manufacturers rushing into the booming drone market and neglecting mechanical optimization just to get products out quickly. Consequently, the US FAA has begun drafting regulations requiring drone registration and using GPS positioning to prevent drones from entering no-fly zones. Furthermore, drone specifications like battery life and flight distance can be used to further manage and restrict flight areas.

Additionally, Taiwan has recently enacted laws regarding drone no-fly zones. The Sun Moon Lake National Scenic Area Administration drafted the Precautions for the Use of Remote Control Drones in the Sun Moon Lake National Scenic Area, designating six areas—including four main piers, Xiangshan, and Shuishe Dam—as restricted flight zones. Violators face a maximum fine of NT$1 million. Although this rule currently applies only to a specific scenic area, it marks a first step.

Key Components Imbue Drones with Greater Intelligence and Safety

Besides regulations and mechanical design, semiconductor components also safeguard drone safety. Yang explained that unlike commercial or professional civilian drones, consumer drones do not require automotive- or military-grade component specifications. As long as automatic obstacle avoidance and indoor hovering functions are sufficiently precise, a certain level of safety can be ensured.

Setting aside mechanical design issues and focusing solely on flight safety, the Microcontroller Unit (MCU) or microprocessor acting as the "brain," paired with precision sensors, can further enhance the drone's flight intelligence. Max Lai further explained that 9-axis or 10-axis motion sensors, distance/light sensors, magnetic sensors, angle sensors, and even current sensors are closely related to drone flight safety. The automatic obstacle avoidance and hovering functions that drone makers heavily emphasize right now require distance/light sensors and magnetic sensors.

Magnetic sensors assist drones—often designed as quadcopters or octocopters—in determining heading directions during flight. They also help calculate the closest straight-line distance to avoid zigzagging and wasting fuel. Lastly, when hovering at a fixed point, magnetic sensors help the drone maintain a level state, allowing users to capture photos or videos from the best angle.

To perfect the automatic obstacle avoidance function, distance/light sensors are crucial. Yang pointed out that using binocular Computer Vision (CV) or infrared (IR) triangulation for depth and position information are currently the two most popular foundational technologies for obstacle avoidance. The key components here are System-on-Chips (SoCs) with high-performance CV computing capabilities or IR sensors.

In addition to IR sensors, some industry players believe that Time-of-Flight (ToF) sensors using lasers offer higher precision than traditional IR sensors, providing better obstacle avoidance for drones. Guozhi Lin, Technical Marketing Manager at STMicroelectronics' Imaging Division, stated that the advantage of laser distance measurement is its millimeter (mm) level accuracy, and the result is not affected by the target object's color or reflectivity.

In reality, sunlight significantly affects the accuracy of ToF light sensors, which tests the R&D capabilities of chip suppliers. Developing products that offer high precision, multi-point/multi-object detection, and real-time reporting will have positive synergistic effects on drone development.

Embedded Vision and Algorithms are Indispensable

Not only do high-precision sensors and high-performance main controllers elevate drone safety, but embedded vision and the algorithms used to process sensor data for obstacle avoidance during flight also play vital roles.

Max Lai believes that for drone flight control, not only must the sensors—the drone's sole sensory organs—be highly sensitive, but the quality of algorithms and communication technologies cannot be overlooked either. Otherwise, even with sensors, if the drone cannot effectively use the data to avoid obstacles, it will fail to deliver its intended professional or entertainment performance.

Due to application needs, professional drones currently have a greater demand for embedded vision (machine vision). However, future consumer drones will also incorporate embedded vision to integrate with Virtual Reality (VR) or Augmented Reality (AR) applications, or as part of their intelligent evolution, bringing consumers a completely different user experience.

Figure: By combining drone machine vision and AR, consumers can obtain more environmental information. (Source: iSentek)

Luo Lin, Senior Marketing Manager of Industrial, Scientific, and Medical (ISM) Technology at Xilinx, stated that some professional or commercial drones are already incorporating AR/VR, requiring 360-degree field-of-view capabilities. This allows the drone to intelligently know which areas to monitor upon arriving at a site and make real-time analyses and judgments.

Furthermore, obstacle avoidance also relies heavily on embedded vision. For instance, DJI has built 4K cameras into its new-generation drones for filming or obstacle avoidance, making the visual computing and analysis functions of embedded vision indispensable. Luo Lin emphasized that when dealing with 4K video at 30 frames per second, the required processing power far exceeds the capabilities of microprocessors or MCUs, making Field-Programmable Gate Arrays (FPGAs) a better choice.

Future Development: Even Smarter

Driven by various special applications, the professional and commercial drone market has gradually taken off. Consumer drones are also expected to have bright market prospects as their usability improves, prices become more affordable, and they offer more entertainment value. Yang believes that the largest application for consumer drones remains aerial photography, with small drones focused on entertainment selfies showing immense promise. To make consumer drones more acceptable, one prerequisite is making them smarter.

The primary use of consumer drones leans heavily toward entertainment and selfies. Therefore, making them smarter essentially means making the selfie function more "foolproof" and easier to operate, much like the once-popular digital point-and-shoot cameras. Yang emphasized that attempts to break traditional aerial photography norms are already visible, such as auto-follow, object tracking, and facial recognition. Features that simplify operations include automatic circling, voice/gesture control, object detection, indoor/outdoor hovering, autonomous return-to-home, and automatic obstacle avoidance. In other words, a smarter drone possesses "vision"—the ability to perceive and identify its surroundings—and an easier capability to take selfies. The key components to realize these abilities are high-performance CV SoCs and diverse sensors.

Max Lai emphasized that as long as they are closely integrated with personal life and step outside purely professional realms, the consumer drone market will flourish immensely.

Recognizing the development of the drone market, Taiwanese chip suppliers have begun positioning themselves and achieved good market results. Besides developing their own component technologies, they are also collaborating with processor companies like Qualcomm, or positioning their own sensors as Sensor Hubs to integrate all sensor and imaging-related components, acting as another brain for the drone's image processing. The efforts of Taiwanese companies prove they will not fall behind in the drone market race.