Panasonic Corporation announced today that it has developed a time-of-flight (TOF) [2] image sensor that uses avalanche photodiode (APD) [1] pixels and is capable of acquiring highly accurate 3D information in accordance with the position of objects from near to far places at distances up to 250 m. The sensor can be applied in a variety of fields including automotive range imaging and wide-area surveillance.
The newly developed TOF image sensor has achieved the integration of the world’s best one million pixels*1 through the area reduction of APD pixels, enabled by the vertical stack structure of the electron multiplier and the electron storage. This is based on the technologies of the TOF image sensor that uses APD pixels announced by Panasonic in June 2018. Furthermore, in addition to long-range and high-resolution 3D range imaging that has been developed by Panasonic, the new sensor has also achieved high ranging accuracy. The simultaneous establishment was previously difficult for conventional TOF sensors and LiDAR. The new sensor has enabled the accurate detection of overlapping people and objects in distant places.
- *1:Comparison of TOF image sensors equipped with multiplication pixels during video imaging operation that have been presented at conferences, etc.
(As of Feb. 18, 2020, according to Panasonic data)
This development brings the following advantages.
1. High-resolution and high-sensitivity TOF image sensor that uses APD with one million pixels, four times the pixel count of Panasonic’s conventional sensor*2.
2. Acquisition of 3D range images with high ranging accuracy using TOF, with 10 cm interval sensing enabled even for long distances from 10 m to 100 m (15 times the ranging accuracy compared with Panasonic’s conventional sensor*2)
- *2:Panasonic’s long-range TOF Image Sensor announced in June, 2018.
This development has been realized based on the following technology developments:
1. High-sensitivity APD pixel technology
The area of APD pixels is significantly reduced while the multiplication performance is maintained through the vertical stack structure of the electron multiplier that amplifies photoelectrons and the electron storage that retains electrons.
2. Photon accumulation and time division indirect TOF technology
The new sensor has realized 3D range imaging with a ranging accuracy of 10 cm intervals for long distances from 10 m to 100 m by converting the number of incoming photons [3] to an integration signal and applying the indirect TOF calculation [4] used for short distances.
Conventional TOF sensors are incapable of detecting weak light signals and are therefore limited to short-distance detection. On the other hand, LiDAR is capable of long-distance detection; however, its low resolution makes it hard to identify small objects. The long-range TOF image sensor equipped with APD (APD-TOF sensor) announced by Panasonic in June 2018 is capable of detecting small objects from short to long distances; however, its ranging accuracy was 1.5 m intervals at the shortest and therefore had an issue of detecting overlapping people and objects in distant places.
[Suitable applications]
Industrial ⁄ monitoring ⁄ automotive sensing cameras
[Notes]
These development results were announced in the 2020 International Solid-State Circuits Conference on February 17, 2020, held in San Francisco from February 16 to 20, 2020.
[Product features]
1. High-resolution and high-sensitivity TOF image sensor that uses APD with one million pixels, four times the pixel count of Panasonic’s conventional sensor*2.
Image sensors such as conventional TOF sensors convert a single photon that has entered their pixels to only a single electron and therefore have an issue of low sensitivity when the signal light is as weak as approximately the single-photon level due to its susceptibility to noise (Fig.1 left). Conventional APD sensors, referred to as SPAD (Single Photon Avalanche Diode), which multiplies electrons in a pixel, have a large pixel area and therefore difficulty in miniaturization (Fig. 1 right). The newly developed vertical stacked APD (VAPD) has reduced the area by vertically stacking the photoelectric converter, electron multiplier, and signal storage, thereby forming very fine pixels of 6 μm in pitch. This has enabled an image sensor that achieves a combination of high sensitivity signals amplified 10,000 times and a high resolution of one million pixels (Fig. 1 middle).
Fig.1 Comparison diagrams of the pixel structure of conventional sensors and
the new vertical stacked APD sensor
2. Acquisition of 3D range images with high ranging accuracy using TOF, with 10 cm interval sensing enabled even for long distances from 10 m to 100 m (15 times the ranging accuracy of Panasonic’s conventional sensor*2)
Generally, TOF ranging calculates distances by measuring the time-of-flight of photons emitted from a light source to hit an object, reflect, and return (Fig. 2). The light reflected from a place more than 10 m away means that in terms of probability, a signal of one photon barely arrives. Panasonic has previously developed a 3D range imaging technology with a ranging accuracy of 1.5 m intervals up to long distances of 250 m using a unique short-pulse TOF method. This was made possible by developing a weak light detection technology that reliably captures the weak signal of a single photon through the use of unique integration circuits built in all the APD pixels, which count the number of incoming photons in terms of probability (Figs. 3 and 4). The company has newly developed a photon accumulation and time division indirect TOF technology that converts the number of incoming photons to an integration signal and applies indirect TOF calculation [4] used for short distances (Fig. 5). This has enabled 3D range imaging with a distance accuracy of 10 cm intervals for long distance (10 m to 100 m), which was difficult for conventional TOF sensors and LiDAR (Figs. 6 and 7). By achieving the high ranging accuracy in addition to long-range measurement and high resolution, the new sensor is capable of acquiring 3D range images in which the position and shape of overlapping people and small objects can be discerned even from a long distance (Fig. 8).
Fig. 2 Explanatory diagram of the short-pulse TOF method
Fig. 3 Principle explanation and an image example of image acquisition by
the APD-TOF sensor for each range using short pulses
Fig. 4 Explanatory diagram of the principle of detecting objects
in distant places using integration in the pixels
Fig. 5 Explanatory diagram of integration photon signals using integration circuits
in the pixels
Fig. 6 Explanatory diagram of operation that achieves the ranging accuracy of
the APD-TOF sensor in accordance with the object distance
Fig. 7 3D range image acquired by the APD-TOF sensor using the photon accumulation
and time division indirect TOF technology
Fig. 8 Comparison of range images by the APD-TOF sensor described
in this announcement and the one announced in June 2018
[Term descriptions]
[1] Avalanche photodiode
An ordinary photodiode generates one electron from one photon. On the other hand, an avalanche photodiode (APD) applies a strong electric field to the one electron generated from a photon to cause the electron to strongly collide with other electrons in the material, thereby generating one electron. This collision will be repeated with increasing scale with the initial collision as a trigger as if it were an avalanche, resulting in 10,000 times the number of electrons. SPAD (Single Photon Avalanche Diode) is a type of APD.
[2] Time-of-flight
The time-of-flight (TOF) refers to the time-of-flight of light. Conventional TOF ranging consists of a light source and a light detector. The light emitted from the light source hits a subject and the reflected light arrives at the detector. During this period, the light is traveling at the speed of light. The distance to an object can be calculated by measuring the time-of-flight of light since the speed of light in the air is constant. This method, which directly measures time, is referred to as direct TOF.
[3] Photon
A photon is the smallest unit of light energy, which cannot be broken down further. It is considered difficult for image sensors used in conventional cameras to detect a single-photon light signal because these sensors have noise with energy of about one photon.
[4] Indirect TOF calculation
Method of calculating the distance between the pulse emission point and the object by irradiating light pulses at the same phase with the sensor shutter and using the out-of-sync amount of the return light pulses to calculate the time-of-flight of the photons. This is a calculation method adopted by conventional short-distance TOF cameras.
SOURCE: Panasonic
