As the automotive industry moves ever closer to an autonomous future, it is becoming ever more important to teach autonomous vehicles (AVs) not just to see, but also to perceive. While granting AVs the gift of sight is an important step towards the future of mobility, the ability to distinguish humans from animals or vehicles from buildings, for example, is a hugely important milestone to overcome.
Many systems have been developed in an attempt to achieve this goal, but, in the minds of some experts, significant doubts remain over the ability of technologies such as radar and LiDAR to operate flawlessly in all weather and lighting conditions. One such plug here could be the use of far infrared (FIR) sensing equipment, which creates an image of a vehicle’s surroundings using infrared radiation. Of those experimenting in this space, Israel-based Adasky is perhaps one of the best-placed players.
As Adasky’s Director of Sales Engineering, Sharon Fiss, explained to M:bility, the company’s decades worth of experience in FIR solutions for civilian and military use has made the transition to automotive far easier. Its wealth of expertise in the field has led to the production of the first high-resolution, thermal imaging camera for the AV. Known as ‘Viper’, this camera measures just 4.3cm by 2.6cm, and works by passively detecting and classifying living and inanimate objects through thermal signatures. The camera registers and combines emissivity—the rate at which an object emits heat—and thermal temperature in order to create an image, in a similar fashion as wild vipers do in order to identify and hunt their prey. This image can then be used by perception algorithms to detect and classify objects around the vehicle.
While vipers in the wild use infrared to seek and hunt, in the AV, Viper instead plays quite the opposite role—protecting, not endangering.
Today, the camera operates using the highest resolution available for automotive, VGA, and can run up to 60 frames a second while detecting a temperature variance of 0.05 degrees Celsius. All of these factors combined allow Viper to precisely identify obstacles, vehicles and pedestrians at a range of several hundred metres, allowing more time for autonomous and driver assistance solutions to react to any given scenario.
While this technology has its uses today, Adasky believes that it will support autonomous functions from Level 3 onwards. In particular, Fiss pointed towards how FIR’s increased perception ability will enhance the general public’s trust in an AV. “Adasky was founded on the understanding that there are huge gaps which remain in the automotive industry for understanding the vehicle’s environment,” said Fiss. “We have seen several high-profile incidents in the last few years that have limited the development of AVs. However, we see great potential in FIR to improve the safety of these AVs. Eventually, we will be able to run logic and decision-making processes to aid an AV, as well as implement intention recognition of other road users.”
Role of deep learning
Not unlike other similar technologies, Adasky’s solution leverages artificial intelligence (AI) and deep learning techniques to make the most of the solution. Specifically, deep machine learning plays a large role in the operation of Adasky’s technology.
Even though uses cases for AI have only recently emerged in automotive in a major way, Adasky has been making use of its resources, including that of its AI algorithm engineers, to develop its solution at a consistent rate since 2015. In that time, it has acquired and annotated over two million objects for its perception features, which run using complex machine learning techniques and end-to-end neural network algorithms. Such efforts will allow Viper to be scaled appropriately with demand. “As long as we continue to acquire data from locations all over the globe, as well as in various weather conditions, we will be able to continue improving our solution at a scalable rate,” Fiss added.
Other competitors
Automakers have also experimented with other 24/7 capable sensors. In 2016, Ford demonstrated as part of ‘Project Nightonomy’ that its Ford Fusion self-driving vehicle was capable of driving in pitch black conditions without FIR assistance. On a closed road in Arizona, Ford’s test mule was able to navigate the course using only radar and LiDAR assistance. Prior to Arizona, Ford also carried out a similar test in snowy conditions at the University of Michigan’s Mcity facility.
In terms of direct competition, one of the most relevant players for Adasky is Oregon-based perception specialist FLIR, which is also developing thermal infrared cameras for AVs alongside its work in the marine, industrial and security markets, to name but a few. However, Adasky remains confident that its solution is the current market leader. “Our FIR solution is the only one currently capable of operating 24 hours a day, seven days a week in any weather condition,” explained Fiss.
Applying Adasky’s FIR solution to the real world could bring significant positives. For example, Tesla’s fatal Autopilot crash in 2016—caused due to the Model S’ inability to distinguish between a white truck and bright skyline—could have theoretically been avoided using infrared rather than direct images to inform the vehicle of its surroundings. “If you take sun glare as an example, our competitors have yet to build a robust solution that can operate in direct sunlight,” added Fiss. “Our FIR camera is also the only one that is able to see through thick fog, dense rain, haze or dust.”
With these use cases already covered, attention at Adasky is beginning to turn towards applying its solutions in other areas of automotive. For example, Fiss pointed towards localisation and mapping as key markets that could be bolstered with the use of FIR. “Alongside the AV evolution, there is now increasing understanding that there are specific needs in the automotive market that could be answered by combining complementary technologies,” said Fiss.
As Fiss highlighted, developing sectors such as last mile delivery and ride-sharing have been labelled as potential fields for the use of FIR technology. “The technologies and infrastructure that will be required for smart transportation solutions to work safely could benefit through the inclusion of FIR,” Fiss continued. “At the moment, we are evaluating the potential value in these areas within the entire automotive ecosystem.”
Overall, the technology is still at an early stage of development, but Viper’s successful debut at CES 2019 could mark a turning point for the technology in automotive. Perhaps the myriad of LiDAR and radar booths which littered the Las Vegas Convention Centre floor at this year’s event could in the coming years be replaced by those touting the value of FIR for the AV.
This article appeared in the Q2 2019 issue of M:bility | Magazine. Follow this link to download the full issue.
