3D ToF (Time of Flight) image sensors or cameras enable us to acquire a real 3D depth map of the real world facing us. The scene, surroundings, and all objects in the captured image are transformed into the digital space in real time. With the built-in algorithms, the captured image data is used for measuring the relative distance between any object and the source of the light as well as the dimensions of any objects. The calculated measurements are used to track motions and to convert the acquired shapes of the objects into 3D models.
The 3D ToF image sensor shines a single modulated infrared light source onto the object of interest and then captures the reflected light to measure the amplitude and phase difference per pixel. The measurements are then used to generate an image of the distance and a gray-scale picture of the entire scene. In contrast to other techniques for 3D depth capture, such as scanning or stereoscopic method, 3D ToF camera has advantages of being extremely accurate, fast and inexpensive.
Working principle of 3D ToF
ToF (Time of Flight) technology utilizes the same principle as ultrasound distance measurement,
Distance = v x ΔTz, where V is the speed of the wave (v = c = 299,792,458 m/s for light and v = 343 m/s for sound) and ΔTz is the time between transmitting and receiving the light reflected by the object as shown in below Figure 1.
There are different ways for measuring the time ΔTz, such as the method of CW (continuous-wave) and pulse-based method. Most CW ToF cameras are based on CMOS image sensors, while pulsed-based ToF systems use non-CMOS image sensors, like CCDs). In common, all ToF 3D camera consist of a modulated light source, image sensor and image processor. The light source can be infrared or near-infrared. When capturing the image, the camera illuminates the scene of interest with the modulated infrared (IR) light. The modulated light is then reflected by the objects in the scene with shifted phases. The speed of light is constant, and the time it takes for the light to return as well as the phase shift is different at each point, so by measuring the phase shift and time of flight at each point, the distance of each point can be calculated at nearly real time.
3D ToF technology enables the system to “see” a real world in 3D and the dimensions as well as the real time distances between any two objects in the scene. This offers an unparalleled capability of depth perception and pattern recognition. 3D ToF camera can capture the depth data of every point in a scene at once it illuminates the modulated light.
AR (Augmented Reality) and VR (Virtual Reality)
In the AR/VR headsets, the ToF 3D camera generates real time 3D point clouds that enable the AR/VR software to better recognize the surrounding around any object of interest in the scene. This enhanced capability allows higher accuracy DSLR quality image effects for object location and facilitate more sophisticated and dynamic interactions between virtual and actual elements of the scene. On the other hand, with the capability of detecting movements of the players, the system can allow the play to intensively interact with the virtual elements with gestures without using HMI (Human Machine Interface).
Robotics can be more capable of recognizing objects and moving accurately with the help of real time 3D depth map of the environment. With the gesture recognition, the robots can interact with human directly in a collaborative application. For industrial applications, the robots with 3D ToF camera can inspect any product more accurately with 3D measurement, grab and place the product at place with high precision.
3D ToF camera sees huge potential in next generation cars. The technology helps the vehicles to acquire an excellent awareness of the environment in real time for rapid maneuvers and obstacle avoidance, which are indispensable capability of autonomous driving. The 3D ToF can be also used to monitor the state of the driver and passengers and take over the control of the vehicle in case of emergency. The gesture detection of 3D ToF camera can be used for setting the inside climate control or switching to an incoming phone call.
3D ToF enables us to recognize objects without contact to avoid cross-contamination. With gesture recognition, the doctors and nurses can perform accurate diagnosis remotely. Wit the 3D ToF technology, the patient can be remotely monitored to quickly respond to falling or hurting.
Security and Surveillance
The 3D ToF cameras provide high resolution 3D depth mapping of objects in the scene of interest. This is significantly advantageous over the traditional 2D imaging in that it offers high levels of confidence in classifying both people and objects in correlation to their precise positions.