ToF: Understand Current Assisted Photonic Demodulator (CAPD)

Time of Flight technology works by measuring the time delay of light traveling from a light source, reflecting off a scene, and into a sensor. The specific type of Time of Flight the Sony IMX556 CMOS sensor uses is called continuous wave (CW) modulation, also known as CW phase-shift or Indirect Time of Flight. Instead of measuring the time delay of a single light pulse, the Helios camera emits continuously modulated light, and the phase-shift between this emitted light and the reflected light is used to calculate distance. In order to accurately calculate the phase-shift, the IMX556 utilizes a current assisted photonic demodulator (CAPD) pixel structure that samples incoming light synchronously with emitted light modulation. CAPD allows for an alternating voltage inside each pixel’s photodiode, creating drift fields that divide and pull electrons to alternating detector junctions. In the simplified example below, modulated light is emitted from the VCSEL (Vertical Cavity Surface Emitting Laser) diode. The light is reflected back into the pixel photodiode where it is converted into electrons and divided between the alternating detector junctions. This allows for better light collection efficiency.

Side Note

Sony IMX556 DepthSense ToF sensor specs:

  • Backside Illuminated CMOS
  • 640 x 480 px
  • Global shutter
  • 10.0µm pixel size
  • 1/2″ sensor size

Side Note

The CAPD ToF image sensor technology was first introduced in 2006 from the founding members of the Belgium based company Softkinetic, acquired by Sony in 2015. Below: CAPD diagram from “Time-of-flight Optical Ranging Sensor Based on a Current Assisted Photonic Demodulator” by D. Van Nieuwenhove, W. Van der Tempel, R. Grootjans and M. Kuijk.

Divide and Conquer

Once the photons are converted to electrons within the photodiode they must quickly be pulled towards the detector junctions for the camera to accurately calculate the phase shift. Located in each detection junction are positive (p+) and negative regions (n+). When a current is introduced to produce a drift field, the electron holes (h+) move towards the lowest voltage p+ region and the electrons (e-) move in the opposite direction towards the highest voltage region (n+). CAPD is extremely fast and effective. It is able to quickly alternate drift field polarity producing very high demodulating contrast. A high contrast ratio means more electrons are directed to the right junction based on their return time, allowing for higher quality phase shift calculations. This results in better accuracy and precision with the DepthSense sensor. In addition, CAPD’s drift field is able to reach deep into the photodiode accelerating as many electrons as possible towards the detection junction. This, combined with Sony’s backsided technology helps produce a high quantum efficiency of 57% at 850 nm. As a result, the Helios camera can achieve an accuracy of less than 5 mm and a precision of less than 2 mm at 1 m camera distance.

1 Frame, 4 Phases

The speed of CAPD allows the IMX556 to sample incoming light with 4 different phases for each depth frame. Each sample, also known as a micro-frame, is phase-stepped by 90° (0°, 90°, 180°, 270°) with each one having 3 stages: reset, integration, and readout. During the reset stage, the pixel voltage is reset back to baseline. The Integration stage occurs when the drift-fields are introduced by the current to direct the electrons towards the detection junctions. During the readout stage all pixel data is read on the sensor. After the readout stage the sensor enters idle time to reduce power and lower heat levels.

It is only necessary to have the 0° and 90° micro-frames to calculate depth. As the distance of an object changes, the values of b1 and b2 in the 0° and 90° micro-frames will change. Ambient light is also easily removed by subtracting the values of b2 from b1. However, even if the collected photons are the same for b1 and b2, there are always slight fluctuations (caused during photon to electron conversion and electron to voltage conversion). It is important to cancel out these fluctuations and their influence on b1 and b2 values. By having 2 additional micro-frames that are 180° offset from 0° and 90° (which effectively reverses the roles of b1 and b2) and by adding the values of b1 and b2 to their opposite 180° micro-frame (and vis-versa) the camera cancels out these fluctuations in each pixel.

CAPD + Backside Illuminated CMOS

CAPD provides an effective way of demodulating and collecting electrons inside the photodiode, but reflected light must first reach the sensor’s photodiode unobstructed. Sony’s backside illuminated (BSI) technology ensures the maximum amount of light reaches the photodiode. Unlike traditional front-side CMOS sensor design, backside illuminated sensors place the wiring layer below the photodiode improving light sensitivity. This placement removes any wiring or circuit obstructions that might otherwise block some of the incoming light.

Reducing unwanted image artifacts is another benefit of the IMX556. Because the IMX556 is a CMOS sensor it performs better against smearing and blooming versus CCD ToF sensors. Smearing and blooming can cause erroneous 3D data and is created when the charge overflows the pixel well capacity, spilling into adjacent pixels. This is usually caused by intense light sources or reflective surfaces. In many ToF use cases, it is important to minimize these factors. The IMX556 CMOS used in the Helios camera is much more forgiving in reducing these image artifacts.

Credit: https://thinklucid.com/tech-briefs/sony-depthsense-how-it-works/

Tech Leader/PM who focuses on 3D (Imaging/Photonics/Sensing, LiDAR/ToF) and 3A (AV/Auto, AR/VR, AIoT/5G)…. linkedin.com/in/ymarc; http://4da.tech/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store