Passthrough

Passthrough, also known as "video passthrough", is a capability of mixed reality/virtual reality headsets where they can display the outside world through cameras to the user in VR. This makes it a sort of pseudo-augmented reality.

One of the difficulties of getting this right is the positional difference between cameras on the outside of a headset and the eyes of the user. So the image has to go through a distortion process. On the Quest Pro and Quest 3, and potentially other devices, this uses a neural network.