Latency is the delay between action and reaction. Having low latency is crucial when using a Head-mounted Display for VR or AR. As you move your head, your HMD should display the resulting changes immediately. Any significant delay will cause your brain to reject the legitimacy of the virtual world and cause you to lose presence. High latency can also cause simulator sickness. In AR, high latency can cause virtual content to be misaligned with real life features.
Minimizing latency is one of VR and AR's biggest challenges. Human brains can detect very small latency in the visual and audio systems. These systems have to bring latency low enough to be undetectable by the brain.
Motion-to-photon latency / End-to-end latency
Motion-to-photon latency also known as the End-to-end latency is the delay between the movement of the user's head and the change of the display of VR device reflecting the user's movement. As soon as the user's head moves, the VR scenery should match the movement. The more delay (latency) between these 2 actions, the more unrealistic the VR world seems. To make the VR world realistic, VR systems want low latency of <20ms and even really low latency of <7ms.
How to Reduce Latency
Decrease Pixel switching time
Increase Refresh rate
Predicting user's head movement.