FOVAS

Figure 1. A VR viewport. (Image: Pixvana.com)

Figure 2. Frustum projection. (Image: Pixvana.com)

Field of View Adaptive Streaming (FOVAS) is a virtual reality (VR) video technique that delivers better image quality with less streaming costs developed by the Seattle-based company Pixvana. It aims to provide viewers of VR video with the best quality possible while also helping content creators save money by streaming fewer data.

It achieves this by optimizing VR video quality using adaptive bitrates for video delivery, depending on the user’s direction of gaze. It breaks the stream into smaller, highly optimized videos that are optimized for a specific field of view, lowering the quality of the video stream for the angles that are not currently being looked at. According to Toyin Adeyemi, from Pixvana, the technique combines “viewbox image projection with multiple viewports to form an adaptive streaming solution for the highest quality VR videos in the market. The upshot for you is the best of both worlds: you get better resolution and lower streaming costs.”

Since there is a great amount of data contained within the highest resolution 360-degree video files, and VR video presents the extra challenge of dealing with where the user is looking at, with FOVAS the bitrate is increased to deliver a higher quality image to the location the user is viewing while reducing the image quality in the other places. This way, it is possible to achieve 8K or 10K video over 1080p bandwidth, as the company claims.

Other companies like Facebook and Google are interested in adaptive streaming in order to provide the best VR video streaming platforms. During October 2016, Valve revealed that it would be partnering with Pixvana to deliver an adaptive 360-degree video streaming system capable of 8k-10k resolution via the same bandwidth as a 1080p stream for the Steam platform. Pixvana has also developed the Pixvana XR Cloud - a platform for delivering and creating VR videos that includes native support for FOVAS processing.

VR video and FOVAS system

VR video is a new platform for storytelling that can provide the user with a deeper sense of engagement, transporting him to new places, with an increased sense of presence. The video quality influences the level of immersion that is created. Higher-quality video will immerse the viewer more since the experience that is being generated is seamless, while lower-quality VR videos will fall short with soft-looking imagery and distracting artifacts.

Currently, creating 360-degree video content is easy; the problem is in distributing this content to users around the world. This has had the effect of making companies rethink the way the stream their video content. Indeed, VR videos have a lot of data. According to Pixvana, “VR video begins as simultaneous streams from multiple cameras, sometimes as many as 24. These streams are then stitched together to create a spherical view that surrounds the viewer. This giant sphere can range from HD resolution to more than 10,000 pixels wide! The finished VR video is then typically squeezed down into 4K resolution (3840 x 1920) so that it can be transmitted over a standard network. But 4K isn’t nearly enough resolution to make a truly lifelike experience with today’s head-mounted displays. Often this results in soft and muddy video that breaks the feeling of immersion.”

With Pixvana’s field of view adaptive streaming solution, the VR video is deconstructed into different streams, each acting as a viewport (Figure 1) into a specific region of the video sphere. Each of the viewports contain enough pixels to fill a headset’s maximum resolution. Besides this, a viewport will contain the user’s current field of view as well as a lower-resolution version of out-of-view areas to maintain peripheral vision.

The mix of high and low-resolution image areas within a single video stream is achieved by using an image projection (i.e. a viewbox) technique that is non-proportional. The viewbox used in FOVAS is a frustum viewbox (Figure 2), which is like a cube but enlarged in the front face and shrunk in the back.