Difference between revisions of "Markerless, inside-out tracking"

From Virtual Reality, Augmented Reality Wiki
Jump to: navigation, search
(→‎Markerless, inside-out tracking Systems)
(One intermediate revision by the same user not shown)
Line 1: Line 1:
 
==Introduction==
 
==Introduction==
−
[[Markerless, inside-out tracking]] is a composite term derived from two separate concepts, and refers to a method of tracking objects in three dimensional space.
+
[[File:Intel Project Alloy.jpg|thumb|Figure 1. Intel's Project Alloy (Image: gizmodo.com)]]
  
−
[[markerless tracking|'Marker-less']] refers to the lack of [[fiducial markers]] used in this type of tracking, while [[inside-out tracking|'Inside-Out']] refers to the method by which data is gathered for the tracking.
+
[[Positional tracking]] is an essential component of both [[virtual reality]] (VR) and [[augmented reality]] (AR), contributing to a greater sense of [[immersion]] and [[presence]]. It determines the position and orientation of an object within the environment. In VR, this allows for the movements of the user to be translated into the virtual environment, and in AR it is essential for the placement of digital content into real objects or spaces. Markerless inside-out tracking is a composite term that defines a form of positional tracking that uses two specific methods: [[markerless tracking]] and [[inside-out tracking]]. <ref name=”1”> Boger, Y. (2014). Overview of positional tracking technologies for virtual reality. Retrieved from http://www.roadtovr.com/overview-of-positional-tracking-technologies-virtual-reality/</ref> <ref name=”2”> Ziegler, E. (2010). Real-time markerless tracking of objects on mobile devices. Bachelor Thesis, University of Koblenz and Landau</ref>
  
−
[[File:F2Ak4iE.jpg|thumbnail|A room covered in fiducial markers for inside out tracking, at [[Valve Corporation]]<ref>Youtube, Video: 'Steam Dev Days', on Channel: Valve, Published on Feb 11, 2014</ref>]]
+
Markerless tracking is a method of motion tracking that avoids the use of markers (also known as [[fiducial markers]]). These markers are usually placed in the environment or in the head-mounted displays (HMDs), helping the system determine the users or camera position. The Markerless method uses instead natural features already present in the environment, for tracking purposes. <ref name=”3”> Virtual Reality Society. Virtual reality motion tracking technology has all the moves. Retrieved from https://www.vrs.org.uk/virtual-reality-gear/motion-tracking</ref> <ref name=”4”> Klein, G. (2006). Visual tracking for augmented reality. Ph.D. thesis, University of Cambridge, Department of Engineering</ref> This method is considered more flexible and effective since there isn’t the need for a prepared environment with markers. <ref name=”5”> Zikas, P., Bachlitzanakis, V., Papaefthymiou, M. and Papagiannakis, G. (2016). A mobile, AR inside-out positional tracking algorithm, (MARIOPOT), suitable for modern, affordable cardboard-style VR HMDs. In Digital Heritage. Progress in cultural heritage: documentation, preservation, and protection. Springer International Publishing, Switzerland</ref> <ref name=”6”> Lima, J.P., Roberto, R., Simões, F., Almeida, M., Figueiredo, L., Teixeira, J.M. and Teichrieb, V. (2017). Markerless tracking system for augmented reality in the automotive industry. Expert Systems With Applications, 82: 100-114</ref> The decreasing cost of computer vision is another factor that has made markerless tracking an attractive alternative of tracking. However, this method still has some problems, such as motion blur and fast motion affecting tracking. <ref name=”7”> Fang, W., Zheng, L., Deng, H. and Zhang, H. (2017). Real-time motion tracking for mobile augmented/virtual reality using adaptive visual-inertial fusion. Sensors, 17</ref> It also demands a trade-off between precision and efficiency. <ref name=”2”></ref>
−
==Marker-less Tracking==
 
−
Because of the difficult computations required for a software to interpret live camera input, tracking solutions that rely on cameras sometimes place [[fiducial markers]] (which may look like QR codes), in view of the tracking camera. This gives the software a known pattern to look for, which makes computation simpler and faster.  
 
  
−
Thus the term, 'marker-less' refers to a system that is robust enough that it does not need the aid of printed markers to aid in its interpretation of the three dimensional space.
+
Inside-out is a term that defines the position of the cameras or other sensors relative to the object to be tracked. If they are placed in a stationary location exterior to the HMD, for example, it is considered [[outside-in tracking]]. In inside-out, the camera is placed in the HMD. <ref name=”8”> Boger, Y. (2014). Positional tracking: "Outside-in" vs. "Inside-out." Retrieved from https://vrguy.blogspot.pt/2014/08/positional-tracking-outside-in-vs.html</ref> There is great interest in inside-out tracking since it simplifies the setup and allows the user a greater sense of freedom by not being limited to a specific tracking space. Inside-out positional tracking has the potential to make VR completely wireless, more convenient and accessible. Some have gone as far as saying that it was the “holy grail” of the VR scene. <ref name=”9”> Robertson, A. (2017). Self-tracking headsets are 2017’s big VR trend — but they might leave your head spinning. Retrieved from https://www.theverge.com/2017/1/12/14223416/vr-headset-inside-out-tracking-intel-qualcomm-microsoft-ces-2017</ref> <ref name=”10”> Durbin, J. (2016). Google: Wireless positional tracking “solved”, but heat still a problem for VR. Retrieved from https://uploadvr.com/inside-out-google-solve-tracking/</ref>
  
−
==Inside-Out Tracking==
+
==Advantages of markerless inside-out tracking==
−
Tracking systems that make use of a camera may be organized into two main branches, 'inside-out' and 'outside-in' tracking. Both terms refer to the placement of the tracking camera itself, with reference to what it is tracking.
+
The benefits of using markerless inside-out are those that come from the individual advantages of each tracking method.
  
−
In an [[inside-out tracking|'inside-out' system]], the tracking camera is placed within the item being tracked (for our purposes, like a [[Virtual_Reality#Devices|head mounted display]]), from which vantage point it look ''out'' at the world around it. It uses its changing perspective on the outside world to note changes in position.
+
===Markerless===
 +
* Can work within unprepared environment;
 +
* No special hardware required;
 +
* No fiducial markers;
 +
* Able to use natural features to calculate position and orientation. <ref name=”5”></ref> <ref name=”7”></ref> <ref name=”11”> Hsia, J.S. (2017). Markerless AR: 4 Things developers need to know. Retrieved from https://developer.att.com/blog/markerless-ar-developers</ref>
  
−
In an [[outside-in tracking|'outside-in' system]], a tracking camera is (or cameras are) placed around the space within which the tracked object will move, and use their changing view of the object itself to measure its changes in position.
+
===Inside-out===
 +
* Increased mobility and freedom;
 +
* No need for external sensors;
 +
* Not affected by occlusion;
 +
* Provides a deeper sense of immersion since the user is not tethered. <ref name=”9”></ref> <ref name=”12”> Langley, H. (2017). Inside-out v Outside-in: How VR tracking works, and how it's going to change. Retrieved from https://www.wareable.com/vr/inside-out-vs-outside-in-vr-tracking-343</ref>
  
−
==Relative Merits==
+
==Project Tango and Project Alloy==
 +
[[Tango]] is a [[Google]]’s project that allows smartphones and other mobile devices to understand their location in space. It is a software and hardware solution that offers inside-out technology and markerless tracking. According to Google Tango’s director of engineering, Johnny Lee, “As you can see [inside-out positional tracking] clearly works on this phone…We’ve even had people strap a tablet sized device with Tango built in into a custom VR headset and the positional tracking worked just as well as it does here…The only thing really holding us back right now are the thermals. Right now, phones just get way too hot if we ask them to run positional tracking and split-screen stereoscopic image at 90 fps…As Daydream matures and Tango continues to improve, the sophistication around tracking will also evolve over the next 2-3 years.“ <ref name=”10”></ref>
  
−
Both systems have merits; 'inside-out' tracking is notable because it can require no outside equipment, which is ideal for a portable device. Unfortunately, in the case of a headset, it places the burden of computation on the headset itself. (A burden which is exacerbated when no fiducial markers are used.)
+
In 2016, Intel announced [[Project Alloy]] (Figure 1) - a VR system. It is also a markerless inside-out tracking device. During CES 2017, Intel showed off a developer kit of the project. Project Alloy uses a tiny camera array called Intel RealSense. These cameras are able to make a 3D map of the environment (e.g. a room) and the objects in it. The headset does not need wires, and it is powered by a battery. <ref name=”13”> Nunez, M. (2017). Intel’s Project Alloy is what a VR system should be. Retrieved from http://gizmodo.com/intels-project-alloy-is-what-a-vr-system-should-be-1790818104</ref>
−
 
 
−
'Outside-in' tracking can be less computationally demanding, and can make use of multiple cameras to make results more stable and consistent. This comes in part from the reduced chance of [[occlusion]]. Unfortunately it requires a controlled environment and more extensive equipment.
 
  
 
==Markerless, inside-out tracking Devices==
 
==Markerless, inside-out tracking Devices==

Revision as of 14:26, 17 July 2017

Introduction

Figure 1. Intel's Project Alloy (Image: gizmodo.com)

Positional tracking is an essential component of both virtual reality (VR) and augmented reality (AR), contributing to a greater sense of immersion and presence. It determines the position and orientation of an object within the environment. In VR, this allows for the movements of the user to be translated into the virtual environment, and in AR it is essential for the placement of digital content into real objects or spaces. Markerless inside-out tracking is a composite term that defines a form of positional tracking that uses two specific methods: markerless tracking and inside-out tracking. [1] [2]

Markerless tracking is a method of motion tracking that avoids the use of markers (also known as fiducial markers). These markers are usually placed in the environment or in the head-mounted displays (HMDs), helping the system determine the users or camera position. The Markerless method uses instead natural features already present in the environment, for tracking purposes. [3] [4] This method is considered more flexible and effective since there isn’t the need for a prepared environment with markers. [5] [6] The decreasing cost of computer vision is another factor that has made markerless tracking an attractive alternative of tracking. However, this method still has some problems, such as motion blur and fast motion affecting tracking. [7] It also demands a trade-off between precision and efficiency. [2]

Inside-out is a term that defines the position of the cameras or other sensors relative to the object to be tracked. If they are placed in a stationary location exterior to the HMD, for example, it is considered outside-in tracking. In inside-out, the camera is placed in the HMD. [8] There is great interest in inside-out tracking since it simplifies the setup and allows the user a greater sense of freedom by not being limited to a specific tracking space. Inside-out positional tracking has the potential to make VR completely wireless, more convenient and accessible. Some have gone as far as saying that it was the “holy grail” of the VR scene. [9] [10]

Advantages of markerless inside-out tracking

The benefits of using markerless inside-out are those that come from the individual advantages of each tracking method.

Markerless

  • Can work within unprepared environment;
  • No special hardware required;
  • No fiducial markers;
  • Able to use natural features to calculate position and orientation. [5] [7] [11]

Inside-out

  • Increased mobility and freedom;
  • No need for external sensors;
  • Not affected by occlusion;
  • Provides a deeper sense of immersion since the user is not tethered. [9] [12]

Project Tango and Project Alloy

Tango is a Google’s project that allows smartphones and other mobile devices to understand their location in space. It is a software and hardware solution that offers inside-out technology and markerless tracking. According to Google Tango’s director of engineering, Johnny Lee, “As you can see [inside-out positional tracking] clearly works on this phone…We’ve even had people strap a tablet sized device with Tango built in into a custom VR headset and the positional tracking worked just as well as it does here…The only thing really holding us back right now are the thermals. Right now, phones just get way too hot if we ask them to run positional tracking and split-screen stereoscopic image at 90 fps…As Daydream matures and Tango continues to improve, the sophistication around tracking will also evolve over the next 2-3 years.“ [10]

In 2016, Intel announced Project Alloy (Figure 1) - a VR system. It is also a markerless inside-out tracking device. During CES 2017, Intel showed off a developer kit of the project. Project Alloy uses a tiny camera array called Intel RealSense. These cameras are able to make a 3D map of the environment (e.g. a room) and the objects in it. The headset does not need wires, and it is powered by a battery. [13]

Markerless, inside-out tracking Devices

Meta 2

Microsoft HoloLens

Markerless, inside-out tracking Systems

Oculus Santa Cruz - Oculus VR

Project Alloy - Intel

Snapdragon VR - Qualcomm

Tango - Google

Windows MR Tracking - Microsoft

WorldSense - Google - inside-out tracking system for standalone Daydream headsets

References

  1. ↑ Boger, Y. (2014). Overview of positional tracking technologies for virtual reality. Retrieved from http://www.roadtovr.com/overview-of-positional-tracking-technologies-virtual-reality/
  2. ↑ 2.0 2.1 Ziegler, E. (2010). Real-time markerless tracking of objects on mobile devices. Bachelor Thesis, University of Koblenz and Landau
  3. ↑ Virtual Reality Society. Virtual reality motion tracking technology has all the moves. Retrieved from https://www.vrs.org.uk/virtual-reality-gear/motion-tracking
  4. ↑ Klein, G. (2006). Visual tracking for augmented reality. Ph.D. thesis, University of Cambridge, Department of Engineering
  5. ↑ 5.0 5.1 Zikas, P., Bachlitzanakis, V., Papaefthymiou, M. and Papagiannakis, G. (2016). A mobile, AR inside-out positional tracking algorithm, (MARIOPOT), suitable for modern, affordable cardboard-style VR HMDs. In Digital Heritage. Progress in cultural heritage: documentation, preservation, and protection. Springer International Publishing, Switzerland
  6. ↑ Lima, J.P., Roberto, R., Simões, F., Almeida, M., Figueiredo, L., Teixeira, J.M. and Teichrieb, V. (2017). Markerless tracking system for augmented reality in the automotive industry. Expert Systems With Applications, 82: 100-114
  7. ↑ 7.0 7.1 Fang, W., Zheng, L., Deng, H. and Zhang, H. (2017). Real-time motion tracking for mobile augmented/virtual reality using adaptive visual-inertial fusion. Sensors, 17
  8. ↑ Boger, Y. (2014). Positional tracking: "Outside-in" vs. "Inside-out." Retrieved from https://vrguy.blogspot.pt/2014/08/positional-tracking-outside-in-vs.html
  9. ↑ 9.0 9.1 Robertson, A. (2017). Self-tracking headsets are 2017’s big VR trend — but they might leave your head spinning. Retrieved from https://www.theverge.com/2017/1/12/14223416/vr-headset-inside-out-tracking-intel-qualcomm-microsoft-ces-2017
  10. ↑ 10.0 10.1 Durbin, J. (2016). Google: Wireless positional tracking “solved”, but heat still a problem for VR. Retrieved from https://uploadvr.com/inside-out-google-solve-tracking/
  11. ↑ Hsia, J.S. (2017). Markerless AR: 4 Things developers need to know. Retrieved from https://developer.att.com/blog/markerless-ar-developers
  12. ↑ Langley, H. (2017). Inside-out v Outside-in: How VR tracking works, and how it's going to change. Retrieved from https://www.wareable.com/vr/inside-out-vs-outside-in-vr-tracking-343
  13. ↑ Nunez, M. (2017). Intel’s Project Alloy is what a VR system should be. Retrieved from http://gizmodo.com/intels-project-alloy-is-what-a-vr-system-should-be-1790818104

Retrieved from "https://xinreality.com/mediawiki/index.php?title=Markerless,_inside-out_tracking&oldid=22286"