{"id":4160,"date":"2014-08-14T11:18:35","date_gmt":"2014-08-14T11:18:35","guid":{"rendered":"http:\/\/coherent-labs.com\/?p=4160"},"modified":"2017-11-13T12:08:59","modified_gmt":"2017-11-13T12:08:59","slug":"head-tracking-for-vr-part-1-overview","status":"publish","type":"post","link":"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/","title":{"rendered":"Head tracking for VR Part 1 &#8211; overview"},"content":{"rendered":"<p>For the last year or so there&#8217;s been a lot going on in the\u00a0<strong>virtual reality<\/strong> (VR) area and enhancing the immersion in the game. Oculus\u00a0VR released Oculus Rift which, pretty much, everyone knows about today (if you somehow don&#8217;t &#8211; check out their <a href=\"http:\/\/www.oculusvr.com\/\">website<\/a>); there&#8217;s also the the <a href=\"http:\/\/www.virtuix.com\/\">Vitruix Omni<\/a>\u00a0which makes you break a sweat and run around to control you character.<\/p>\n<p>Most recently, Amazon announced their <a href=\"http:\/\/www.amazon.com\/Fire_Phone_13MP-Camera_32GB\/dp\/B00EOE0WKQ\"><strong>Fire Phone<\/strong><\/a>, which had one very cool feature &#8211; it can track your head and adjust the (<a href=\"http:\/\/techcrunch.com\/gallery\/heres-what-the-amazon-fire-phones-crazy-3d-head-tracking-looks-like\/\">see here<\/a>). This Fire Phone head tracking feature got our attention here at Coherent Labs and we wanted to see if there&#8217;s any piece of software that offers that functionality for an ordinary webcam so it can be used in a game. Much to our surprise there weren&#8217;t really a lot of alternatives &#8211; e.g. <a href=\"http:\/\/facetracknoir.sourceforge.net\/\">FaceTrackNoIR<\/a>\u00a0or the Unity plugin from the Fire phone SDK, which is Android-only, or many people who did some experiments but never released anything. That got us the idea to create our own very simplistic head tracking library that other developers can use.<\/p>\n<p>Our quick research showed several options for doing the headtracking:<\/p>\n<ul>\n<li>Having a special motion dection device (e.g. Wii Remote)<\/li>\n<li>Depth camera (e.g. Kinect)<\/li>\n<li>Ordinary webcam<\/li>\n<\/ul>\n<p>Motion detection devices offer high precision but are somewhat annoying because you have to wear some trinkets so they work. One of the guys that experimented with this used the Wii Remote to track his head position, but the accuracy was reportedly better if you move the sensor bar instead of the remote &#8211; so he strapped the sensor bar to his head, which looked very funny.<\/p>\n<p>The depth camera is like an ordinary camera, but it can provide\u00a0depth information about the image. This is great, but these are not too common at the moment. Plus, you&#8217; d have to some extra work to detect where the head is, unlike motion detection devices.<\/p>\n<p>Ordinary webcams are a commodity now &#8211; laptops have them and you have a lot of cheap options for desktops. This is the most accessible option so far, but they have it has the disadvantage that it neither gives you depth information, nor you know for sure where the head is. Those can be overcome with some calibration and processing power, though &#8211; after all today&#8217;s processors are quite fast. The accessibility and cost have a much higher impact for the end-user, so this option seemed like the winner to us.<\/p>\n<p>The\u00a0initial plan is to use OpenCV for detecting the head position and its features so we can estimate the rotation and image space position. Using input from the user about the distance of her head in specific camera capture, the field of view of the camera, some trigonometry and approximations, we can make a pretty good guess about the depth using only the 2D image. After we have some estimation of the head&#8217;s position and orientation, we can use the data any way we like &#8211; change where the player is looking at, make a small adjustment to the camera&#8217;s position, modify the perspective and much more.<\/p>\n<p>There are of course some other details such as dealing with noise &#8211; i.e. the head detection in images is not always perfect and can give you bogus results that you&#8217;ll need to filter but that&#8217;s a topic for a future post.<\/p>\n<p>In the next part we&#8217;ll look into what&#8217;s needed for all this to work with OpenCV, the actual code that makes it work and performance measures and considerations.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For the last year or so there&#8217;s been a lot going on in the\u00a0virtual reality (VR) area and enhancing the immersion in the game. Oculus\u00a0VR released Oculus Rift which, pretty much, everyone knows about today (if you somehow don&#8217;t &#8211; check out their website); there&#8217;s also the the Vitruix Omni\u00a0which makes you break a sweat [&hellip;]<\/p>\n","protected":false},"author":10,"featured_media":9780,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[410],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Head tracking for VR Part 1 - Coherent Labs<\/title>\n<meta name=\"description\" content=\"In this blog post we make an overview of the head tracking for VR and its recent development.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Head tracking for VR Part 1 - Coherent Labs\" \/>\n<meta property=\"og:description\" content=\"In this blog post we make an overview of the head tracking for VR and its recent development.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/\" \/>\n<meta property=\"og:site_name\" content=\"Coherent Labs\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/CoherentLabs\/\" \/>\n<meta property=\"article:published_time\" content=\"2014-08-14T11:18:35+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-13T12:08:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/coherent-labs.com\/posts\/wp-content\/uploads\/2014\/08\/stencil.blog-post-feature-35.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Nick\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CoherentLabs\" \/>\n<meta name=\"twitter:site\" content=\"@CoherentLabs\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nick\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/\",\"url\":\"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/\",\"name\":\"Head tracking for VR Part 1 - Coherent Labs\",\"isPartOf\":{\"@id\":\"https:\/\/coherent-labs.com\/posts\/#website\"},\"datePublished\":\"2014-08-14T11:18:35+00:00\",\"dateModified\":\"2017-11-13T12:08:59+00:00\",\"author\":{\"@id\":\"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/0f7e122b7a2587b2ccaf7fe8f9964f94\"},\"description\":\"In this blog post we make an overview of the head tracking for VR and its recent development.\",\"breadcrumb\":{\"@id\":\"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/coherent-labs.com\/posts\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Head tracking for VR Part 1 &#8211; overview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/#website\",\"url\":\"https:\/\/coherent-labs.com\/posts\/\",\"name\":\"Coherent Labs\",\"description\":\"Cross-Platform Game UI Middleware\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/coherent-labs.com\/posts\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/0f7e122b7a2587b2ccaf7fe8f9964f94\",\"name\":\"Nick\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/cf9bc3aa3b82bb10f4cc082fad0faa59?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/cf9bc3aa3b82bb10f4cc082fad0faa59?s=96&d=mm&r=g\",\"caption\":\"Nick\"},\"url\":\"https:\/\/coherent-labs.com\/posts\/author\/nick\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Head tracking for VR Part 1 - Coherent Labs","description":"In this blog post we make an overview of the head tracking for VR and its recent development.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/","og_locale":"en_US","og_type":"article","og_title":"Head tracking for VR Part 1 - Coherent Labs","og_description":"In this blog post we make an overview of the head tracking for VR and its recent development.","og_url":"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/","og_site_name":"Coherent Labs","article_publisher":"https:\/\/www.facebook.com\/CoherentLabs\/","article_published_time":"2014-08-14T11:18:35+00:00","article_modified_time":"2017-11-13T12:08:59+00:00","og_image":[{"width":1200,"height":600,"url":"https:\/\/coherent-labs.com\/posts\/wp-content\/uploads\/2014\/08\/stencil.blog-post-feature-35.jpg","type":"image\/jpeg"}],"author":"Nick","twitter_card":"summary_large_image","twitter_creator":"@CoherentLabs","twitter_site":"@CoherentLabs","twitter_misc":{"Written by":"Nick","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/","url":"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/","name":"Head tracking for VR Part 1 - Coherent Labs","isPartOf":{"@id":"https:\/\/coherent-labs.com\/posts\/#website"},"datePublished":"2014-08-14T11:18:35+00:00","dateModified":"2017-11-13T12:08:59+00:00","author":{"@id":"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/0f7e122b7a2587b2ccaf7fe8f9964f94"},"description":"In this blog post we make an overview of the head tracking for VR and its recent development.","breadcrumb":{"@id":"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/coherent-labs.com\/posts\/head-tracking-for-vr-part-1-overview\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/coherent-labs.com\/posts\/"},{"@type":"ListItem","position":2,"name":"Head tracking for VR Part 1 &#8211; overview"}]},{"@type":"WebSite","@id":"https:\/\/coherent-labs.com\/posts\/#website","url":"https:\/\/coherent-labs.com\/posts\/","name":"Coherent Labs","description":"Cross-Platform Game UI Middleware","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/coherent-labs.com\/posts\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/0f7e122b7a2587b2ccaf7fe8f9964f94","name":"Nick","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/coherent-labs.com\/posts\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/cf9bc3aa3b82bb10f4cc082fad0faa59?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/cf9bc3aa3b82bb10f4cc082fad0faa59?s=96&d=mm&r=g","caption":"Nick"},"url":"https:\/\/coherent-labs.com\/posts\/author\/nick\/"}]}},"_links":{"self":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts\/4160"}],"collection":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/comments?post=4160"}],"version-history":[{"count":1,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts\/4160\/revisions"}],"predecessor-version":[{"id":9387,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/posts\/4160\/revisions\/9387"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/media\/9780"}],"wp:attachment":[{"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/media?parent=4160"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/categories?post=4160"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coherent-labs.com\/posts\/wp-json\/wp\/v2\/tags?post=4160"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}