Microsoft shows off HoloLens 2 mixed reality headset at MWC

The Power of Spatial Audio and Mixed Reality Collaboration: A New Era for Companies Like Mattel

Spatial audio is revolutionizing the way companies collaborate and work together. By using this technology, whole spaces around us can be utilized to collaborate as if we're all here together in the same room. This is not just a novelty, but a game-changer for industries like manufacturing, design, and product development.

Companies like Mattel are already harnessing the power of spatial audio and mixed reality (MR) collaboration to transform their work processes. At Mattel, they are undergoing a massive digital transformation that touches all aspects of their business. From designing and developing products to marketing and manufacturing, technology is playing an increasingly important role.

One of the key areas where spatial audio and MR are being used at Mattel is in their design and development process. Traditionally, designers, engineers, marketers, and manufacturers would meet face-to-face to discuss ideas and collaborate on projects. However, with the rise of remote work and global teams, this became increasingly difficult. Spatial audio and MR have provided a solution to this problem.

To demonstrate this, Mattel invited Gurjit, their CTO, onto the stage to show how spatial is being used to transform their design process. With the help of Hololens and Spatial, Gurjit explained that they can now bring together diverse teams of designers, engineers, marketers, and manufacturers from all over the world into a virtual "spatial project room". This reduces the need for travel, allowing everyone to come together in the same virtual space.

Gurjit showed how this spatial project room works by introducing his team's idea for a new line of aquatic toys. With just a few clicks, they could visualize their ideas and collaborate on them in real-time. "How about sea turtles?" one team member suggested. "That's really cool," another responded. They then discussed other ideas, such as sharks and jellyfish, all within the same virtual space.

What's impressive is that every object in this spatial environment is physical and tangible, allowing users to interact with them in a highly immersive way. Team members could scroll through images, pick up and manipulate objects, and even create 3D models of their designs. This tactile experience makes it easier to collaborate on design concepts and reduce the need for digital content.

Gurjit also showed how this technology can help streamline the design process by reducing the need for sketches and drawings. "I can actually pull up these sketches I did on my phone last night using that same spatial app," he explained. "I just pull up those photos and hit Send, and they're instantly transformed into this digital environment."

This technology also allows team members to contribute to the design process at any time, from anywhere in the world. The entire space is a persistent digital object that can be accessed and contributed to by anyone, whether they're using an AR or VR headset, or even a PC or mobile phone.

The benefits of this technology are numerous. It reduces the need for travel, allowing teams to come together more easily and reducing costs associated with business travel. It also speeds up the design process, allowing teams to collaborate on designs in real-time. Perhaps most importantly, it fosters collaboration and trust among team members who are working remotely.

One of the exciting applications of this technology is in training environments. Hololens has been used in a variety of different settings, from construction sites to operating rooms to even the International Space Station. In these environments, spatial audio and MR have enabled immersive and interactive training experiences that were previously impossible to achieve.

To demonstrate this, Maria was invited onto the stage to show how Hololens can be used for collaborative training. She explained how she had created a mixed reality experience using View Forea Veal and Azure Spatial Anchor, which allowed her to seamlessly join in on Alex's experience from afar. With just a few clicks, Maria could participate in the training scenario, interacting with 3D models of real-world equipment and even simulating gear failures.

Alex, who was already working through a simulated maintenance task using Hololens, was able to hand over control of the scenario to Maria. She then took over the simulation, using her own iPad to join in on Alex's experience. This seamless transition between remote and local participants is made possible by the power of spatial audio and MR.

As Maria explained, View Forea Veal allowed her to create a single mixed reality experience that could be accessed and contributed to by anyone, without worrying about which platform it would be viewed on. This means that team members can collaborate on training scenarios in real-time, regardless of their location or device.

The potential for spatial audio and MR in collaborative environments is vast and exciting. By harnessing the power of this technology, companies like Mattel are unlocking new possibilities for collaboration, innovation, and productivity. As we move forward into a world where remote work and global teams become increasingly common, spatial audio and MR will play an ever-more important role in transforming the way we work together.

"WEBVTTKind: captionsLanguage: ensay hello to hololens to now these are the smallest and most power efficient 2k displays in existence now they allow us to dramatically grow immersion while shrinking the size of the displays but immersion is about more than just the Holograms of place in the world immersion is also about how you interact with them hololens 2 adapts to you it adapts your hands and goes beyond the gestures to give you the satisfaction of direct manipulation letting experience what it feels like for the first time to actually touch a hologram to top it off hololens 2 now also understands precisely where you are looking enabling us to do things like understanding your intent and adapting the holograms in real-time to your eyes then using the same cameras we've enabled windows hello iris base biometric authentication the most secure and reliable enterprise-grade solution we optimize hololens to you for a long way and comfort keeping with microsoft's tradition of inclusive design with 3d scans the heads of thousands of people across a wide range of ages genders and ethnicities and we then use this data to create a device that's once again sets a highest bar for both organ onyx as well as comfort hololens 2 has a universal fit system that works on the broadest and most diverse range of people whether you were glasses or not putting it on should be as simple as putting on your favorite hat we reduce the weight of hololens to you by making the front enclosure entirely out of carbon fiber making a light and durable and a device that is ready for the modern workplace and we also fundamentally change how that weight is distributed making you feel as if the device is floating on your head now as a result of these and hundreds of other changes I'm incredibly proud that with hololens 2 we more than triple the comfort of the device hololens 2 sets a new standard for comfort in mixed reality headsets now the list of innovations in hallows Institute goes on and on but instead of telling you about it why don't we show you now it's Alex mentioned Holland's too is very comfortable it fits just like a hat and the only thing that's even more effortless is time automatically signed in with Windows hello and RS syndication the Holland's - is actually signing me in as I put on my device now not only does the Holland's to recognize me it also recognizes my hands look at this fully articulated hand tracking and as I move my hands around the Holland's - is actually calibrating to my unique hand size and of course not only does the Holland's to recognize me and my hands it also recognizes the world welcome to my mixed reality home this is the place where I have all the apps and content that I use every day let's check some of them out now I've seen many people put on Holland's the first time and the first thing people do when they put on the device is they reach out their hands and try and touch the Holograms and now you can look at the way that the hologram is responding to my hand almost inviting me to touch it and in fact I can just grab this corner to resize it or I can rotate it or move it that's right we are touching hologram this is an app i've got called spatial let me just put it right there I've got another app here called euphoria of view now it's a little big so let me just use two hands here to make it smaller and then rotate it so you can see there we go and then let me put it down here in your spatial maybe make it bit smaller yeah that's nice alright now let's switch gears and talk about a different kind of application I've got a browser over there but it's kind of far away and I don't really want to walk over there so let me just call it over with my voice follow me this is a browser that's running Microsoft teams which is a tool that we use back home to collaborate let me see what the team's been working on ok it looks like they've got a surprise for me in the playground app I just have to say the words show surprise all right so let me just open up that Start menu here and then place the app and then launch it so now we're actually exiting my mixed reality home and going into an immersive experience but notice that that browser that I had actually followed me in now this can actually be really useful when you have things like emails or PDFs that you need to reference while you're doing your work I don't really want to follow me around though I'm showing you all this cool stuff so let me just put it over here and then we'll get back to it later welcome to the playground we spent years exploring and refining interactions for Hollins - and the playground is just a tiny sampling of the many prototypes that we built tested and learn from our approach is basically to try out as many things as we could and look for the things that stood out so for example here I've got three sliders each of them is controlling this wind farm simulation but each in a different way using a different interaction technique one of the things we tried is this touch slider here so here I could just stick my finger on the slider and have it go to a particular value to control that wind speed there it felt okay we also tried this push slider so this guy kind of nudge from side to side kind of like an abacus which was interesting now the interaction that really took home the cake though was this pinch slider the way that works is you just pinch it and move it wherever you want and what we found was that people really like that tactile sensation for their fingers touching as they grab and then released and across the board for all interactions the audio and visual feedback was a grab move and then release we're all really critical for making this experience still connected oh this is just so satisfying I can't wait for you all to try this out all right now let's move on to a different kind of control buttons how do you press buttons on Hollins - well you just reach out and press them now one interesting thing that we found about buttons was that the size of the buttons actually impacted the way that people interacted with them so for example for the smaller one most people would use one or eighty two fingers but for the large one pretty much everyone use their entire hand and this is kind of interesting when you think about it because these objects don't really weigh anything they're just digital things but despite that people would treat them like real things almost as if the bigger one had more weight I just love the way these move and the sounds they play when I press them it's great all right how about something that uses all ten fingers well to test that out we built the piano so here I can just play a chord or I can play the keys one of the time all right so where's that surprise that the team had for me oh that's right I had to say those words show surprise look at that palming bird over there it's gorgeous I wonder if it will fly to my hand yeah oh wow this is beautiful I just love the way that it's following my hand around I've got to tell the team they've done a great job and in fact I don't even need to use my hands to do this because I can use my eyes and my voice that's right Hollins to has eye tracking so I could just look over this browser here and look at the bottom of the screen to scroll it and then send my message start dictation the hummingbird looks great exclamation mark sense so this is what we mean by instinctual interaction by combining our hands our eyes and our voice Holland's two works in a more human way today companies tackling the world's biggest problems are increasingly spread across the world and the hololens lets us work together as if we were standing next to each other face to face to show you how that works let me just flip into spacial here with my Hollins to a materialize the room hygena i Anand hey everyone it's great to be here on stage holographically it's great to have you here Jenna can you tell everybody a little bit about what you're seeing sure I can see your lifelike avatar which we generate in seconds from a 2d photo and we can walk around the stage with spatial audio and use this whole space around us to collaborate as if we're all here together in the same room cool now to show you how companies are using spatial to transform the way they work let's invite spend Gurjit CTO of the iconic toy brand Mattel onto the stage hey guys how awesome is this so at Mattel we're undergoing a massive digital transformation it's touching all aspects of our business this includes the way that we're using technology to design and to develop our products our classic brands like Barbie and Hot Wheels and fisher-price they have diverse teams of designers engineers marketers and manufacturers that are spread all over the world they can now come together in a spacial project room reducing the need to travel as as much to get everybody on the same page ok guys let's come up with some ideas for a line of aquatic toys yeah how about sea turtles oh that's really cool let's try it sharks that cool how about jellyfish so all we have to do is say the words and they're instantly visualized right before our eyes you can even click into one of these bundles and they expand it to a full-blown internet search complete with 3d models images and webpages you know why don't we click on adjust the images here so we can get some good inspiration for this new aquatic line mm-hmm sure every object you see in spatial is physical and tectile so you can scroll through or pick up images you like and toss them up on the wall with physics and we don't have to just stick with digital content I can actually pull up these sketches I did on my phone last night using that same spatial app I just pull up those photos and hit Send and they're instantly transformed into this digital environment knife drawing it's so easy to fill up your roommate ideas so we built this tool to help you quickly organize all these ideas so let me select all of these and let's make it read if goes to the wall now this entire room we've created is a persistent digital object that anyone can access or contribute to at anytime whether they're using an AR or VR headset or even a PC or mobile phone so that's right now we can have virtually rich visual rooms that we can keep up for the life of the product that means no more tearing down war rooms all the time so spatial and hololens are helping us drive improvements in our digital design and development process changing the way we create new toys by bringing people together from all over the world to collaborate in the same virtual room we're overcoming a natural barrier to our collective success that's people's desire for direct face-to-face interaction when building commitment and Trust we're so excited to see faster time-to-market and reduce need to travel as well as the many other benefits that we're gonna unleash at Mattel as we collaborate in mixed reality thank you thanks Alex now keeping with the theme of time to value we have seen hololens get used in a variety of different environments environments ranging from construction sites to operating rooms to even the International Space Station I see lots of different training experiences here there's the blower one Maria was talking about let me go ahead and select it now I need to place it on stage let's pick center stage and place the big flower Wow wow it's beautiful so alex is now viewing the experience that maria published from v4u studio which includes live iot data streaming in from an actual blower out in the field while this experience offers significant benefits from a time and cost of training perspective thus far it's only been geared for individual participation but now let's make it more collaborative by leveraging the azure spatial anchor service within view forea studio and veal we can let Maria seamlessly join in Alex's experience Maria can simply take out her iPad and launch view forea veal once Maria points her iPad at the same location where Alex placed the blower she can participate in the mixed reality experience making it far more interactive and collaborative that's right Jim now that Alex and I can share the same experience I can create a number of different training scenarios Alex let's simulate a gear failure mode oh I see there your vibration went down and I have your threshold exceeded please inspect let's go ahead and select that Alex I think you might be dealing with a broken gear tooth and you will probably have to replace the gear assembly all right overlap my sleeves and get some work done here let's do it I'm ready right I'm gonna start the disassembly instructions for the gear assembly there it is I see it now that alex is using Holland's - he has access to a wide range of hand and eye tracking capabilities they're all available and before your studio and view all right let's put it back that is so awesome euphoria studio allowed Maria in Hayden to create a single mixed reality experience without worrying about which platform it would be viewed on\n"