Walk-Through MR World Approach for Students Using Mixed Reality

J. Inst. Eng. India Ser. B https://doi.org/10.1007/s40031-022-00799-3 REVIEW PAPER Walk‑Through MR World Approach for Students Using Mixed Reality Manasvi J. Maasthi1 · A. S. Prajwal Hebbar1 · H. L. Gururaj1 · V. Janhavi1 · K. Harshitha1 Received: 21 August 2021 / Accepted: 21 August 2022 © The Institution of Engineers (India) 2022 Abstract Knowledge is prodigious, and learning has no boundary. The curiosity of the one to learn, discover and invent decides the future of the world. Educational Institutions’ main objective is to provide qualitative knowledge to the students and supporting tools used for learning play a major role. The conventional education system is arduous to be used in the current situation of the global pandemic of COVID-19. New methods and tools are required to make learning and imparting knowledge more effective. Applications like Google Meet, Zoom, Cisco WebEx are being used in schools and colleges. When it comes to simulations in technical education, for instance, to develop any electrical circuits, robots, buildings, etc., software like MATLAB, 3Ds MAX, GNU Octave is being used. These methods neither are interactive nor provide an immersive experience to the user. To subdue this problem, Mixed Reality (MR) technology can be utilized as a boon by developing an application where students can have interactive classes, submerging themselves, and gaining the required knowledge. Also, the technical students can simulate their experiments onto the real world, providing an idea of how the world may look when new things are adopted and can undergo a walkthrough experience in the MR world. Keywords Mixed Reality · Education · MRTK · Immersive · Interactive * Manasvi J. Maasthi 1 Introduction In the current situation, everyone is petrified of the word COVID-19. This pandemic is directly or indirectly affecting each and every person across the globe. Overcoming this misfortune is strenuous until there is an inoculation. Inevitably, pandemic has become a part of life. Nevertheless, time and technology are directly proportional as technology grows in accordance with time. New methods like online classes are being conducted by schools and colleges so that students are not deprived of education. Similarly, the standard of living is changing day by day. Envisaging the world as the dream is fun to think of. What if this can be true! Fortunately, there is a technology called Mixed Reality, which helps a person to envision the virtual world created by them over the real world. By adapting this technology many applications can be built in different domains, for instance, healthcare, education, entertainment, gaming, etc. Utilizing this futuristic approach in the education system can benefit in numerous ways. The most important upshot of education is to gain knowledge, and there are many ways to implement interactive pedagogy. Virtual reality as well as augmented reality is already being used for educational purposes. In virtual reality, the projection will be simulated in a virtual world. In augmented reality, the projection will be limited to the screen and is not that interactive. To overcome this, a technology called Mixed Reality [1] is deployed, which can be considered as a union of virtual reality and augmented reality as shown in Fig. 1. This technology provides an interactive and immersive user interface (UI) and user experience, respectively. Henceforth, there can be a transfiguration in the education system. Department of Computer Science and Engineering, Vidyavardhaka College of Engineering, Mysuru, Karnataka 570002, India 13 Vol.:(0123456789) J. Inst. Eng. India Ser. B weight of the polygons is reduced in order to provide a costeffective photo-realistic image. Protection of Sensitive Information Fig. 1  Mixed Reality Literature Survey V‑Touch Immersive Experience Normally, the virtual objects are less interactive which in turn affects over a good immersive experience. To subdue this situation, vision tangible objects can be employed which will provide an excellent experience with its tactile feedback. This kind of interactivity with objects can be achieved using a touch interface which works without physical touch called V-Touch [2]. Telepresence with a Fine UI Number simulations of 3D generated avatars are used in telepresence [3] which basically contains 2 main approaches: Video-based reconstruction and 3D model rendering. In video-based reconstruction, first, the whole avatar is captured via camera and analysed. Next, the static and moving objects in the background are eliminated, offering a complete 3D avatar as a result. Then, the generated 3D avatar is subjected to 3D model rendering to overcome the visible lines in the polygon edges, hence providing a clear avatar. In order to provide a sublime user interface, it would be ideal to merge the factors of the virtual and physical world. This can be employed in Mixed Reality successively providing a better immersive experience [4]. Light‑weight Image Rendering Image rendering can also be interpreted as conversion of two-dimensional objects into three- dimensional objects. Two types of image rendering can be observed offline rendering and real-time rendering. In offline rendering, each and every frame of a video captured is converted from 2 to 3D, providing a high-quality photo-realistic image which on combining gives a high-resolution video that is expensive. Light-weight online rendering [5] can be used, where the 13 While providing an input, sensitive information can be unknowingly indulged to the device. In the same way, while using an application, the sensitive information can be extracted. To circumvent this, the method of input sanitization can be employed which essentially eliminates all the sensitive information from the input stream of data. While displaying, the device might want to check the surface information to decide the displayability of the surface. In that instant, sensitive information breach can emerge. To elude that, a technique called privacy-preserving rendering can be implemented which prevents the device from checking the information on the surface. Rather it just displays the output if in case the surface is found displayable [6]. Walk‑Through MR World Experience Mixed Reality can be considered as an eminent technology for the world today. Using this technology in education, tremendous changes can be observed like providing an interactive classroom experience to the students. To design a mixed reality project, tools such as Unity Editor and Mixed Reality Toolkit (MRTK) [1] can be used. Due to this COVID-19, there has been a shift in the education environment. Abode has become the new school; online classes have become a new trend. Applications like Zoom, Google Meet, Microsoft Teams, Discord, etc., are being used. These applications rank down when it comes to interactivity, in turn students end up losing focus toward education. As this pandemic may take a wh (...truncated)