Background: Virtual Reality (VR) has emerged as a new technology that simulates real-world experience in an immersive virtual environment. The advances in computational power and the maturation of game engine technologies allow interaction with virtual objects in a way that was never possible before. Furthermore, availability of sensing technology (e.g., eye and motion tracking) facilitates the acquisition and analysis of eye and body movements in the VR environment. VR simulations are becoming an important part of AMSC systems. This is an exciting time as the technology can facilitate development of digital twins to monitor and control AMSC systems. VR can also improve learning outcomes by tapping into the five senses to improve students’ information retention. 

Research Objectives: The objective of this project is to develop a virtual factory that represents AMSC of car manufacturing. The virtual factory will be used to study the impact of automation on AMSC systems as well as be utilized for workforce training and student learning. Our team has developed the main platform of the virtual factory including the virtual building (Figure 1, left) and internal structure and equipment (Figure 1, right).

Figure 1: Sample snapshots from the VR car factory

Sample eye tracking data was collected and analyzed using state transition (Figure 2, right) and recurrence plots (Figure 2, right). The plots show how participant transitions from looking at one object to another in VR while performing a car assembly and welding.

Figure 2: State transition (left) and recurrence plot (right)

Research Plan: (1) extend the VR factory to include advanced manufacturing processes (e.g., 3D printing) and supply chain operations (order fulfillment), (2) develop a smart order management dashboard to track customer orders in the virtual factory, (3) study impact of automation on AMSC system, (4) collect and analyze eye and motion tracking data about user behavior and learning in virtual environment. 

REU Student Outcomes: This project fits a student with coding skills (e.g., computer science). REU student will gain hands-on experience in VR technology (hardware and software) as well as learn AMSC concepts. Student will be supervised by Dr. Faisal Aqlan and Dr. Li Yang (Industrial Engineering) and participate in writing conference and journal papers.

Required Skills: Knowledge of Python, Unreal Game Engine (C++),  Machine Learning, 3D Printing, Production Planning.