To say that Industry 4.0 will reshape manufacturing across Europe—and globally—would be an understatement. This Fourth Industrial Revolution is driving a more digital, data-driven future for manufacturers, with robotics and automation already transforming production lines, streamlining industrial processes, and creating new skills and opportunities for workforces.

But as more and more robotics boost productivity and precision, workers and engineers must quickly get to grips with programming and operating these systems, often within complex and highly automated production environments—and this isn’t always easy. Workforce access to industrial robots can be limited due to cost, safety concerns, and a need to avoid disruption on the factory floor. But this is where initiatives such as the European Union Horizon project MASTER—Mixed Reality Ecosystem for Teaching Robotics in Manufacturing—are making an impact.

Launched in January 2023 to support industrial robotics education and training, MASTER has developed an open, extended reality (XR) platform that exploits the latest virtual, augmented, and mixed reality technologies to provide immersive and interactive learning experiences for trainees. Organizations can build and run their own XR training and simulation environments in which several remote users can interact in real-time within the same scene, practicing tasks and experimenting with robotic systems in safe and controlled conditions.

Master project coordinator Panagiotis Karagiannis

For Panagiotis Karagiannis, MASTER project coordinator and project manager at the Laboratory for Manufacturing Systems and Automation (LMS), University of Patras, Greece, XR lies at the heart of Industry 4.0, and is set to, as he put it, “create a new paradigm” in training. “A lot of people can be afraid of new robotics—they can unexpectedly make quick movements and sudden sounds, which can be intimidating,” he said. “But working inside a digital environment with XR brings you closer to these systems without these problems—and we’ve created a training environment that feels like a game.”

As the project draws to a close, the MASTER Open XR platform addresses the “fear of new technology” issue and more. Training factory workers through its immersive environment can also reduce a firm’s costs and circumvent the inevitable safety issues. “The platform is an affordable educational tool for factories,” Karagiannis highlighted. “You don’t need to develop the robots, you don’t need to set up the production line, you can create all of this virtually, which of course is cheaper.”

“Also, in a manufacturing setting, numerous systems are operational, and safety can be very complex, bringing so many restrictions,” he added. “But by creating a digital environment, we avoid all of this.”

Training for the masses

MASTER’s Open XR training platform is based on an enterprise XR platform, VIROO, developed by Spanish VR tech firm Virtualware. Karagiannis and project colleagues have added three extra features to the platform to ease training: code-free robot programming, gaze-based interaction, and safe robotic environments, which mimic the factory floor by using virtual zones equipped with sensors to trigger a speed reduction or protective stop when a human enters the defined space.

In addition to these features, EU-funded XR researchers have contributed new XR tools and educational content, via add-ons, to enhance the platform, which have all been tested via user studies. Examples include a voice-prompted tool for spawning, moving, and deleting objects such as robotic arms and forklifts, a photo-realistic digital twin of a cargo ship for maritime safety training, and a haptic glove so users can practice teleoperating a humanoid gripper in VR.

User feedback, so far, has been positive with participants recommending applications to others, and reporting to be either “satisfied” or “very satisfied” with the platform’s ease of use. Content creators noted they would use the platform again for similar tasks, while trainees stated they enjoyed the graphics and interaction with 3D objects, and highlighted improvements in specific skills, particularly in understanding lecture content.

Karagiannis also noted how the development and validation of these tools fostered significant collaboration, especially amongst SMEs. As a case in point, he highlighted a software development kit, “Dreamer,” that reconstructs 3D objects to be embedded in virtual scenes for training. Here, a self-supervised AI learning agent trained on streams of images rapidly extracts a 3D mesh representation of a physical object, so users can create content for populating simulated scenes. He then pointed out how, in a later sub-project, Xrplained, researchers used assets created in Dreamer to explore robot kinematics and better understand the complex principles behind robotic positioning, orientation, and motion.

“We are proud of how we brought together different SMEs to develop, use, and customize different platform assets and exchange knowledge across different applications,” he said.

But MASTER collaboration hasn’t just been about SMEs. Add-on developers have included researchers from larger firms, including Nokia Bell Labs. “All in all, the project has developed more than 30 assets—including 3D models, safety zones, as well as beautiful user interfaces and a virtual assistant—delivering a big impact,” Karagiannis commented. “We can really showcase to industry that through collaboration, we can create educational scenarios that deliver good results and provide a supportive environment for the training operators and engineers.”

A user tests an advanced virtual reality headset and motion-tracking controllers, which feature exposed LED tracking rings for precise spatial positioning. (Source: European Union Horizon)

Next moves

As well as being based on Viroo’s enterprise-ready platform, MASTER Open XR platform is compatible with Open XR and ROS, an open-source framework for robot software development. This interoperability enables integration into existing industrial environments, allowing the platform to be more easily adopted by organizations across Europe and beyond, where it can be further developed and scaled for myriad manufacturing applications.

Looking ahead, the focus will shift from platform development to ecosystem expansion and real-world deployment. Karagiannis would like to see project groups within organizations developing even more virtual assets and scenarios that will grow the platform even further. “We could see the platform move beyond education—additional XR assets could also be used for, say, technical support and maintenance,” he said.

Further into the future, Karagiannis firmly believes that AI could be integrated across the platform to generate yet more XR assets and add-ons. “XR is one of Industry 4.0’s core technologies because it deeply involves humans in industrial processes and, as we reach Industry 5.0, with its even more human-centric approach, we will see this to an even greater extent,” he said. “There is so much potential for our platform here.”