University of Wales Trinity Saint David (UWTSD) and MADE Cymru’s ESF funded projects partnered up with Avanade on an applied industry innovation project that uses Microsoft technologies to uncover immersive solutions for manufacturers of the future. The ambition is to help manufacturers (large and small) to explore further opportunities to rethink new ways of working, apply more resilient operations and deliver agile innovative solutions for the factories of the future.
Avanade is the leading provider of innovative digital and cloud services, business solutions and design-led experiences on the Microsoft ecosystem. With 39,000 professionals in 25 countries, they are the power behind the Accenture Microsoft Business Group, helping companies to engage customers, empower employees, optimise operations and transform products, leveraging the Microsoft platform. Majority owned by Accenture, Avanade was founded in 2000 by Accenture LLP and Microsoft Corporation.
The aim of the project was to utilise the HoloLens 2 mixed reality capabilities, to innovate advanced on-site troubleshooting capabilities by developing immersive interactive guides. MADE Cymru researched appropriate equipment to test the HoloLens 2 functions and, after exploring different options, based the project around a MAZAK Machining Centre and a Universal Robot based in UWTSD.
Once the equipment was identified, the team started to compile as much information as possible to help Avanade begin the development stages of the HoloLens guides. Information consisted of official user manuals, photographic references, CAD models, 3D virtual tour (allowed Avanade to explore and understand the work environment – over distance due to pandemic), as well as process reference videos.
During this process there were various discussions on how to break down procedures and processes, along with several draft versions of how the guides could be constructed. As the guides developed, they then started to work on what other content would be most suitable to accompany the main content. At this point in the process, they began reconfiguring and editing CAD models to assist with specific interactive aspects of the guides, as well as editing video and photo content for supporting reference material.
As content began to be inserted into the guides, hands on testing began on the HoloLens 2, where the usability and interactivity were tested to gain a sense of the functionality of the guides along with the immersion of the material. At this point, MADE Cymru were observing the live tests via remote viewing, a useful function of the HoloLens device, where you can view what the user sees on a monitor, including the augmented guides, in the user’s environment.
As the guides were progressing, a key challenge was finding the balance between the guides being immersive and visually impressive but also contain meaningful content. The end result is a balance of all three, allowing the user to focus on following the guides for the procedures, however the reference material and models were important supporting features to help illustrate the current step.
Once all parties were happy with the finalised guides, the information and files were then transferred to MADE Cymru, viewed and ready to use on the two HoloLens 2 pieces of kit that MADE Cymru had. MADE Cymru is now beginning the expansion of the initiative, by gathering internal and external projects to further develop and innovate useful ways of integrating mixed reality into the workplace and educational environments.
Benefits to manufacturers
Workers can access a database of guides, troubleshooting documentation to help quickly solve any issues they have on the factory floor.
Further assistance can be called in via remote assist viewing over video call, allowing workers to have direct communication with the manufacturer in case of specific bespoke issues they may have with machinery.
Managers can use data on the guides to gain understanding whether specific machinery is having issues and, if so, what issue by relating to the guides that are being used.
A larger catalogue of guides could be developed in order to hold interactive inductions for new workers. This could include new workers using the HoloLens equipment themselves and having the immersive experience of interacting and navigating guides in front of new machinery. Alternatively, experienced workers could give a guided tour of factory floor using remote viewing, accessing various guides/programs which display supporting information, videos, CAD files etc., allowing new workers to view remotely, saving time and money for companies.
The immersive experience could lead to other avenues such as interactive virtual tours and interactive machinery galleries.
Health and safety training could be enhanced, with interactive guides explaining various warnings and exit routes in case of emergencies.
Benefits to education
Students can run through hardware inductions in a safer environment, interacting with holographic versions of the hardware, before getting hands on with the real hardware.
In larger cohorts, this equipment can almost allow students to access small troubleshooting guides, where they are unfamiliar with processes that will become second nature, however they have quick and easy access to guides to take them through step-by-step.
As the HoloLens equipment is a headset, this allows the students to learn hands free, meaning they can still be aware of their surroundings without being concerned of extra equipment that might be in the way (e.g. iPad, laptop to view guides).
The ability to remote learn, a lecturer could be using the HoloLens along with visual guides and material within a workshop lesson, however students from anywhere across Wales (and indeed the world) can observe and join the lesson.
David Morgan Avanade Global Manufacturing Marketing said, “Immersive mixed reality solutions are one of many Industry 4.0 solutions driving transformation in the manufacturing sector. Avanade is delighted to support the UWTSD and MADE Cymru ambition of enhancing digital skills for the manufacturing workplace and workforce of the future.”
Graham Howe, MADE Cymru said, “Never has it been more important for manufacturers to look at ways of embracing Industry 4.0 technologies such as HoloLens 2 to advance their processes, systems and productivity. Covid has accelerated the need to consider and understand which of the new technologies will give the most economic benefits. Working with Avanade on the HoloLens initiative is super-boosting what we can offer manufacturers – and even educators – a great example of industry working with a university to produce something spectacular that makes a tangible difference to an organisation.”
MADE Cymru is working on several projects where the aim is to convert existing factories into smart factories through automation, additive-manufacturing, advanced designing techniques and through employment of cutting-edge research.
The team aims to support businesses so that manufacturability can be brought in-house or within Wales and join together with the existing chain of commands of the company. This improves and enhances current processes and products through the logistics of Industry 4.0; this not only has positive economic benefits, but also educates industries about niche technologies to future-proof for the future.