Augmented Reality in Enterprise Maintenance: Is it Worth the Hype?

Augmented reality makes it possible for maintenance workers to view accurate, context-aware instructions over cyber-representations of machines and equipment rather than reading paper manuals, which boosts safety and productivity.

Moreover, the use of smart glasses leaves the hands of the worker free to perform maintenance and repair tasks while viewing the instructions. These benefits of AR and smart wearable devices aren’t new – they’ve been around for many years, yet it’s only recently that we’ve seen such maintenance modalities in the field.

In an effort to understand the benefits and the hype behind AR in maintenance, we interviewed Dr. Markus Große Böckmann, CEO of Oculavis GmbH, a company specialized in AR solutions for enterprise maintenance and field service engineering.

Markus was kind enough to shed light on the use of AR in enterprise maintenance, while sharing with us insights and best practices for to optimize AR deployments. Finally, we take a look at what lies ahead regarding AR deployments in the years to come.  
‍

Q: Could You Briefly Describe The Main Components of an AR Solution?

A: An AR solution consists of mobile hardware like a smartphone, tablet or smart glasses and the corresponding software with the application. However, in the case of a real AR solution, algorithms from commercial providers like Wikitude or open-source algorithms like OpenCV must be also used to track objects or elements.
‍

Q: AR Has Been Around For a While - Why Is There So Much Buzz About it Lately?

A: AR projects have been around since the '60s at the Massachusetts Institute of Technology (MIT). In Germany, we started to hear the hype back in the ‘90s. The problem was that the hardware wasn’t ready for it at that time. People were wearing heavy desktop PCs as backpacks. High-performance low-weight devices didn’t exist yet. So AR at the time was not convenient or practical.

The mass production and miniaturization of smart phones in the past 10 years allowed for a new class of wearable devices: smart glasses. While several companies were already working on smart glasses, it was Google in 2012 who really opened the market.

Google already had a lot of influence and when they officially announced the glasses, a lot of other enterprises followed suit. Now, six years later, we have a competitive industry creating some really intriguing products. And the second  generation of glasses are much more useful tools that enable productivity. So, there is nothing “new” about AR as a concept, but the fact is that technology acceleration is now enabling AR deployment in production tasks.

It is also important to understand that "real" AR is about exploiting positional information.

In particular, “real” AR involves overlaying virtual information at a specific position over an object or in the room. To this end, the objects must be tracked by the AR device’s camera or sensors and information like descriptions or animations are added to the scene.

However, nowadays, nearly all applications connected with smart glasses are called “AR-applications”, even though most of them are simply displaying pictures, texts or video on glasses, without exploiting positional information as in “real” AR. AR is clearly a marketing buzzword, but it’s important to distinguish between real AR and simple smart glasses applications.
‍

Q: Does AR Really Provide Value in Enterprise Maintenance?

A: Simple image, text and video display applications can be very useful for setting up machine set-ups, providing maintenance instructions and training maintenance employees. Depending on their needs, employees can control the devices hands-free via speech recognition; they can be walked through step-by-step instructions.

Besides instructions, probably the most valuable application of AR is remote service or remote support. The key elements of a remote support application include a high definition video stream,  high-quality audio (especially in loud environments) and the first-person perspective of smart glasses with tools to collaborate and document the situation.

Even when they’re not using “real AR” technology with object tracking, this application is highly valueable in industrial maintenance settings. It can help to greatly reduce travel costs of service technicians and the downtimes of machines by having a new and intuitive way of interacting with equipment.

However, there are also cases where "real" AR can be hugely valuable, because of its ability to identify objects or combine information with objects. Specifically, it provides views inside parts or products in the form of ‘digital x-ray’ of the machinery or equipment.

This makes it far easier for novices to understand technical drawings and products or even to find failures at a machine. Visualizing a machine in AR is convenient for training and showing how the mechanisms of products work in actuality, rather than merely explaining it.
‍

Q: Could You Describe Some Tangible Use Cases of AR In Enterprise Maintenance?

A: We see a strong trend to implement remote support solutions in various application fields of the mechanical engineering sector and their customer sectors like automotive or energy. Everywhere we see complex products and engineering knowledge is necessary, remote support processes make sense to avoid travels and downtimes. Some of the most prominent use cases are:

1. Remote operator support: The operator has a problem or specific question at a machine. He/she can be hands-free and use smart glasses to get support from an external expert. The latter can support the operator to find a failure for a problem or instruct him/her on how to use the machine.
2. Remote quality pre-inspection or remote audits: When auditing a process or making a pre-inspection of a complex technical product, experts are travelling. As a remote procedure this can be changed and audits can be done remotely at hand of checklists.
3. Remote maintenance support of field technicians: Today, novice technicians cannot be sent out alone to the field, because they do not yet possess the required knowledge. Transforming this procedure to a remote process allows keeping the expert at home and supporting the field technicians on the road.
4. Remote outgoing goods inspections: When suppliers have manufacturing issues, the outgoing goods inspection can be replaced by a remote inspection of the customer at least for a certain time till the problems are solved. In this way, transport processes of defect goods are avoided.
   ‍

Q: What Is the Optimal AR Visualization Device Today?

A: There are pros and cons regarding the use of the various devices. The advantages of using smart glasses is that they are hands-free, enable first-person view of the camera and are an excellent choice when there is a need to receive information in field of view.

On the downside, smart glasses are still a young technology and relevant hardware standards are still in their infancy. Moreover, user can’t input text, they have generally a small field of view, and they have a higher learning curve than your average smartphone.

A final problem that the smart glasses industry hasn’t yet tackled is that of regular glasses: people who wear glasses or spectacles are likely to encounter problems trying to wear smart glasses on top of their regular glasses. So the industry will need to resolve these issues.

On the other hand, smartphones are mainstream and widely available with very high adoption rates within users’ (including maintenance workers). But they do not offer hands-free operation, which limits the interaction flexibility for the workers that use them.

Overall, which device an organization should use is dependent on the application and the business case.

Smartphones are already available widely and nearly every person has one and knows how to use it. Smart glasses are very good in cases where people need their hands to work. In addition, the first-person view of the camera is very helpful for another person to see what the user sees.

As an example, with smart glasses, workers are able to follow instructions and act faster when repairing machinery in a factory or a plant, since they are able to use their hands and see the instructions at the same time.

This cannot happen when they have to hold their smart phone. However, smart phones are ubiquitous and typically cheaper than smart glasses, which is a reason why some companies opt to sacrifice speed for the sake of cost.
‍

Q: What Is Hindering Widespread AR Deployment in Enterprise Maintenance?

A: There are indeed some barriers that companies need to overcome as part of their AR endeavors. The first one relates to the safety and security of the information. Service technicians are travelling around and need information on mobile devices, which are provided via cloud applications.

Besides order information, there are many cases where sensitive information (e.g., sensitive production data or personal data) must be accessed, which raises security and safety concerns.

The good news is that most organizations today have procedures for analyzing data storage and accessing risks associated with new applications. Hence, data security is no longer a show stopper, but rather a factor that can delay deployment and roll-out of an AR solution.

A second barrier to overcome concerns the use of cameras on shop floors. The use of cameras is forbidden in some shop floor environments (e.g., such as military or automotive shop floors). In such environments, there is a need to specify and apply appropriate organizational measures, which makes AR deployment more complicated.

A third challenge I see relates to the configuration of firewalls and other security devices. Remote AR procedures require network traffic to pass through firewalls, which are likely to restrict traffic and/or block connections. Thus, AR deployments need to consider the need for changes to firewall configurations.

Finally, low internet speed in rural areas or in buildings can also hinder AR deployments. In particular, when there is no cloud access for applications or for remote service videocalls, remote processes are not possible.
‍

Q: Could You Provide Some Recommendations for Plant Owners or Operators Who Are Considering AR Adoption?

A: I recommend that the following requirements and activities be taken into account:

• The workers’ council should be well-informed and engaged in the project implementation process. AR provides a lot of benefits for workers, such as increased safety and productivity at lower effort. It’s important that these benefits are properly communicated to the council and that reluctance and politics barriers do not affect the project.
• The project team must work closely with the IT department to clarify integration of devices and details of cloud-based solutions.
• Let the users choose their preferred device, as soon as this is in-line with project budget. Keeping the users’ happy and engaged is a key to successful deployment and operation of AR solutions.
  ‍

Q: What Improvements Do You Expect To See in AR Technology in the Next 5-10 Years?

A: In the next 5-10 years, we will see improvement of smart glasses in terms of weight, wearing comfort and performance. In the industrial environment, glasses will become more robust. This will allow a widespread adoption of AR applications.

I believe that in 10 years over 95% of companies with a complex product like a machine tool or complex manufacturing procedures will use smart glasses and AR maintenance solutions.

In the software area, new tools will be created to easily model 3D AR instructions, in way that is as easy as we use office applications nowadays. Tracking algorithms will allow to really detect objects from CAD models, which is currently missing. Also, the performance and accuracy of tracking algorithms will be also improved step-by-step.

Overall, I would say that AR applications will find their way into industrial maintenance without any doubt.
‍

Augmented Reality: On the Horizon To Streamline Maintenance

There you have it. While AR is a much-hyped technology, it also shows a lot of real promise. If properly deployed, organizations should be able to save time and money, and increase productivity, efficiency, and wrench-time. AR has the added benefit of engaging employees – everyone, after all, wants to feel equipped to excel in their work. Organizations stand to gain more and more as the technology continues to evolve.