What is the scope of augmented reality

Augmented Reality: an asset to production

Most people immediately think of the intuitive usability of office software when they hear the term “user experience”. But technologies like Augmented Reality (AR) can also offer workers a better user experience in the shop floor.

In fact, software providers are increasingly concentrating on delivering increasingly modular and personalized applications that can be adapted to the needs of mental workers. But an appealing user experience is also important for other employees.

Technologies for augmented reality (AR) help workers in the factory to work more productively and precisely - whether in product assembly, machine repair or equipment service. AR tools, from non-immersive displays to immersive mixed reality applications, are conquering the work environment - the shop floor, the field service and more. And since these applications are being developed on open, modular platforms, it is quite possible that they will soon be integrated into core business systems.


The SAP has a new magazine called Horizons by SAP in which leading personalities from various companies in the technology sector around the world describe their vision of the future of IT. Brian Ballard, founder and CEO of Upskill, explains how technology can make augmented reality workers more productive.


To learn more about how AR can improve the user experience for the shop floor workforce, Horizons by SAP spoke to Brian Ballard, CEO of Upskill, the industry leader in AR software.

Thanks to gaming technologies, augmented (AR) and virtual reality (VR) inspire the imagination. But AR tools also have a practical role in the workplace, don't they?

Brian Ballard: For us, this is a world of wearables where AR with glasses and visual devices opens up a whole new field of vision. There is now a type of AR on smartphones and tablets too, whereby the camera sees the world and the user adds information. There are different tools for the different work areas, but augmented reality is definitely gaining importance in manufacturing.

On the way to the digital factory with augmented reality

Is that part of what is commonly referred to as the “digital factory”?

Exactly. AR is a tool for the workforce in the factory, in the warehouse and in the field. It complements the workers' tools with better access to information and real-time connections with the surrounding systems. AR will establish itself as a force multiplier for the workforce. The new type of information will revolutionize the workplace, and AR makes this information accessible to workers.

What new developments are there in augmented reality technology?

New technologies are emerging all the time, in both mixed reality and assisted reality devices. Microsoft recently launched the new HoloLens 2, mixed reality data glasses that are more than previous generations designed for businesses. And the new devices also offer more convenience. In some ways, greater comfort is even more important than some of the technical improvements, because greater comfort makes people use the devices more intensively.

But there are also many technical improvements.

That's right. In assisted reality, the batteries now last significantly longer, so that devices such as the Glass Enterprise Edition can be carried all day. Manufacturers have targeted corporate use cases with more robust devices, better cameras and more sophisticated language processing. With better sensors in the devices, we can certainly also integrate more complex server-side or cloud-based processing in order to further upgrade the entire system and make it more intelligent.

This really increases the reach of the technology.

Such devices can be found on drilling rigs, in warehouses and in factories. People wear them all day every day as part of their work equipment. The more companies integrate the devices into their work processes, the closer it is to a workforce without a desk. I don't think there will ever be a return to the old paper-based technology.

This is an investment that can be used to improve working conditions in production. Are the CIOs ready to invest in technology for the workers?

Companies must interact with their employees in the practical area in a software-based manner. That is not the norm yet. In most companies, brain workers receive support in the form of Excel spreadsheets, e-mail and word processing programs. But digitizing the other 50 percent of the workforce is not a priority in most cases. With AR, that changes. The CIOs and their colleagues such as Chief Digital Officers (CDOs) need to be aware of this and develop a plan for how together they can add value to this part of the company.

Individual production thanks to augmented reality

Manufacturing companies are increasingly making individual products. That means more product variants and more complexity. How can AR make workers more productive in these circumstances?

Leading manufacturing companies are already using AR to seamlessly guide workers through customized or changing work tasks. There is enormous potential here for increasing productivity with increasingly complex or customized products. In addition, new skills and abilities can be imparted to the workforce more quickly, and strategies for widely distributed, flexible production can also be implemented more easily. With AR, a tight digital thread of data can be stretched between design and manufacturing. A designer can change a product specification and with a click of the mouse pass the information directly to the employees in the shop floor. We have been talking about this for years, but until now there has always been a wall between systems and people. This is now finally torn.

It sounds like this connection shortens the feedback loop between design and production. Does AR promote collaboration in other ways too?

See what I see features allow experts to see things from the user's perspective. Imagine a designer changes something in the design and the operator discovers a problem. Since they both see the same thing, the designer can quickly make changes and resubmit the updated design. This feature is often used in the field so that a field worker can discuss a repair problem with an expert and get immediate help.

Does modularization change the design and use of AR technologies?

In any case. We are thinking here of building platforms to modularize individual work processes. With the platform approach and the modular components based on it, companies can use application modules without foregoing the security and processing functions of the platform.

Integration of AR in ERP and MES systems

Will more modularization - of both AR and corporate systems - help developers integrate more AR and VR into business processes, for example into ERP (Enterprise Resource Planning) or MES (Manufacturing Execution System)?

That is the obvious advancement of the technology. In the past ten years, productivity increases have been in the single-digit range. In processes with AR, on the other hand, improvements of 30, 50 and 100 percent can be observed across the board. I think that once the technology is widely adopted, productivity will skyrocket across the global economy.

What is currently preventing the integration of AR in ERP and MES systems? Where is there a need for improvement among software companies?

As far as assisted reality is concerned, integrating with these systems is pretty straightforward. Traditional data structures are easy to use. In two or three years, when more of the design and development process is developing a 3D component, we need to develop a pipeline between design and assembly on the shop floor. How can 3D models be broken down and presented in such a way that they can be used quickly in the periphery? We've teamed up with partners to find out how to break down workflows into an AR-native format. Very exciting things are happening there.

Impact of 5G networks

5G networks are casting their shadows ahead. How will this advance AR?

5G will make a huge difference, especially when the next generation of 5G-enabled devices finds its way into manufacturing. 5G will not only go hand in hand with bandwidth, but also a quality of service that will be crucial in factories that rely on the Internet of Things. It may take a few years to build the infrastructure - chips and transmitter masts - but I look forward to 5G with great excitement.

How should companies approach the topic of AR technologies in a modularized world?

There is no one-size-fits-all solution with AR. You have to use the right combination of technologies to solve the problems at hand. We always advise you to distinguish between operational technology (OT) and information technology (IT). OT is about identifying the relevant use cases. A certain degree of digital maturity must be achieved in IT in order to be able to use the technology on a large scale.

We are currently observing pilot implementations that take manufacturing problems into account - such as the creation of special cable connections in a data center. Is that an area where AR should be used?

We approach the subject from the opposite direction. Repetitive processes can be a great starting point for AR. Let's take the example of the data center. Hardly anyone can remember everything that is relevant for working on the individual cabinets in the data center. But the company likely has detailed records of the construction and maintenance of each individual server. Now, showing this information to employees can help prevent errors and confusion in the data center. Technology that takes the frustration out of workers and makes their jobs a little easier is a wonderful opportunity to increase goodwill.

Does that mean that increasing efficiency isn't the most important benefit of AR?

Productivity is important. But it tends to be more important to our customers that the quality is right from the start. Warranty claims against manufacturing companies can quickly reach tens of millions. If the risk of such claims can be reduced right from the start through higher manufacturing quality, that is a huge advantage.

How do companies measure AR profitability?

Let's take a warehouse as an example. By bringing information into the workforce's field of vision and no longer having to put tools down to read instructions, productivity can be increased by 10 to 20 percent. In manufacturing companies that manufacture capital goods, AR often increases productivity by 30 to 50 percent. That’s great! But if it is possible to avoid even a single quality problem, this saving alone outweighs the purchase price of the system.

What is your advice to companies that want to get started with modular AR technologies?

You shouldn't think of this as an experiment. Instead, they should focus on an area where real business benefit can be achieved. Sometimes decision-makers put a toe in the water but then don't jump in, which means they don't provide the right resources for the project. It can happen that the project does not start properly until the fifth, sixth or seventh attempt. In fact, companies tend to get greater benefits from large-scale AR projects than from smaller ones.

What's the better approach?

Organizations should identify a problem that needs to be resolved and that AR can help with. This should be a relevant problem that affects the workforce, production systems, IT and mobility. If companies know who to work with right from the start, they will achieve their goals much faster and often more cost-effectively than with a series of experiments that come to nothing.

Who should be on the team?

Sometimes those responsible from the operational area come up to us, such as production area managers. We ask them who they work with in the IT area. If you don't already have someone, we encourage you to contact the CIO team. If an IT team approaches us who would like to develop an interesting function, we ask who on the operational side will be using the technology. The two sides of the company will work more and more hand in hand in the future. So we have to make sure that both sides are represented.

This post was also published on LinkedIn.