Skip to main content

Assistance systems supported by artificial intelligence foster vocational rehabilitation

Empower frontline workers
Insights overview

KI.ASSIST (“AI.ASSIST”) research project successfully evaluates assistance systems supported by Artificial Intelligence (AI) to increase the occupational participation of people with various disabilities. [By Lisa Gruber]

The purpose of the research project KI.ASSIST, which was launched in 2019, was to increase or even make possible the occupational participation of people with disabilities by using AI-supported assistance systems. The task was to identify which technologies are suitable for specific limitations and which framework conditions they require to promote the vocational independence of the rehabilitees.

Objectives of KI.ASSIST

  • The elimination of barriers in the educational and workplace environments of people with disabilities
  • To enable the rehabilitees to learn and work independently and self-determined
  • Reinforcing digital learning methods in the training and induction curriculum
  • Person-independent transfer of expertise and work routines
  • Efficient preparation of rehabilitees for the first labour market
  • Creating recommendations for action for modern vocational rehabilitation supported by AI technologies
  • Establishment of an information and exchange platform with different participation formats

Project overview

The goal of the funded research project KI.ASSIST was to increase the participation of people with a wide range of disabilities in vocational rehabilitation. Several vocational promotion centres (BFW), vocational training centres (BBW) and workshops for people with disabilities, as well as their umbrella organizations, participated in the research project KI.ASSIST (2019-2022), which was funded by the German Federal Ministry of Labour and Social Affairs (BMAS). The research partner was the German Research Centre for Artificial Intelligence (DFKI). In so-called learning and experimentation spaces, one of the questions investigated was whether AI-supported assistance technologies can support the acquisition of skills as well as the learning of processes in a wide range of professions. The project also evaluated the extent to which these technologies can be used outside vocational rehabilitation facilities to promote the participation of people with disabilities in working life.

TeamViewer contributed its Augmented Reality (AR) system Frontline to projects that focused on trainees in IT and technical professions, as well as on nursing professions inside and outside the facility.

AR shows off its strengths

In industry, clear step-by-step instructions have been improving business processes in such areas as logistics and maintenance over many years. In particular, the AR-solution TeamViewer Frontline xMake, which focuses on manual manufacturing processes, was used in the learning and experimentation rooms in Munich and Pirna (Germany).

xMake operates on data glasses, wearables (i.e. mini-computers that can be worn on the body) and mobile devices. In the KI.ASSIST projects, the choice fell on smart glasses models from the TeamViewer partners Vuzix and RealWear. These devices, already established in industrial contexts, are worn on the head and have a small screen located directly in the user’s field of vision. In addition, the devices are equipped with a camera, microphone and speaker.

xMake assists wearers of data glasses with pre-programmed step-by-step instructions (workflows). Depending on the task and information to be conveyed, videos, images or texts can be projected onto the screen of the data glasses. A spoken instruction is played back via the speaker of the data glasses. Users can control the flow of the instructions individually by voice command. This means they can pause the tutorial to perform the action that was just described. In addition, they can repeat individual steps, comment on them, or even cancel them. If work results are to be validated, the workflow can be set up in such a way that individual work steps must be confirmed before the instructions can be continued. Speech recognition and control of the data glasses are based on an AI model pre-trained by TeamViewer for speech and image recognition.

Create your own AR applications – without any programming knowledge at all
  • Using the Creator included in Frontline, the step-by-step instructions can be easily prepared and customized with a few clicks in the web browser without any programming knowledge
  • Thanks to the simple and flexible customization, the required level of detail of the instructions can be developed and tested together with the users in practice
  • Inserting and editing photos, sound and video recordings is easy with Frontline Creator and does not require any special user knowledge

Where innovative technology creates advantages

In all learning and experimentation rooms, the use of data glasses was found to be beneficial for many target groups, especially for routine activities with the same procedure over and over again. Only in the case of people with severe physical or cognitive disabilities did the use of data glasses prove rather difficult because of the technical handling. Speech recognition did not always work properly in case of speech problems.

Since some AI-supported assistance systems have to be prepared for each task with a precise fit, they cannot necessarily be used spontaneously. Skills for the various AI assistance systems, such as data glasses, biofeedback tracking, navigation system, voice control, need to be built through training and consultation.

Reliably build expertise for IT and electronics professions: BFW Munich

At BFW Munich (Vocational Promotion Centre Munich), TeamViewer Frontline was used by over 40 trainees to build up expertise and to learn and practice work processes. The use of AR technology has proven particularly promising for people with learning disabilities or those with mental and physical impairments.

First, trainees working in IT and electronics tested the step-by-step instructions via data glasses. Positive side effect for these trainees: They are working their way into next-gen technologies. The rehabilitation patients confirmed that the speech recognition worked well. All found the work and learning support helpful. Only the unfamiliar handling of the new high-tech devices initially led to habituation effects such as dizziness for some users.

Testing the technology, especially in low-threshold work such as the installation of network cables, should provide initial empirical values for further application scenarios. Podiatry and the commissioning and control of electronic circuits are envisaged. In other areas of application, for example in the kitchen, the data glasses were perceived as rather annoying and were not considered providing helpful support.

Communicate processes in nursing in a simple and structured way: AWO Workshops Pirna

The AWO (Workers Welfare Institution) Workshops Pirna for people with disabilities also tested TeamViewer Frontline, to teach 33 of their 550 employees work processes in the care sector in a structured way. Some rehabilitees with cognitive disabilities and learning disabilities also work in jobs in the primary labour market, such as in residential homes and in assembly. Different step-by-step instructions (workflows) were created for their tasks and tested with two different pairs of data glasses. It was important to bear in mind that data protection guidelines limited the use of data glasses with cameras at certain public locations outside workshops.

Advantages of using AI-supported assistance systems

  • The technologies are suitable for use in a very broad target group of people with a wide range of disabilities
  • Since the data glasses are suitable for many applications wirelessly and without an Internet connection, they can be used largely independent of location
  • The degree of detailed workflow support can be designed exactly to the task and target group
  • Rehabilitation students learn and work according to their individual pace by repeatedly looking at individual steps of the work instructions, interrupting the instructions and practising individual steps
  • Augmented Reality solutions offer a hands-on alternative to learning on a PC that also leaves both hands free for parallel work

Summary: the technology was received very positively

Within the framework of the project, the experts agree that the supportive function of the technology in the workplace has already reached a high level of maturity and is suitable for a broad target group. Frontline can be used flexibly for a wide range of applications and has demonstrated direct added value in productive use. For many activities, this means that there is no need for accompanying personnel, which frees up capacity and reduces the cost of guidance and support. Overall, the technology was received very positively during the project, so that individual institutions have decided to continue it independently – for example BFW Munich – and it remains to be seen how many will follow and trust the advantages of AR in vocational rehabilitation.