Using speech recognition, haptic control and AR to enable human-cobot collaboration in manufacturing assembly

My name is Patrik Gustavsson and I’m a PhD student within IPSI. My research tries to combine three technologies, speech recognition, haptic control, and augmented reality, to enable a more natural collaboration with an industrial robot in an assembly manufacturing. I started my research in early 2015 and plan on being finished early 2020. This work is funded by University of Skövde, Volvo Cars Corporation and the Knowledge Foundation.

Patrik Gustavsson HRI IPSI

Current collaborative robots are mostly used to work in close proximity with the human in an industrial setting, but in order to fully be able to collaborate with the robot they need to assist each other. Therefore, a new programming-by-guidance system will be created in this research as one step on the way to fully collaborate with industrial robots.

Teaching the robot how to execute its task

The system thath I will develop will make it possible for humans to assist an industrial robot by teaching the robot how to execute its task. This project will combine three technologies, speech recognition, haptic control, and augmented reality to achieve an effective and intuitive programming-by-guidance system that can be used within the assembly manufacturing industry.

The project have three main motivators:

  • Allowing workers, with no robot programming expertise, to teach the robot how to execute its task in an assembly manufacturing system.
  • Reducing the development time of the robot by introducing advanced programming-by-guidance technology.
  • Showing that augmented reality can add additional information that is useful when programming the robot such as: Coordinate systems of work objects and tools, path of the robot for collision detection, robots possible movements when moving the robot by hand.

New type of programming-by-guidance 

This project aims to introduce a new type of programming-by-guidance of an industrial robot into an assembly manufacturing setting. This will be realized by researching the three technologies to find state-of-the-art solutions that will function in an assembly manufacturing setting for human-robot collaboration. These technologies will then be combined into a complete artifact to enable an intuitive programming-by-guidance system, which then can be applied on a use-case within assembly manufacturing.


Contact information

Patrik Gustavsson
Ph.D student
School of Engineering Science
Work: +46 (0)500-448507

Supervisors and Mentors

People directly involved in my research are:

  • Principal Supervisor – Professor Lihui Wang, University of Skövde, KTH
  • Primary Supervisor – Associate Professor. Anna Syberfeldt, University of Skövde
  • Industrial Mentors – Rodney Lindgren Brewster (Volvo Cars Corporation)
  • Board of directors of IPSI