Augmented reality visualization and edition of cognitive workflow capturing

Abstract

The aim of the COGNITO project is to design a personal assistance system, in which Augmented Reality (AR) is used to support users in task solving and manipulation of objects. Due to its sensing and learning capability, the COGNITO system automatically creates workflow references by observing a shown task in learning mode. After the workflow has been learnt, the system can be run in playback mode, in which it explains the previously learnt task to the operator. The system compares the user activity in real-time with the workflow reference and provides adequate feedback. This system is composed by four main modules. This paper focuses on the last module – the 3D graphics engine – which is the basis to the development of both the augmented and the virtual reality player. Additionally, it also presents the template of actions editor which is an editing tool that enables non-programmers and non-3D-experts to prepare and accompany the composition of visualizations for end-users.