Setbacks of the development of a concept inventory for Scrum: contributions from item response theory

Abstract

Scrum is the more common framework for agile project management. Agile project management requires frequent feedbacks and delivered items in projects with dynamic requirements and changes. Training learners in Scrum permits building agility in solving problems and teamwork competencies. Measuring training effectiveness is essential to identify students' learning lacks or misconceptions to improve the training outcomes. To assess the development of competences, it is possible to use concept Inventories, which are an essential educational tool to observe students' learning gain between two moments, before and after training. Additionally, the Item Response Theory may be applied to concept inventory items to identify latent characteristics as guessing, difficulty, and discriminant values. Guessing is related to an arbitrary answer to one question and gets the correct answer with common learner knowledge. Difficulty characteristic is related to student knowledge level to one question. Discriminant characteristic considers that learners with high score get accurate answers to the questions. Thus, this work aims to present some of the main setbacks of developing a concept inventory for Scrum, supported by the Item Response Theory. In this way, other researchers may understand how to develop a concept inventory and some of the main obstacles they may have to overcome or avoid. The Item Response Theory offers some indexes and criteria values to each latent characteristic to improve the concept inventory questions. Therefore, this work focuses on the process of conceptualizing, building, applying, and improving a Scrum Concept Inventory in a training situation with engineering students.