ANUKARNA : a software engineering simulation game for teaching practical decision making in peer code review
Ritika Atal, Ashish Sureka · 2015
Application of educational and interactive simulation games to teach important concepts is an area that has attracted several Software Engineering researchers and educators attention. Previous research and studies on usage of simulation games in classroom to train students have demonstrated positive learning outcomes. Peer code review is a recommended best practice during software development which consists of systematically examining the source code of peers before releasing the software to Quality Assurance (QA). Practitioners and Researchers have proposed several best practices on various aspects of peer code review such as the size of code to be reviewed (in terms of lines of code), inspection rate and usage of checklists. We describe a single player educational simulation game to train students on best practices of peer code review. We define learning objectives, create a scenario in which the player plays the role of a project manager and a scoring system (as a factor of time, budget, quality and technical debt). We design a result screen showing the trace of events and reasoning (learning through success and failure as well as discovery) behind the points awarded to the player. We conduct a survey of the players by conducting a quiz before and after the game play and demonstrate the effectiveness of our approach. Keywords—Peer Code Review, Simulation Game, Software Engineering Education and Training, Teaching Critical Decision Making I. RESEARCH MOTIVATION AND AIM Software Engineering (SE) being a practice-oriented and applied field is taught primarily (at University Level) using instructor-driven classroom lectures as well as team-based projects requiring hands-on skills. In comparison to classroom lectures, team-based hands-on projects require more active participation and experiential learning. SE Educators have proposed and shown positive student learning outcomes by teaching certain concepts using simulation games. Some of the advantages of teaching using simulation games are incorporation of real-world dynamics such as critical decision making under multiple and conflicting goals, encountering unexpected and unanticipated events, allowing exploration of alternatives (discovery learning) and allowing incorporation of learning through doing and failure [1][2]. Peer code review consists of reviewing and critiquing team members source code in order to detect defects and improve the quality of the software [3][4][5][6][7]. Software code review is practiced in several open-source and closed-source software project settings. There are several best practices on various aspects of peer code review such as the code review size, coverage and rate. Code reviewer expertise, reviewer checklist and usage of tools (such as mailing lists, Gerrit or Rietveld) also play an important role in influencing the impact of code review on software quality [5][6]. The work presented in this paper is motivated by the need to teach the importance and best practices of peer code review to students using a simulation game. The research aim of the work presented in this paper is the following: 1) To develop a web-based interactive educational SE simulation game or environment for teaching benefits and best-practices of peer-code review process. 2) To investigate a learning framework and model based on discovery learning, learning from failure, evidence and reasoning for teaching concepts on the practice of peer code review. 3) To evaluate the proposed learning framework and tool by conducting experiments and collecting feedback from users. II. RELATED WORK & RESEARCH CONTRIBUTIONS In this Section, we discuss closely related work and state our novel research contributions in context to the related work. We conduct a literature survey of papers published on the topic of teaching Software Engineering concepts using simulation games. Table I shows the result of our literature review. Table I lists 9 papers in reverse chronological order and reveals that teaching Software Engineering concepts using simulation games is an area that has attracted several researchers attention from the year 2000 until 2013. We characterize 9 papers based on the tool name, year, simulation topic, University and interface. We infer that teaching software engineering processes and project management are the two most popular target areas for simulation games. The game interfaces various from simple command line and menu driven model to animated and 3D interfaces. Researchers have also experimented with board games in addition to computer-based games. In context to closely related work, the study presented in this paper makes the following novel contributions: 1) While there has been work done in the area of teaching Software Engineering processes and project management skills, our work is the first in the area 1st International Workshop on Case Method for Computing Education (CMCE 2015) 63 TABLE I. RELATED WORK (SORTED IN REVERSE CHRONOLOGICAL ORDER) AND TOOLS ON TEACHING SOFTWARE ENGINEERING CONCEPTS USING SIMULATION GAMES SNo. Tool Name Year Simulation Topic University Interface 1 AMEISE [8] 2013 [Bollin’13] Managing a SE project (focussing on software quality) Alps-Adriatic University, Carinthia Tech Inst. and Linz University Menu based, Command Interface 2 DELIVER [9] 2012 [Wangenheim’12] Earned Value Management Federal University of Santa Catarina Board Game 3 ProMaSi [10] 2011 [Petalidis’11] Project Management T.E.I. of Central Macedonia, Serres-Greece Java based, Desktop environment 4 SimVBSE [11] 2006 [Jain’06] Value-based Software Engineering University of Southern California Animated, 3D interface 5 Problems and Programmers [12] 2005 [Baker’05] Software Engineering Processes University of California, Irvine Card Game 6 SimjavaSP [13] 2005 [Shaw’05] Software Process Knowledge University of Tasmania Menu-based, Graphical Interface 7 Incredible Manager [14] 2004 [Dantas’04] Project Management Experiential Learning Brazilian University Button Driven, Command Interface 8 SimSE [1] 2004 [Navarro’04] Software Engineering Processes University of California, Irvine Web-based, Graphical Interface 9 SESAM [15] 2000 [Drappa’00] Software project Management Stuttgart University, Germany Text Based, Pseudo-Natural Language Command Interface of building and investigating simulation games for teaching best practices for peer code review. 2) We propose a simulation game to teach peer code review practices based on learning by failure, success and discovery. We demonstrate the effectiveness of our approach, discuss the strengths and limitations of our tool based on conducting user experiments and collecting their feedback. III. GAME ARCHITECTURE AND DESIGN A. Learning Objectives In this game we define 12 learning objectives covering multiple aspects of peer code review. These learning objectives are captured in our pre game questions1 and post game questions2. Table II shows 6 of the 12 learning objectives (due to space constraint), corresponding decisions and the questions which we asked to the player in game. Table II shows the structure that we follow to design the game questions. Each question is designed keeping in mind the learning objectives of the game. Based on the learning objectives we come up with a situation or a decision which best questions that objective. An evaluation question is then formed around this decision which challenges the player’s knowledge to its best. We therefore present different situations to the player where they have to make decisions like whom to assign the task of review process (Figure 1), what inspection rate to choose (Figure 2), when to start with review, steps required to foster good code review culture in team etc. A decision can map to one or more learning objectives. Similarly two or more evaluation questions may map to one decision. B. Unexpected Events We introduce unexpected events and unforeseen circumstances (such as internal conflicts between the team members, attrition and change in deadline or demand from the customer) in the game to make it more realistic. Unexpected events are unobservable and our goal is to examine the response and the decision making ability of the player to unexpected circumstances. Figure 3 shows a screenshot for the simulation game in which the player is presented with an unexpected 1http://bit.ly/1ddyitO 2http://bit.ly/1GEdBBX situation. As shown in Figure 3, the project manager is encountered with a situation wherein a developer quits the team or organization just one month before the release date.