Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers

Debugging is often difficult and frustrating for novices. Yet because students typically debug outside the classroom and often in isolation, instructors rarely have the opportunity to closely observe students while they debug. This paper describes the details of an exploratory study of the debugging skills and behaviors of contemporary novice Java programmers. Based on a modified replication of Katz and Anderson's study of novices, we sought to broadly survey the modern landscape of novice debugging abilities. As such, this study reports general quantitative results and fills in the picture with qualitative detail from a relatively small, but varied sample. Comprehensive interviews involving both a programming and a debugging task, followed by a semi-structured interview and a questionnaire, were conducted with 21 CS2 students at seven colleges and universities. While many subjects successfully debugged a representative set of typical CS1 bugs, there was a great deal of variation in their success at the programming and debugging tasks. Most of the students who were good debuggers were good novice programmers, although not all of the good programmers were successful at debugging. Students employed a variety of strategies to find 70% of all bugs and of the bugs they found they were able to fix 97% of them. They had the most difficulty with malformed statements, such as arithmetic errors and incorrect loop conditions. Our results confirm many findings from previous studies (some quite old) - most notably that once students find bugs, they can fix them. However, the results also suggest that some changes have occurred in the student population, particularly an increased use of debugging tools and online resources, as well as the use of pattern matching, which has not previously been reported.