But don't take my word for it. McConnell provides plenty of evidence for the efficacy of code reviews in Code Complete:
.. software testing alone has limited effectiveness -- the average defect detection rate is only 25 percent for unit testing, 35 percent for function testing, and 45 percent for integration testing. In contrast, the average effectiveness of design and code inspections are 55 and 60 percent. Case studies of review results have been impressive:
* In a software-maintenance organization, 55 percent of one-line maintenance changes were in error before code reviews were introduced. After reviews were introduced, only 2 percent of the changes were in error. When all changes were considered, 95 percent were correct the first time after reviews were introduced. Before reviews were introduced, under 20 percent were correct the first time.
* In a group of 11 programs developed by the same group of people, the first 5 were developed without reviews. The remaining 6 were developed with reviews. After all the programs were released to production, the first 5 had an average of 4.5 errors per 100 lines of code. The 6 that had been inspected had an average of only 0.82 errors per 100. Reviews cut the errors by over 80 percent.
* The Aetna Insurance Company found 82 percent of the errors in a program by using inspections and was able to decrease its development resources by 20 percent.
* IBM's 500,000 line Orbit project used 11 levels of inspections. It was delivered early and had only about 1 percent of the errors that would normally be expected.
* A study of an organization at AT&T with more than 200 people reported a 14 percent increase in productivity and a 90 percent decrease in defects after the organization introduced reviews.
* Jet Propulsion Laboratories estimates that it saves about $25,000 per inspection by finding and fixing defects at an early stage.
Every time I see these statistics I get a little nervous because it seems like someone is trying to make an argument that tries to diminish the contribution testers make to a software project.
"Darn, we don't do code reviews, so I guess we have to have testers..."
But then I remind myself the issues are deeper and things are never as easy as they sound.
Jeff's previous post and the current one, make it sound like you just snap your fingers, start doing code reviews, and your problems are gone.
Problem is, reviewing code well is a hard skill to possess. Instead of the 80/20 rule it seems like the 20/80 rule: you spend all your time (however much you have) on code reviews finding the easy 20% of the bugs you could find while the other 80% lay dormant.
Part of what bugs me is the statistics and when they were taken. Different times, teams, languages, projects than yours. This doesn't wipe out the data, I still believe it, but it doesn't translate straight across either.
They say it takes 10,000 hours to become an expert at any particular thing. That works out to three hours a day for 10 years. How long have you spent doing detailed, thoughtful, deep code reviews? Here is the start of the problem.
I believe in code reviews and was very enthusiastic about them on a previous job. I worked really hard and got our team to start doing them. Then, ran into the problem that no one had developed the skill to get anywhere near the results of the statistics. And no one was interested in working very hard to improve their skills. Our code review experiment died a quiet death.
I do like being reminded of these statistics. Which ends up reminding me code review is a hard skill to learn and we all probably need more practice.
Toward that end, Scott Hanselman has a series on quality code to read. Worth checking out as a start.
No comments:
Post a Comment