Robert O’Callahan came up with a handy idea for Mozilla QC — automated layout regression testing. Based on an idea implemented by Ian “Hixie” Hickson for Opera, this would automatically test the daily Mozilla builds for new layout bugs.
At first, I couldn’t figure out how such a task could be automated. After reading over the planned outline, I realized that it was a rather elegant approach. In short, the script (Perl, in this case) would keep a set of known-good screenshots of various layout tests. Then, on each day, the script would the build through the series of tests, taking screenshots along the way. At that point, it would just be a matter of doing file comparisons between the known-good screenshots and those from that day.
The code isn’t yet complete, but it’s moving forward. The trickly parts are in dealing with unruly builds (as you may have guessed). For instance, if that day’s build crashes on one of the tests, then the script would need to detect that, kill the process and then restart Mozilla for the next test sequence. In all, I have a good feeling about this development.