Before and after

In my new job, one of the features I'm testing is a web-based installer on Windows. One type of tool that can be very helpful for installer testing is a system snapshot tool. Tools of this type take a "snapshot" of the system before or after an install or uninstall and compare the two for differences. This type of snapshot is not a backup. The purpose is not to restore the system at a later time, but to determine what has changed and whether the actual changes match the expected changes.

After looking at several free and commercial snapshot tools, I settled on an open source tool called SupermonX. SupermonX snapshots can track the state of files, registry settings, and services, generate comparison reports across specific file or registry folders, and verify if a report matches an expected result. Output is stored in text format, and optionally, XML format. SupermonX also includes an Explorer-like user interface for viewing snapshot files, and command line options for running the snapshots and reports via batch script.

These features make SupermonX a good candidate for use in automated testing of the installation process. As time permits, I plan to build some tools around it to automatically snapshot before and after installs and verify that expected files, registries, and services are as expected.

Even without automation, a snapshot tool like SupermonX (I pronounce it "super monks" as in Chinese martial arts film) can help in quickly understanding what an installer is doing. That can help to generate good questions for your developers about what the installer should be doing.

With automated installer testing you can ask another class of questions, such as "I noticed that the whatever.htm file is no longer getting installed in today's build, is that intentional?" If it is intentional, your developer will probably be impressed that you are paying such close attention that you could catch a change that he didn't bother to tell you about. If it wasn't intentional, you may have a bug. Even better.