An Open Testing Manifesto

Open Testing is a concept I began thinking about seriously after attending the CAST 2007 software testing conference earlier this summer.

I've been working in software quality for almost fifteen years, in testing and development for large and small companies, on embedded software, desktop applications, and web applications, using both agile and traditional methods.

By now I've left behind a vast body of work in software quality: test plans, test cases, test scripts, test results, and defect reports. Unfortunately most of this work is not shareable. Most software test engineers are in the same position.

The reason is simple: most of us have spent our careers working for companies or other entities that consider the testing activity to be proprietary information.

This fact shapes the level of sharing and collaboration possible between software testing professionals. What tends to get shared is information that isn't proprietary.

We share success stories and horror stories. We collect these into articles, presentations, blogs, books, even distinct software testing schools. We debate among ourselves which testing practices, tools, and certifications are best or whether any of them are any good at all.

What is largely missing is the collective body of testing work that could be used to evaluate some of the competing claims, as well as to promote greater collaboration and innovation in software testing.

One possible response to this situation is a practice I call Open Testing.

Open Testing is the practice of testing software in an open and public manner. In Open Testing, test plans, test cases, test metrics, and defect reports are publicly available so that the quality of the software under test and the quality of the testing activity can be assessed openly.

Some of the potential benefits of Open Testing include: providing a collective body of testing work that can be analyzed; providing a way for testers to "get your work out there"; providing better information to users about the quality of software releases and products.

Open Testing is definitely not about sharing your employer’s test cases, test results, or other intellectual property without their permission. If your employer decides to open up their quality to the world, good for them. But many employers aren't going to do that.

Open Testing can still be practiced by testing professionals who can't share the testing work from our day jobs. Simply download a publicly available software release onto your home computer, test it for a few hours a week on your own time, and report any bugs you find. Then blog about it.

Open Source Software is a good choice to start Open Testing. Many open source projects actively encourage testing, and have public online bug databases.

In this blog, I will write about examples of Open Testing in industry, open source projects that are requesting testers, tools that support and enable open testing, testing professionals who are practicing Open Testing, and my own journey into Open Testing.

If you are already doing Open Testing, let me know about it and I would be happy to link to your page or article here.