AI and Machine Learning for Testing

Last year I didn't make it to the 2017 PNSQC conference. I wish I had, because I missed one of the coolest talks on test automation I have ever seen. I ran across it by chance, just last week, after taking an introductory course on machine learning.

Jason Arbon, CEO of Test.ai (AppDiff) presents his company's approach to using Artificial Intelligence and Machine Learning to automating the process of designing and running tests.

https://www.pnsqc.org/ai-and-machine-learning-for-testers/

For many years I have been experimenting with model-based testing as an approach to automate the design of functional test cases and accelerate testing and improve test coverage. Model based testing is a powerful approach, but since it was introduced over 20 years ago, the software industry has been slow to adopt its paradigm of using an iterative process of modeling application behavior and generating tests. It can be difficult for testers to convince management to invest substantial time or money in new approaches.

Now the world has machine learning, which is a different iterative process of humans teaching machines using sets of data, but which has applications in many domains, not just software testing.

But even with the proven success of machine learning, it can still be hard to convince companies to investigate in testing technology. Jason worked for Google and Microsoft previously, which have immense resources, but he still had to start his own company to make his dream happen.

Test.AI uses a neural network and machine learning approach, and provides an application for testers to teach their "AI brain" how to understand the application they are testing. Their brain was trained by being given the data from thousands of mobile apps, with help from crowd sourcing. It can also learn from libraries of test cases written by testers. This apparently makes it resilient to changes in the UI as well. And each new application tested by the AI makes it a little smarter.

Test.AI seems to have solved, or be well on the way to solving, multiple problems at once, ranging from test coverage to making automation resilient to changes in the application UI.

In summary, an excellent talk and exciting work that could transform the software testing industry.

UPDATE: I have since learned of two more startups who are trying to use machine learning to do test automation. The robots are coming. Are you ready?

https://www.testim.io/
https://www.mabl.com/