Software testing services continue to show vitality in their adoption of tools to increase automation. One of the areas in which vendors are investing heavily is AI, not just leveraging AI to increase automation through an analytics approach, but also in testing of AI systems. Back in November 2017, we reported on how Infosys was expanding its testing service portfolio in the areas of AI, chatbots, and blockchain. Infosys had created several use cases for ML, focusing on test case optimization and defect prediction. It was also exploring how to test chatbots and validate the responses of a chatbot.

We recently talked to Infosys about the progress it has been making in this area over the last year.

Adding more use cases for ML-based automation

Infosys continues to invest in developing additional AI use cases, and has expanded from use cases around test suite optimization, defect prediction, and sentiment analysis into three new areas:

• Test scenario mining, identifying areas of applications that are most used to help prioritize testing activities
• Defect analysis, to understand how a defect impacts functionality and identify areas of an application/code most affected by defects; also, to assess how defects may impact several applications. Infosys is also looking to identify defects that are similar and understand at what release the defects have occurred
• Traceability, linking testing requirements with their use cases to estimate test coverage.

Infosys is now systematically introducing these ML use cases with clients, and is currently at the PoC and implementation stage with 25 clients.

Infosys continues to work on additional AI use cases such as:

• How a new release impacts test cases
• Optimizing coverage
• Finding performance anomalies in response time of APIs, script execution, or websites.

Testing ML systems

Testing ML systems is also a priority. Infosys is initially focusing on several use cases across deterministic MLs (i.e. that will always produce the same output) such as bots, and on non-deterministic systems (e.g. defect prediction).

For bot testing, Infosys has been working on making sure a chatbot can provide the same response to the many different ways a question can be asked. For one client, it has created an algorithm for generating text alternatives around a question. It then validates that the chatbot’s response is consistent for all question alternatives, using Selenium.

In addition, Infosys is working on voice bots, and image and video recognition. For image recognition, it creates alternatives to an image and validates that the ML system recognizes items on the image.

Testing MLs is only just beginning; vendors such as Infosys are working on the challenge, and in use case after use case, are creating comprehensive methodologies for ML testing.

Some autonomous testing is on the horizon

Infosys is working on several pilots around autonomous testing. This is a bold ambition. Infosys has based its first autonomous testing approach on a web crawler. The web crawler has several goals. It scans each page of a website to pick up defects and failures such as 404 errors, broken links, HTML related errors. More importantly, the web crawler will create paths/transactions across one or several screens/webpages, and then create Selenium-based test scripts for these paths/transactions. This is the beginning, of course, and the first test use cases are simple transactions such as user login or order-to-pay in an online store.

NelsonHall will be monitoring the development of autonomous testing with interest.