I have touched several times on how AI is being used in the context of software testing for reducing the number of test cases, optimizing coverage, and estimating the number of defects in an upcoming release of an application. And, as AI technology is becoming pervasive, we are expecting more use cases to emerge for software testing soon.

We recently had a discussion with Santa Clara-based Infostretch about how it is using AI in the context of test case migration, and how it is helping a large U.S. financial institution in optimizing the number of test cases, and porting test scripts from HP/Micro Focus UFT to Selenium.

This surprised us, as NelsonHall has not detected any major client appetite for test script migration in the past, with enterprises taking a bimodal approach, maintaining existing estates of test scripts in their current formats and using open source testing software or less expensive testing software products (than from mainstream testing software) for their digital projects.

Infostretch believes that with the emergence of AI, clients will reconsider their estates of test scripts and start porting them to open source software tools such as Selenium, the de facto standard in web application testing.

Infostretch in brief

Founded in 2004, Infostretch initially provided project testing and certification services around mobile apps, with a large U.S. communication service provider its first client. The company expanded its capabilities to testing commerce applications (Adobe, SAP hybris, Salesforce), and to DevOps/agile. QA/testing remains the core capability, although the company now also provides support in the development of websites, mobile apps, and software products, using an agile/DevOps approach. Infostretch today has 900 employees and services ~60 clients.

Helping a large U.S. financial institution to optimize its 150k test case estate

Infostretch’s’ AI testing activity began with a U.S. financial institution with a large estate of 150k test cases. Under a new CIO, the company decided to reduce its script estate while maintaining test coverage; in support of this, Infostretch used ML to identify redundant test cases, based on test case semantic similarities. The approach was complemented by a statistical combinatorial-based testing (CBT) engagement. Infostretch identified that 17% of the estate was duplicate test cases.

Infostretch then worked with the client on porting test scripts from a variety of test execution COTS (including HP/Micro Focus UFT) to Selenium, and also on automated manual test cases. Infostretch’s approach relied on identifying existing testing objects within UFT through a ML approach, and converting them into Selenium and Java objects. This initial effort then provided preliminary Selenium scripts, which test automation specialists complemented using QMetry.

The level of test conversion effectiveness varies as a function of the complexity of the test script. For simple ones, Infostretch argues that it can automate up to 90% of the test script migration. The proportion decreases with customized applications, and when objects are no longer standard.

In total, for this large U.S. financial services organization, Infostretch estimates that in the eight months this migration project has been going, it has helped the client increase its automation level from 45% pre-project to currently ~65%. The company has:

• Automated up to 70% of all manual test cases
• Migrated 30% to 40% of all test scripts. Infostretch differentiates between standard test scripts and those that have been customized, and believes that for this client, up to 70% of all test scripts can be migrated in an automated manner.

ASTUTE offering

Infostretch launched its AI-based Agility, Quality and Optimization in Software Testing (ASTUTE) offering in March 2018. With ASTUTE, Infostretch has grouped all its capabilities, from QA consulting, model-based testing, to test case optimization, to test case execution, test data management, and environment provisioning. ASTUTE includes several AI tools/bots:

• Tobot, for test case optimization
• Predictive QA, for predicting the number of defects in a next release
• Prescriptive QA, for prioritizing application features based on defect history
• Result Analyzer, using ML to help categorize automation defects into categories (e.g. errors and failures related to applications, environments, data, test scripts, or unknown), and to help automation engineers proceed to root cause analysis, fix the issue, and re-run the test script
• Testing infrastructure optimization, for identifying test environment needs, based on release history.

Changes in testing industry dynamics

It is encouraging to see an expansion of the use of AI tools in testing, particularly to niche firms such as Infostretch. There is ample room for innovation in the testing industry, and firms that specialize around AI testing are likely to be at the vanguard of innovation.

Infostretch claims that client demand for test script migration is accelerating, and is involved in a number of large projects, including one payment processing client with a 15k test case estate. The use of AI and automation tools, and of open source software in migration activity, is triggering a significant change in the testing industry, and this will also have a significant impact on tester skills.