Since its creation, the software testing services industry has focused on regression testing, i.e., ensuring that previously developed software still runs after a new build. Regression testing is the core of all QA activities and has been widely adopted as part of multi-year managed services contracts. The financial services industry, with its extensive application estates updated by one or two releases per year, has been the largest spender.

For many years, the testing industry has left out progression testing, i.e., testing new applications and new features (rather than enhanced ones, as found in multi-year contracts). There is a reason for this, as new-build projects are short-term in nature: they do not easily accommodate the longer-term view and costs of building automation over time.

With the widespread adoption of agile, the situation has changed to some extent: agile projects, focusing on iteration and speed, have required functional automation to support accelerated development.

COTS functional automation suffers from lack of time

Then there is systems integration/COTS testing. COTS testing has largely remained outside of test automation. Organizations are challenged by a lack of time and budget to drive functional automation for their ERP/COTS projects.

Technology plays a more significant role: several specialized ISVs have emerged, including Worksoft, Panaya, and Provar (Salesforce). Their tools have focused on handling the technology specificities of these COTS.

In addition, testing services providers have complemented specialist tools with repositories of test cases aligned by a transaction. The more advanced services providers have also used model-based testing (MBT) for modeling transactions. However, the test case repository approach has its flaws, as clients will need to customize test cases. Acceptance of MBT has been somewhat limited, as it requires creating another layer of testing artifacts.

AI helps to redefine COTS functional testing

The real gamechanger for COTS testing has come from AI. For instance, Atos released its Syntbots xBRID to address Salesforce, SAP, and Pega projects.

xBRID is a next-gen record-and-playback tool. While testers will complete a transaction in Salesforce xBRID will capture the activities performed by the user. It will generate test scripts automatically and avoid the human-intensive scripting phase.

xBRID is also a testing framework whose execution relies on the Eclipse Integrated Development Environment (IDE). Testers will thus need to apply the usual discipline and componentize test cases, e.g., log in or application launch, to provide common sub-activities across transactions. xBRID works for web applications and with mobile apps (Appium). It also integrates with BDD test frameworks.

Perhaps more important, xBRID helps with test script maintenance. Atos will identify all the objects in a UI/screen and create libraries of such objects with their location and other characteristics. xBRID will scan each UI/screen during each sprint, identify changes, and update objects. As a result, test scripts will handle UI changes such as field name or position change and go through execution. This is an important step: test scripts can be fragile, and their maintenance has in the past been heavily human labor-intensive.

Handling complexities of SaaS applications and Salesforce

Atos continues to develop xBRID. Each COTS has its specificities; for instance, Salesforce has two UIs, Classic and Lightning, that complicate test automation. Also, Salesforce, with its three annual releases, brings continuous changes in HTML, CSS, and APIs.

A feature of SaaS applications and Salesforce is that they force clients into several upgrades per year. With on-premise applications, organizations can decide when to update and upgrade, but with Salesforce and SaaS applications, clients need to test whenever the ISV deploys a new release. Having these regular releases provides a case for investing in functional automation. Tools like xBRID will help.

Atos estimates that with xBRID, it can save up to 90% in testing time. What appears to be a massive reduction is because xBRID replaces mostly manual test activities. It is essential that the industry increasingly targets automated test script generation, something Atos calls “Automation of Automation”. Automated test script creation and maintenance is a paradigm shift for the industry.