NelsonHall continues to examine the AI activities of major testing service vendors. In the past 18 months, many testing service vendors have expanded their AI capabilities around analytics, making sense of the wealth of data in production logs, defect-related applications, development tools, and in streamed social data.

We recently talked with Expleo with regard to its AI-related initiatives. Expleo is the new company that has resulted from the acquisition of SQS, a QA specialist, by Assystem Technologies, an engineering and technology service organization.

Expleo highlights that its expertise lies around rule-based systems, an area that is now considered part of ML, and for the past 12 years it has created several rule-based systems use cases (e.g. defect prediction and code impact analysis). It now has around ten AI use cases, several of which are now in widespread use in QA (e.g. for sentiment analysis, defect prediction, and code impact analysis).

Other use cases remain specific to Expleo. One example of an Expleo-specific use case is related to false positives, identifying test failures that are not due to the application under test but caused by external factors such as a test environment not being available or a network suffering from latency, during test runs. Expleo has developed an IP, automatic error analysis and recovery (EARI), that relies on error classification and a rules engine. EARI will launch a remedy to the false positive by the applying a ‘last-known good action’.

Expleo continues to invest in developing AI use cases. The testing industry is mature in automation once test scripts have been created, but remains a manual activity before the creation of scripts. Expleo is currently working on creating test scripts from test cases written in plain English, using NLP technology. Another AI use case is a real-time assessment of software quality or script auto-generation based on user activity and behavior.

AI in QA is still in its infancy phase and many issues remain to be solved. Expleo is taking a consultative approach to AI and testing, educating clients about the ‘dos and don’ts’ of AI in the context of QA. The company has compiled its best practices and wants to help its clients redefine their test processes and technologies, taking account of the impact of cognitive technologies on organizations.

Data relevancy is another priority. Expleo points out that clients tend to place too little emphasis on the data being used for AI training purposes. Data may be biased, not relevant from one location to another, or just not large enough in volume for training purposes. Expleo has been working on assessing the data bias, based on data sampling and a statistical approach to AI outputs. Once complete, Expleo can identify the training data causing the output exceptions and remove it from the AI training data.

Expleo is also working on bringing visibility to how ML systems work, with its ‘Explainable AI’ offering. The company highlights that understanding why an ML system came to a specific outcome or decision remains a challenge for the industry. Yet, understanding an AI’s decision process will soon become a priority for compliance or security reasons. An example is around autonomous vehicles – to understand why and how vehicles will make decisions. Another example is for compliance reasons, being able to prove to authorities that an AI meets regulatory requirements.

With its new scope and size (15k personnel), Expleo is expanding its QA capabilities towards the engineering world, around embedded software, production systems, IoT and PLM, which will require further investment in AI. This is just the beginning of Expleo’s AI and testing story.