Wipro recently briefed NelsonHall on its GenAI investments for quality engineering, discussing the creation of use cases and sharing the thinking behind some of its decision-making.

Wipro’s GenAI investments for QE are part of the company’s ai360 program, a $1bn investment that includes activities in developing use cases, training, and GTM. The launch of its QET GenAI Platform is part of this initiative.

It has identified ‘quick win’ use cases, including:

• Automated requirement analysis, test scenarios, case and script generation from user stories
• Test design recommendations
• Synthetic test data generation
• Knowledge management
• Data transformation and validation.

Like its peers, Wipro highlights the benefits of standard prompts, e.g., LLM’s output accuracy, lesser output variability, and capturing the client’s application and testing context. Wipro has created libraries of standard prompts, classified by role (UX designer, developer, tester, architect, BA, and application support) across the software development lifecycle.

RAG and Prompt Engineering

Beyond prompt engineering, Wipro wants to improve the accuracy of the LLMs. Rather than fine-tuning LLMs (through training the models on additional training data sets), it has chosen the retrieval-augmented generation (RAG) approach, which essentially relies on creating vector databases of the client’s testing artifacts. With the RAG approach, Wipro believes it takes a more relevant method to include the specific context of the client’s applications. To that extent, the company has created a tool that goes through various document formats (e.g., .doc, .pdf, .ppt) and creates a data set in a vector database.

For several use cases (e.g., test script generation, test data), Wipro wants to be LLM-agnostic and will connect with GenAI COTS (e.g., ChatGPT 3 and 4 and Azure OpenAI). It supports most test execution tools and languages (e.g., Selenium, Eggplant, Appium, and Playwright).

Looking to the future

The company is developing several GenAI use cases targeting specific tasks. Examples of these include locating an error in a Selenium script or writing a VB macro to migrate data from ALM to JIRA. Wipro is building a repository of use cases covering testing activities, taking a bottom-up approach.

There is a clear focus on helping clients beyond the interest stage and consulting engagements to PoCs and deployment. To facilitate client adoption, Wipro is looking to make its GenAI services enterprise-grade with assured data privacy and security. Options offered by the company include hosting on the client’s premises or its own.

Investments in GenAI will continue to be a priority in the foreseeable future. The company recently invested in data transformation and validation. Wipro plans to bring further depth to its user story analysis; it is exploring how to make user stories more standard and consistent within an enterprise. Current writers of user stories tend to have their own style. Wipro believes that GenAI can bring some standardization while increasing overall user story quality. The company also wants to go into more depth regarding automated root cause analysis beyond traditional defect classification.

Bringing an enterprise-grade service

Beyond LLM use case depth and standardization, Wipro believes that it will differentiate its value proposition by offering an enterprise-grade service. The company highlights it has taken several steps in this direction.

Wipro provides access to LLMs through its ai360. ai360 wants to ‘guardrail’ LLMs and systematically monitors and controls LLM usage. It ensures:

• The right usage, for instance, taking a persona-based approach and providing access to the right model
• Cost control, in a FinOps approach
• Reporting for corporate and regulatory compliance purposes.

Wipro has also worked on decreasing the time to create test scripts from an initial 15 minutes to one minute, relying on proprietary Python test script libraries.

The company highlights it has also progressed well in LLM output consistency. It finds the LLM output/responses to English language prompts can be unreliable. To overcome the challenge, Wipro created a library of UML models for specific processes (e.g., completing an online purchase transaction). It will update the UML libraries for each client and subsequently create test cases and scripts. With this approach, the company believes it can also increase test coverage.

Wipro points out that clients hesitate to move from demos and PoCs to deployment. The company believes its enterprise-grade approach will help organizations make the move and will continue to invest in it.