NelsonHall has commented several times about the role of platforms in quality assurance (QA) and how these are playing a central role in functional testing in the world of agile methodologies and continuous testing. Platforms take a best-of-breed approach to software components and rely, by default, on open source software, sometimes including expert functionality from COTS.

In this blog, I look specifically at Ballista, a performance testing platform launched by Tech Mahindra.

The rise of functional testing platforms

The initial purpose of QA platforms was really to integrate the myriad of software tools required in continuous testing/DevOps with Jenkins at its core. This has evolved: IT service vendors have been aggregating their IP and accelerators around their continuous testing platforms, which become central automation points. The value proposition of platforms still centers on automation, but it is expanding to other benefits such as reducing testing software licenses.

Software license and maintenance fees represent approximately 35% of testing budgets, with the rest being external testing services (~40%) and internal resources (~25%). While software-related expenses represent an essential source of potential savings, few IT service vendors have yet invested in creating full alternatives to COTS in the form of platforms.

Tech Mahindra launches Ballista performance testing platform

Tech Mahindra has started with its performance testing and engineering activities. This makes sense: performance testing and engineering is more reliant on testing software tools than functional testing is: ~70% of performance budgets are spent on software license fees.

Tech Mahindra designed Ballista, relying on open source software, with JMeter (performance testing) at its core, along with Java-related frameworks and reporting tools such as DynamicReports and jqPlot, along with Jenkins (CI), in the context of DevOps.

The value of Ballista comes from integrating the different tools and avoiding tool fragmentation and silos, and from its functionality, which ranges from performance testing to monitoring of production environments (e.g. systems, application servers, and databases), bridging the worlds of testing and monitoring.

Ballista also has stubbing/service virtualization capabilities to recreate responses from a website (HTTP).

Tech Mahindra has created dashboards across the different tools, creating reports from tools that did not previously communicate. It has also worked on improving the UX for accessing the various open source tools.

Tech Mahindra’s approach to performance QA differs somewhat from functional testing platforms mainly in two areas:

• Ballista uses open source software and Tech Mahindra is not licensing for its IP. Thus, Ballista has no testing software license or maintenance fees attached to it. The IP helps Tech Mahindra in differentiating its performance testing and engineering capabilities
• When functional continuous testing platforms are implemented, the existing software investment of an enterprise is taken into account. By contrast, Ballista is offered as a complete solution, thus replacing some of the tools already present at the client.

Tech Mahindra will continue to enhance Ballista. Features on the horizon include:

• More comprehensive stubbing capabilities, e.g. middleware (IBM MQ Series)
• Production data masking for performance testing
• Further investment in UX, for instance, installing an agent on users’ browser, and collecting insights about topics such as e-commerce website conversion rates, along with performance issues.

Platforms will automate the full SDLC

The traditional boundary between software tools and IT services has become more porous thanks to IT service vendors and QA pure-plays investing in testing platforms. The functionality of platforms has been mostly functional, expanding into UX testing selectively, and now into performance testing. This is good news, as platforms are helping traditional test automation, expanding from its test script silo right across the SDLC. We will be monitoring the client response to Tech Mahindra’s no-cost performance platform along with market reaction to this “as-a-service” innovation.