Tech Mahindra recently updated NelsonHall on its performance engineering (TMPE) capabilities.  The comments below complement an article on TMPE we wrote in January (see here).

TMPE, part of Tech Mahindra’s Testing Services practice, is in high growth mode, having increased its headcount from 450 in early 2014 to 800 currently.

Much of this growth has been driven by cross-selling into Tech Mahindra’s major testing and overall accounts, in particular in the telecoms, retail, and financial services sectors. Key offerings include mobile app performance testing and newer performance offerings such as ILM, test data management and non-functional assessment and strategy.

After the acquisition of Satyam, TMPE reviewed its service portfolio and pursued four major directions:

• Strengthening its existing performance testing portfolio
• Expanding testing capabilities to performance engineering
• Taking a shift left approach to introduce performance earlier in the SLDC
• Taking a shift right approach, with the intent to monitor applications in production and feedback this information into new projects/for new releases.

TMPE has continued with these efforts, adding accelerators such as SPARTA, making its mobile app testing more comprehensive, and introducing a consulting approach on its shift-right capabilities.

SPARTA

Scalability, Performance, Availability and Reliability Test Assurance (SPARTA) is a performance testing framework comprising of methodologies, check lists, IPs and tools to provide program-level testing services in non-functional aspects. With SPARTA, TMPE addresses two main client scenarios:

1. Introducing non-functional testing earlier in the life cycle project, as a traditional shift-left approach
2. Building virtualized performance testing environments for mitigating the traditional challenges of not having a dedicated performance test environment.

SPARTA has several service sub-components:

• Non-functional static testing
• Component non-functional validation (see below) and performance profiling (based on using tools including CAST Software and Visual Studio Profiling)
• Test environments (see below) and tool optimisation
• Non-functional test execution and
• Production performance monitoring.

TMPE highlights that while clients have become more sensitive to the need in functional testing to have an adequate test environment, they have yet to envision test environments from a performance perspective. TMPE has introduced a consulting service for this need, based on SPARTA to plan, design and execute performance testing with limited and/or virtualized test environments.

Non-Functional Service Simulators/Component Validation

As part of SPARTA, TMPE has created non-functional service simulators (NFSS) with the intent of taking a service virtualization approach to test environments for performance testing, expanding from the now traditional service virtualization for functional testing. NFSS simulates applications, from a performance standpoint, and integration with those (based on interface protocols including http, ftp, IBM MQSeries, XML and SOAP). Configuration time for NFSS is about three days per ten interfaces (provided it works on the protocols listed above).

TMPE provides NFSS as part of its service and as part of its differentiation strategy: it is thus able to save on service virtualization license costs.

SPARTA has been used in five client engagements to date.

DevOps Performance Engineering and SPARTA

TMPE is now applying SPARTA to the larger context of DevOps/agile development and collecting production data related to performance.

TMPE will then include provide application performance management services on production environments and make recommendations to its clients e.g. ranging from hardware-related upgrades to re-coding, re-configuring and application architecture changes into the design of subsequent releases.

TMPE is positioning SPARTA as a structured way of collaborating within the context of short implementation times and numerous releases found in agile and DevOps. TMPE is also pushing the same approach for waterfall and hybrid development methods.

New Offerings

Non-Functional Assessment and Strategy

TMPE’s non-functional assessment and strategy (NFAS) methodology formalizes its performance testing capabilities, e.g. in supporting clients set up performance testing CoEs/SSCs. Offerings include assessments of the client’s processes, tools, personnel and specific needs in terms of volume and SLAS. NFAS provides a strategy, a business case, a technology/process/tool/personnel roadmap.

Cloud Performance Testing

TMPE has formalized its cloud-performance testing capabilities into a structured offering, which has four main components:

• Cloud performance consultancy: based on client volume and SLA requirements, TMPE will help clients test the performance of applications on public and private clouds
• Cloud performance monitoring, applications on the cloud and private clouds
• Application performance testing on cloud: services range from scaling up verification; availability; as well as ensuring major processes on a web site can perform
• Testing application performance from cloud across geographies. This is an extension from the above offering, and takes a more systematic view of performance, depending on where the end-user is located and where cloud datacenters are.

TMPE has defined a bundled license and pay-per-use service, Performance Testing as a service (PTaaS). Hosting is done from cloud datacenters such as those of AWS or is on-premise. COTS license fees are included in the price.

PTaaS has required substantial work from TMPE in negotiating short-term licenses from ISVS (HP for Performance Center and Neotys for Neoload), and also in creating a price estimation mode. The pricing model is based on several criteria including assessing the complexity of applications (“complexity-based pay-per-use”), the number of transactions, creation or updating of test scripts, number of servers involved and the type of performance testing activity (stress, load, mobile, single cycle, multiple cycles.

TMPE is positioning PTaaS for existing relationships where clients and Tech Mahindra can work together understanding the pricing model and the client gaining transparency on its pricing.

PTaaS has currently four clients. Potential targets for this offering include ISVs and internet-centric firms with web sites processing high volume of transactions.

Mobile App Performance Testing

TMPE has completed its mobile application testing offering, which initially included conducting end-to-end performance testing at the server level, from profiling based on code analysis to test execution.

Among new features are:

• Evaluation of application performance on the device through the introduction of a monitoring agent on the mobile device. This is to measure performance and to assess  the impact of applications on device resources such as battery, CPU, and memory
• Emulation (3G and 4G) networks, devices (when not available) and combinations around OS and web browsers.

This offering is for large engagements, supporting clients engaged in adoption of mobility e.g. banking, retail, telecom service providers. Technology partners include Perfecto Mobile (for mobiles that Tech Mahindra does not have), HP and Neotys, and Soatsa.

Timescale of mobile app performance testing varies from two weeks to several months.

Roadmap

TMPE will continue to develop its service portfolio in several directions, including:

• Verticalization of offerings: e.g. in healthcare, application monitoring of health exchanges in the U.S. and assessing performance from different locations from an end-user perspective; in retail, to support seasonal peak performance needs
• Building new offerings around M2M and IoT: Tech Mahindra highlights an example in the electric automotive industry, where consumers starting their cars before the morning rush hour drive car diagnosis activity at its peak, thus requiring high level of server activity and influencing performance.

----

Tech Mahindra continues to impress with its continued investment in its testing service offering. In the past five years testing has expanded from a pure execution focus in many directions: test support services (environment provisioning and management; test data; and service virtualization/release management); different applications natures (ERP vs. non-ERP); different programming techniques (waterfall, agile/DevOps and to a lower extent toward hybrid), and data. The potential for testing expansion is vast; digital with mobile apps and big data will bring further expansions.

To some extent, the same is happening for performance testing services, which are moving from execution to shift-left, to shift-right with all the underlying methodologies, IPs and structured offerings. As Tech Mahindra Performance Engineering gains in scale, its offering will also continue to further develop.

And there is more to come: another topic found in testing has strong similarities with performance engineering and testing overall, in terms of having to be introduced earlier in the development lifecycle: than security testing. There is no question that over time security testing will also evolve. The range of testing services will therefore continue to expand.

Tech Mahindra’s non-functional service simulators are a good example of this service expansion. Service virtualization is becoming more well-known in the area of functional testing. The company now is using the same concept for performance testing. And Tech Mahindra is taking the same approach with DevOps applied to non-functional testing. Service virtualization and DevOps applied to performance engineering shows the investment that Tech Mahindra is doing in its non-functional service portfolio.

Cloud computing has become, like in functional testing, a major topic for non-functional. This is true whether testing applications on the cloud or IaaS services or testing applications from different geographies. PTaaS also brings a pay-per-use pricing element for a combined services and software tools offering.

The good news is that unlike functional testing, non-functional tends to be much more automated and will not drive in the inflation in career tester headcount we have witnessed with functional testing.

The downside is that testing is becoming even more a technical activity: testing services vendors need to educate their clients and highlight they will need to take non-functional into account early in the SDLC. Major horror stories in the market will help driving client education further up, especially in the context of retailers during the Christmas season. This is not to say that performance engineering will become overnight a major discussion topic: awareness among clients for performance engineering exists; yet it is somewhat inhibited by its technical nature. Therefore the best for clients to better understand risks associated with the performance of systems and applications in production is to start discussions with their software testing vendors, early in the SDLC.