NelsonHall has reported previously on how Tech Mahindra is developing its performance engineering capabilities through its Tech Mahindra Performance Engineering (TMPE) practice. TMPE has transitioned its services portfolio from performance testing initially to performance engineering, shift-left, and production (see here), helping client organizations to take performance into account at the software design phase. Also, TMPE unveiled SPARTA, its performance testing framework. SPARTA is the equivalent of a traditional functional testing framework and is part of TMPE’s program to automate most performance-related activities (see here).

And now, in the past year, TMPE has expanded its service offering to Internet of Things (IoT) performance testing with IoT-Performance Test Assurance. This is a niche offering: TMPE has had two main client engagements (with a healthcare device manufacturer and an automotive OEM), and a number of small engagements.

Interest from clients is growing quickly, and performance as much as security is one of the main concerns regarding IoT. But does IoT require a different approach to traditional performance engineering and testing?

The Challenge of IoT Complexity

One of the fundamental differences between IoT and a business application is its complexity. IoT systems require integration with many different technologies, going through the traditional IT infrastructure level (servers in the datacenter), networks, and applications (IoT platform). Add to this device diversity/embedded systems, different communication protocols, and the high volumes of data, and this adds up to a much more complex situation. Plus, the architecture of IoT means that interfaces are more frequent and usage of emulation/service virtualization more spread.

This is the theory, but how does it work in practice? To find out, TMTP has developed a methodology around a ‘typical’ IoT process. For example, for an electric car IoT implementation, performance services focus on:

• Stress testing: early morning, during the 7:00am to 8:30am period, when many drivers start their vehicles, therefore putting pressure on networks and servers
• Spike testing: during normal conditions of vehicle usage, for continuous synchronization of data between the vehicle and the datacenter
• Spike testing again, this time for handling emergency situations, e.g. a car crash, for critical communications
• Performance testing cases: endurance/soak testing (for making sure the systems can handle the volume of data provided by cars over a long period of time), stress testing (to understand the upper limits of the system), and resilience and failure testing.

To address IoT performance testing needs, TMTP has developed an accelerator for automating the process, OSPT Framework. OSPT has several elements: a UI, open source-based monitoring tools, simulator and service virtualization assets, and reporting and analytics.

And More Complexity Ahead...

Clearly, IoT performance testing is at the beginning of its journey. Complexity is increasing with storing data, across public and private clouds, around telecom protocols, APIs and interfaces. Obviously the lack of standards in the IT industry will trigger many testing needs. And there is more to come, not only from an IT perspective, but also from use cases: e.g. autonomous/self-driving vehicles will require fast vehicle to vehicle communication. Yet further complexity comes from different communication protocols co-existing and the need for further speed of processing from vehicles’ CPUs.

From a testing service point of view, IoT performance engineering and testing is a white space, with few testing services vendors having expanded their testing offerings in a structured manner. It is therefore refreshing to see Tech Mahindra Performance Engineering taking the lead here. As always, we welcome the creation of an accelerator to drive service industrialization and repeatability.

NelsonHall has commented about the Security Services practice of Tech Mahindra in a different blog to provide a full view of Tech Mahindra’s non-functional activities.