We recently talked to crowdtesting specialist firm Testbirds about how the firm has adapted to COVID-19 and the macroeconomic conditions.

In the past two years, the crowdtesting industry has realigned its capabilities around agile testing, primarily providing services for mobile apps and websites along with chatbots and connected devices/wearables. The realignment has been successful. Nevertheless, the crowdtesting industry has remained a niche industry.

Impact of COVID-19 on buyers’ use of testing specialists

Significant industries for crowdtesting such as retail, travel & hospitality, and automotive that are being particularly badly impacted by the COVID-19 pandemic are also among the largest spenders on crowdtesting and would be expected to have cut their spending abruptly. This is true to some extent, with several clients putting their projects on hold. However, other industries such as energy & utiltiies, telecoms, media & broadcasting, and financial services have maintained their spending.

Testbirds has witnessed increased activity from both existing and new clients that were not prepared to conduct testing from home and are using Testbirds to complement their QA operations.

One example where a client has increased its use of Testbirds is a large automotive OEM, which put its plant workers on furlough but continued its investment in its digital projects. Testbirds has conducted UI testing for several mobile apps, including ones for its motorcycle and passenger car businesses (one acting as a remote control for vehicle connectivity and entertainment).

In parallel, Testbirds hired a channel director for its activities 12 months ago and initiated a partnership program at the beginning of 2020. The company highlights it has signed eight formal partnerships, including five tier-one QA vendors, and started joint projects. In contrast, 12 months ago, Testbirds had only tactical agreements for one-off projects.

Testbirds is accommodating its services for the indirect channel, letting the QA vendor manage the project and access all of the Testbirds IP, community and project manager. Testbirds continues, however, to supervise its community and its automation investment, even if it is white labeled.

Will COVID-19 reduce client concern around trust & security?

In the past, clients have expressed two primary concerns around crowdtesting: trust (will crowdtesters tell my competitors about my new app?) and security (will they hack my system?).

Testbirds believes that clients that have experienced work-from-home and enjoyed benefits such as higher productivity and work quality will now increase the use of crowdtesting. It emphasizes all of its testers are vetted and go through a thourough onboarding and qualification process, and its systematic NDA policy. Also, Testbirds is mitigating the IT security risk through secure connections and access to staging environments (i.e. a replica of testing or production environments).

Crowdtesting becomes more automated

With crowdtesting now emerging out of its niche, Testbirds is getting ready for changes. The company is increasing its investment in automation, focusing initially on crowdtesting member identification and notification (using AI and its companion mobile app) and automating communications and defect login.

Testbirds highlights that clients typically start their journey with exploratory testing and usability studies, combining functional and usability testing, and then expanding to its deeper usability research and functional testing portfolio. The company wants to further improve its UX capabilities (which already represent 50% of its business) and is soon to release an ML-based algorithm to go through to open question responses and categorize them.

Longer term, Testbirds wants to push waterfall testing to its client base, expanding from agile crowdtesting. This presents new opportunities for the company, given that most of the mobile apps/websites it tests are connected with back-end applications that rely on waterfall development. The company highlights it has already started this journey and developed relevant methodologies.

The sweet spot of crowdtesting is ‘out of the lab’

What is striking about the adoption of crowdtesting is that clients are using crowdtesting as an alternative to traditional functional QA manual activity. Clients turn to crowdtesting for their functional needs and then only gradually expand to usability testing. Testbirds highlights its clients use equally functional and UX testing and is now converting them to adopt further services.

We think the crowdtesting sweet spot also lies in providing QA ‘out of the lab’, whether at home with real network conditions, in the street, or the store. Accordingly, we expect clients will, over time, increase their focus on omnichannel testing and give more attention to usability testing. So far, demand has been relatively limited. COVID-19, by accelerating the transition to digital, is changing this. Now is the time for the crowdtesting industry.