We recently talked to test IO, the crowdtesting vendor that was acquired by EPAM Systems last April. We wanted to understand the crowdtesting positioning of the company, learn more about the User Story crowdtesting offering launched in December 2019, and understand how test IO fits within the larger EPAM organization.

Founded in 2011, test IO today has 200 clients, many in the retail, media, and travel industries, with a sweet spot around customer-facing applications.

test IO has positioned its crowdtesting portfolio in the context of agile

test IO has focused on functional testing (along with usability testing), targeting agile projects in recent years. Crowdtesting in the context of agile development projects (i.e. continuous testing) remains a priority and test IO recently launched an offering named User Story testing.

The crowdtesting industry has to date relied on two main offerings: exploratory testing and test case-based testing. Under the exploratory testing model, a crowdtester gets few instructions on what to test and goes through the application-in-test by intuition. With a test case-based approach, the crowdtester relies on detailed instructions on how to complete a task or a business process.

With its User Story approach, test IO is promoting a method that lies somewhere between exploratory testing and test case-based testing. In agile development methodologies, user stories are the instructions given to software developers in the form of “as a [user/admin/content owner/], I want/need [goal] so that [reason]”. The challenge from a testing perspective is to test these user stories by setting up acceptance criteria that are specific enough to be tested but still meet the spirit of agile that relies on loose requirements and iteration.

Speed is of the essence, as with agile projects. test IO argues it can achieve User Story testing in approximately two hours, from the moment it sends a mobilization campaign to its members to the moment it receives defects back from crowdtesters. The company highlights that working with a population of crowdtesters with the right level of testing skills helps in achieving speed. test IO believes it has enough members to react quickly to any project invite while providing the right defect coverage.

Exploring portfolio synergies with EPAM

test IO has also expanded its delivery model from public crowds and private crowds (e.g. employees of the client) to the EPAM internal crowd. test IO can rely on EPAM’s 6k testing practice members to expand its reach and bring to the client career testers with vertical experience while reassuring a client that its application-in-test will be exposed to EPAM personnel only.

User Story testing and internal crowds are just the beginning: test IO and EPAM intend, over time, to expand their crowdtesting capabilities to specialized services: performance testing will be the first of these.

AI is also on the agenda. One of its first AI use cases has been around crowdtester selection, based on the technologies and experience needed by the client. A current priority is defect review. For each crowdtesting project, test IO will review the defects logged by crowdtesters and remove duplicates. The company wants to automate most of its test duplicate activity to free up time and focus on data analysis.

In a later phase, test IO wants to run through the defect data it has built during its nine years of existence and identify crowdtesting best practices and application defect patterns.

test IO hints that EPAM is exploring how to best use its experience in crowdsourcing and expand it to other software services activities. We will be monitoring with interest how EPAM develops the crowdsourcing model of test IO. Despite claims of attracting young talent through different employment models, most vendors still rely on permanent or freelancer positions. With test IO, EPAM may be able to invent a new employment model that will expand from testing to other application service activities.