Provision of analytics is important for Learning BPO (LBPO) providers to be able to demonstrate ROI for clients. The late Don Kirkpatrick, who passed away in May at age 90, developed back in the 1950s four levels to evaluate learning effectiveness. Taking a look at these levels, starting with the lowest:

1. Reaction; evaluation about how the participants felt about their learning experience including course feedback forms and comments from participants including through communities
2. Learning; evaluation of the increase in knowledge after the participants learning experience including pre-post course test scoring and feedback in performance reviews and appraisals
3. Behavior; evaluation of applied learning through observation of on-the-job behavior including from customer feedback and peer review
4. Business results; evaluation of effect on the business including financial results and customer satisfaction

Utilization of Kirkpatrick analytics to measure business results is important for several reasons, including:

• Being able to demonstrate the value of learning programs and to receive future budget allocation
• Reducing wastage of ineffective learning
• Improving content design to enhance the employee learning experience

Learning services providers continue to show the most mature analytics processes of all HR services.

• For a multinational consumer goods company, Accenture helped increase sales via development of a training campaign to address slow sales growth, market share and brand penetration in China. The campaign resulted in a ~$50m increase in sales
• For a U.S. headquartered aerospace and defense provider with 80k employees globally, Raytheon Professional Services, using 6 sigma training, increased sales per employee from $175k to nearly $250k, +43%

Some business metrics are about reducing costs, rather than increasing sales:

• For a German bank needing to reduce 3rd party vendor costs by 15% and enable key personnel to focus on higher value activities, IBM improved rate cards, renegotiated contract terms and performed vendor rationalization; resulting in a 23% savings
• For a metals and mining client’s IT learning group, Talent2 developed standard templates, updated and validated ~2,000 core materials to be common, resulting in 19% less “how to” calls to the helpdesk.

Vendors continue to make investments in analytics:

• Aon Hewitt and Xerox partnering with Knowledge Advisors to enhance their analytics offering and help clients improve learning effectiveness and business impact
• GP Strategies acquired the Effective-People and Effective-Learning companies in Denmark

There are numerous examples of improved results, but there is still a long way to go. Beyond Kirkpatrick level 1, deployment and utilization by clients of levels 2-4 decline each phase. All vendors offer levels 1-4 but there is variation. Some offer levels 1-3 at no additional charge, with level 4 contracted separately or on a project basis. Others offer 1-4 at no additional charge.

• Xerox finds most companies will do level 1, some will use level 2 and fewer use levels 3 and 4 due to cost and time to measure on the job behavior and business results
• Similarly, Talent2 finds utilization rates by clients at around Level 1: 100%, Level 2: 80%, Level 3: 15%, Level 4 <10%

NelsonHall believes that utilization seen by Talent2 and Xerox are consistent across LBPO, with Level 2 Learning closer to ~50%.
   
Per NelsonHall’s Q4 2013 Learning BPO market analysis, analytics capability is among the key vendor selection criteria. A major challenge for vendors is not just providing analytics as part of the service but getting clients to take the extra step and measure business impact. Clients need to take ownership and make the investment to measure impact to the business, which in addition to job skills and employee development, is the purpose of training employees!