The CEO’s Guide to the Business of Training, Part 3 helps you to understand the role of evaluation, and how you can use it as a proactive tool.

Did you know that evaluation is bigger than the training?

Evaluation is something that, like strategy, affects your entire business. In the first article in this series we shone a light on the role that your corporate strategy plays in your training program. In the second article we thought about how to measure it effectively, so that you as the CEO can get to a clear understanding of your return on investment.

Before we get started, I’d like you to imagine that you and I have been working together to put the principles from articles one and two in place for your business.

Imagine this:

Right in the beginning, you conducted a gap analysis, to better understand whether or not your staff have the skills to make your strategic plan a reality. We thought about what would be required, and how we would know whether or not we were successful.

Once we had this plan, you went back to the drawing board to understand what kinds of measures you already have that can demonstrate success. Where you didn’t have the right measures in place, we worked together to create them.

Then, using our new set of measures, which were perfectly mapped to your corporate strategy, we spent some time and completed a baseline measure. Together, we analysed the data to understand the existing variance. This is helping us to close a specific, identified gap.

And finally, we staged a trial, tracking all of the requirements and costs. We made sure that the trainees were supported before, during, and after the training. What we learned was that the training was successful, and created the specific change we were after.

This is cause for celebration! Or is it?

In discussing this outcome together, over a celebratory wine after work, you were so enthusiastic. I felt bad about bursting your bubble.

‘Gina, I am so excited,’ you were saying. ‘That trial was such a success. This training program will be fabulous. We should roll it out immediately.’

I put down my wine glass and smiled. ‘That’s great news, and I agree: It looks good. But it’s only one session. How are you evaluating the entire program?’

You realise the impact of the question.

The smile on your face faded as the question hit its mark. At best, evaluation is cultural: Evaluation considers entire practices and standards. It allows you and your executive team to collect, analyse and use information to improve effectiveness, learning and accountability.

Why your company needs an evaluation policy

Having an evaluation policy allows you to set high standards for your organisation and its leadership. It also allows you to think about the place that evaluation has in your organisation as a whole. Your learning and development program then becomes an indispensable way to strategically improve your performance. It also means that knowing how to design, manage and use evaluations is a skill that becomes part of your program. Such a structure gives you more confidence to roll out ongoing training.

An evaluation policy allows you to understand changes

Having an evaluation policy allows you to whether change happened, and to understand why and how that change occurred. Just because your team was trained, the changes at the end may be due to other factors that just measuring training will not tell you.

Here’s an excerpt from the evaluation policy of the Bill & Melinda Gates Foundation that demonstrates the role that evaluation plays in that organisation:

Evaluation is the systematic, objective assessment of an ongoing or completed intervention, project, policy, program, or partnership. Evaluation is best used to answer questions about what actions work best to achieve outcomes, how and why they are or are not achieved, what the unintended consequences have been, and what needs to be adjusted to improve execution. When done well, evaluation is a powerful tool to inform foundation and partner decision making about how to optimize scarce resources for maximum impact. It is distinct from other forms of measurement that focus only on observing whether change has occurred, not why or how that change occurred. (Source.)

An evaluation policy allows you to make good decisions.

Effectively measuring the training within your learning and development program will tell you whether or not the training creates the change you want. Evaluating the program is the next step: This is what helps you to make good decisions.

What is the difference between measurement and evaluation?

The key difference is that measurement is specific, and evaluation is broad.

By taking a measure, you can understand whether or not something has been effective. By completing an evaluation, you can understand whether that something also has value.

As the Institute for Teaching, Learning & Academic Leadership points out, evaluation takes a whole-of-picture view. It’s not just whether Training Session 1 with Trainer Bill was a good session. It’s also whether the skills were applied back in the real world, and whether the leadership was supportive of the new way of working.

As the Harvard Business Review points out, the people you train can appear to be doing the right things, even if they’re doing them incorrectly. If you don’t have good evaluation practices, you might not learn this fact until it’s too late.

How to build an evaluation program for your learning and development

Your evaluation policy will specify how you evaluate your programs. But if you don’t know how to build an evaluation program, where do you start?

The basics of an evaluation policy and program

Like any policy, you want to specify the purpose of the evaluation program and its place in your organisation.

The Center for Creative Leadership gives us some excellent questions that will kick-start your thinking.

They are:

  1. Who are the people who care most about the evaluation findings?
  2. What changes do we expect from the program?
  3. What is most critical to measure?
  4. How will we gather useful data?
  5. What did we learn?
  6. What did we do about it (what we learned)?

Hint: You will already have some of these if you have been following this series. You will already know what your measures are, and what your strategic goals are. If you don’t, please follow the links to read through them, because it will make applying the questions above much easier!

A good evaluation program builds on your understanding at each stage

The door that your evaluation program really unlocks for you is the impact of your learning & development program. You don’t realise it, but the other doors were already unlocked for you when you started thinking about strategy and measurement.

The University of Michigan created this process that explains how your evaluation program closes the training loop:

The evaluation cycle includes a needs assessment, a process evaluation, an outcome evaluation, and an impact evaluation. Each part is at a different stage, in order: Before the program, new program, established program, and mature program.
Include an evaluation of the process

As your learning and development program grows in maturity, you want to be able to deliver better training, more effectively, and more efficiently, with better outcomes, every time you run a session. Instead of doing the same thing over and over again, iterate and do it better every time you run it.

This is why the process above evaluates the process. It asks: What impact is your learning and development program having on your business?

You will want to understand:

  1. The up-front impact, in arranging training days and people away from work.
  2. The impact of the methodology: Maybe you can deliver training in more effective ways.
  3. The impact on the people being trained: Maybe you can find ways of making the training more engaging, more pleasant to do, refreshing to participate in.
  4. The impact on the supervisors, leaders and executive: Are there ways in which you can improve the relationship between these people and the training you deliver, and how they follow up and help to embed the new skills and behaviours.
  5. The impact on your progress towards your strategic goals.

A clear method of evaluation gives you, and your team, the ability to understand impact. This understanding enables them to better make decisions about training, leadership, and investment.

As David Ballard, director of the American Psychological Association’s Center for Organizational Excellence wrote for the Harvard Business Review:

While there are many uncertainties about the future of work, one thing is sure: Adapting to a rapidly changing environment requires a strategic approach to training and development. It’s the best way to ensure that employees can see what is possible, solve previously intractable problems, and do the work that will lead to a prosperous future.

Next in the series: Designing for long-term success

Coming up next in the CEO’s Guide to Training is how to design a learning and development program for long-term success. We will take you through some of the principles of great learning program design, so that you can make sure that your training programs are successful.

Make sure you don’t miss out on this final instalment by getting posts sent straight to your inbox: