Evaluating Leadership Development: A How-To Guide
Several months into a new leadership program for first‐time managers, and an HR team is breathing easy.
The launch and follow‐up sessions were well‐received, with enthusiasm spilling over into the use of mobile and self‐paced online courses. A second cohort is slated to begin soon.
Then, the discussion turns to measuring impact — finding ways to present the initial good news and demonstrate value over time.
Unfortunately, this team has already missed the best window for getting an evaluation started. The most useful and effective evaluations for measuring impact are those that are designed at the beginning — parallel to designing your program or initiative — and not tacked on later.
“You can never go back to the beginning,” says Tracy Patterson, the director of evaluation services in our Leadership Analytics practice. “It’s much better to think carefully about evaluation at the onset than try to work from a retrospective lens.”
If the idea of measuring the impact of your investments in development creates anxiety for your HR organization, we can help. At CCL, we address evaluation early on in conversations with HR leaders looking to create or expand their leadership development efforts. Evaluation and techniques for measuring impact are threaded throughout the discovery, design, and implementation process for all our custom leadership development offerings.
Our 6 Recommended Steps for Measuring Impact
Research-Based Recommendations
Based on our extensive experience, research, and best practices in the field, we suggest that clients take these 6 steps toward measuring the impact of leadership development in the earliest stages of program planning and design.
1. Engage evaluation stakeholders.
The primary purpose of evaluation will stem from the needs of stakeholders, so identifying who they are and learning about their expectations, concerns, and perceptions is important. Stakeholders could be those who are involved directly — participants or delivery team — or those who could invest in the initiative or are affected by it.
Then, determine whether all stakeholders are in agreement about the initiative and aligned in terms of outcomes or intended impact. Without this step, you miss an opportunity to plan for, and collect, valuable information for measuring impact. When you put the effort in at the beginning, you can increase the likelihood that the results will be both relevant and helpful.
2. Connect with the initiative design.
Evaluators need a solid understanding of the organizational challenges driving the need for leadership development and who the initiative targets and their leadership needs. Then, the more difficult part: figuring out the specific measurable objectives of an initiative. Evaluation design is based on knowing the connection between each program element and what it’s trying to achieve:
- What broad goal is addressed by the initiative? What are the objectives? What’s the context?
- What knowledge and skills is the program designed to develop? What actions or behaviors reflect those desired lessons learned?
- How will we know learning has occurred? What should be signs of success? What degree of improvement is expected?
- How will evaluation information be used and by whom?
When this step is taken early on, it also can strengthen the design of the initiative. It can reveal details or disagreements that had been overlooked, as well as unnecessary elements or gaps in the program design.
3. Focus the evaluation.
Narrowing the scope to what is most critical to measuring impact is the next step. It’s easy to end up collecting more information than you need, or miss collecting something that ends up being of great interest to your stakeholders if you breeze through this part of the process. Plus, you can also avoid conflict down the road if you can simplify and summarize the evaluation at this point, so stakeholders get another chance to understand and buy into what you’re doing to measure impact.
4. Confirm resources and expectations.
Are resources available to match expectations? Often, the 2 are not aligned. Taking a reality check at this point allows time, people, and budget to be realistically factored into the evaluation process.
5. Determine data collection methods.
Surveys, interviews, focus groups, observation, and documentation are common data collection methods, and each has advantages. Different combinations will work best, depending on what data will be gathered, from whom, how, and when. This is also the time to talk through the burdens placed on the people providing data for measuring impact, address issues of data security and confidentiality, and adjust the timeframe for data analysis and interpretation based on the final design.
6. Plan for communication of results.
There are many ways to share evaluation information and often several audiences. Determine the best way to communicate upfront, so it can be factored into the planning. The goal is to put useful impact data into the hands of stakeholders when they need it and in a way that encourages discussion and action.
Evaluating the impact of leadership development is always a complex challenge. Taking these 6 steps, in the beginning, gives you the best opportunity for success.
Some Dos & Don’ts for Measuring the Impact of Your Investments
Best Practices Based on Experience
Over the years, we’ve seen the good, the bad, and the ugly of evaluation. We’ve helped HR teams with a wide range of needs and know-how. Take a look at these tips for how not to measure impact, and instead, let the Dos guide you down the right path.
1. Don’t delay.
As noted above, it may be tempting to focus on delivering the initiative now and worry about the outcomes later. But if you wait to design the evaluation and start measuring impact until after implementation, you miss the chance to ensure that the design is tied to expected outcomes and to build measurement into the process. One of the benefits of evaluation is the thought process that goes into it — thinking about what matters enough to measure later helps to clarify the goals of the effort today.
Do design the evaluation as you’re designing the initiative, and well before it’s implemented.
2. Don’t deny.
It’s easy to assume good things will happen when you provide learning and development. But your efforts may miss the mark, or have unintended consequences. If you don’t take a good, research-based look at measuring impact, you lose the chance to learn from mistakes and capitalize on successes.
Do use evaluation as a tool to enhance organizational learning.
3. Don’t rush.
External deadlines and directives for development may create urgency — but don’t forget to focus. Slowing down to ask the right questions is essential for getting results that will matter to your organization.
- Do spend time on discovery. At CCL, we have a comprehensive process of partnering with you to uncover and understand your leadership development needs.
- Do discuss and define the purposes of each evaluation.
4. Don’t work in isolation.
Make sure appropriate stakeholders (Executives? Participants? Their managers? External funders?) are clear on expected outcomes and what they want from the evaluation. If you wait until after evaluation data are gathered to include stakeholders in measuring impact, they can argue about what was wrong with the process, rather than use the data to move towards a shared goal.
- Do involve stakeholders at all stages of the process.
- Do create organizational support for leadership development initiatives and evaluation.
5. Don’t oversimplify.
Don’t just focus on what you can control or the most obvious metrics. Evaluation can, and should, factor into the complexity and reality of leadership development in organizations.
Do consider multiple measures, multiple perspectives, and measures at multiple points in time in order to gain a comprehensive perspective of program processes and outcomes.
6. Don’t override.
If you or a business leader or colleague doesn’t like a part of the process of measuring impact or a question that’s asked, don’t just change it.
Do work with your evaluation experts to deal with concerns or changes — and to understand the evaluation design and methodology.
7. Don’t blame.
Evaluation data will likely reveal problem areas. Don’t use the measurement data as ammunition; instead, use it as fuel for improving and learning.
Do be honest about the data, even if it shows shortcomings.
8. Don’t halt progress.
Measuring impact gives information that can help you create broader change and learning at your organization — but not if a few slides make it into your next presentation, and then the report is filed away.
Do use the evaluation findings. Figure out how to effectively communicate the findings, identify specific courses of action, create action steps, and see them through.
Evaluating Leadership Development at the Organizational Level?
A Special Note About Measuring Impact of a Culture Change Initiative
As noted above, while there’s often a temptation to delay measuring impact until after an initiative is completed, it’s best not to wait. That’s true for individual leadership development efforts, and also true for evaluating the impact of a culture change initiative. Measuring impact isn’t as simple as a box you can check at the end — especially when it’s about your transforming your organization.
Organizational culture change is about changing beliefs, assumptions, and attitudes — and changing these things is never easy or fast.
Strategically collecting, analyzing, and using data along the way can tell you whether your culture change initiative is successful and provide insights that can make your efforts more effective. Using data to gain insights throughout the process can shift beliefs and practices — and lead to real, long-lasting organizational change.
That’s why we recommend moving from a mindset of measuring results to ongoing evaluation. Make learning integral to the change process. Think about the impact desired and what you want to achieve early, and use data to drive insights throughout the culture change process.
As with other types of evaluations, the most effective way of measuring the impact of a culture change requires planning before the initiative even begins. Read our full white paper for more advice on how to measure your organization’s culture initiative, including:
- The big questions to ask;
- The mindset shifts that are needed;
- What you should measure;
- How and when to measure; and
- How you’ll use the results.
While you could skip ongoing measurement of the impact of your culture change initiative, just as with measuring the impact of other types of investments in your people, the ultimate outcome is likely to be less positive and impactful.
Ready to Take the Next Step?
We can help your HR team create scalable but personalized leadership development to ensure you get the results that matter most to your organization. We can provide customized leadership development for your unique context and culture, and our Leadership Analytics experts can help you with diagnosing your current state and measuring impact
