How We Built a Shared Measurement System
March 2025
This article is rated as:
The ability to systematically measure and evaluate impact is important for most organizations. Shared measurement systems offer a holistic approach to tracking data across multiple programs or organizations.
A shared measurement system is a collaborative framework that allows discrete groups to systematically collect and report data using standardized metrics and methodologies. The shared system facilitates the comparison of outcomes across different entities and provides a comprehensive view of the overall impact of initiatives.
A shared measurement system focuses on common indicators rather than common data collection tools. While it is likely that data collection will be similar (or even the same) across the different groups, the system itself doesn’t necessitate shared tools. By using common indicators and definitions, organizations or programs can align their efforts, share best practices, and drive collective progress toward shared goals. The data are reliable because everyone is measuring and tracking in the same way.
Building a measurement system is not without its challenges. One common issue is data quality. It is essential to ensure that the data collected are accurate and complete. There are many ways to tackle this problem, which could include training or implementing quality control measures.
We recently built a shared measurement system so that dozens of programs could measure their impact in a standardized way. Here’s a step-by-step description of what we did.
1. Form an advisory group – We wanted to make sure our plan was grounded in the real experience of those responsible for tracking and reporting measures. We needed to ensure that our proposed activities and solutions were feasible, so we met regularly with a group of five representatives.
2. Current state analysis – We needed to understand what the current state of measurement looked like: what was working and where were the pain points? To do this, we:
designed surveys, interview and focus group guides to hear from as many individuals as we could
did a substantial document review with materials shared by the client; the documents focused on what resources were available to support the current measurement system
had a tour of the current measurement system to understand the process of collecting and submitting data, and to understand how to fit a new measurement approach into the existing infrastructure (i.e., web portal)
reviewed past years of submitted data to understand data quality issues
(If your project has existing data collection tools, those would be included here as well)
In this case, the current state analysis garnered some important information, including that program leads wanted opportunities to share the impact of their work that was not captured in the KPIs. More on this later.
3. Jurisdictional scan – We completed a quick jurisdictional scan to look at other large organizations that had shared measurement systems. We wanted to know what change management strategies they employed and other tips to make the development and implementation of the measures smooth and effective.
4. Build familiarity with guiding documents – In this example, the development of the measurement system was guided by a Framework, complete with a logic model, outcomes, objectives, goals, and KPIs. The Framework was the visionary document that laid the foundation for why a new measurement approach was necessary and where it needed to align. As opposed to the current state analysis document review, which focused on historical documents, knowing the visionary framework inside and out helped us to ensure that we were meeting the organization’s vision and needs.
5. Create measures – Eventually, you have to begin putting all of your information gathering to use and begin to craft the measures that will make up the system. In this case, we had to ensure that all the KPIs were covered, and that the measures aligned with the Framework’s goals, outcomes and objectives. The measures had to be relevant, reliable, and feasible to collect.
We collaborated closely with stakeholders to ensure that the measures not only reflected their priorities but also provided valuable insights into the organization's performance and impact.
To actually build those measures, we started with a deep understanding of what they currently used: what worked and what didn’t. Where alignment existed, we kept what we could to reduce the amount of change being introduced. For new measures, we used literature reviews and even AI to help us create measures to align to the KPIs.
Remember how we learned in the current state analysis that the programs wanted more opportunities to share impact? We worked with our client to develop a way for programs to offer case studies or success stories in the form of short narratives. This allowed the programs to optionally share impactful stories, but also made sure the data were useable on the receiving end without being overwhelming. While this was technically outside of the scope of what was needed, it helped to gain buy-in from the programs.
6. Field test – Field testing is always a good idea, which was reinforced through learnings from the jurisdictional scan, where other organizations advocated for participatory approaches to designing shared measurement systems. We shared drafts of the measures with program leads to gain their feedback on how feasible and relevant they thought the measures were. This definitely helped us to refine the system.
7. Standardize – We learned early on that one of the major issues with the current measurement system was the lack of definitions for key terms. Thus, confidence was low in the data submitted because each group could be interpreting the question slightly differently. We used the Delphi process with a group of subject matter experts, in this case, to come up with standard definitions for core terms, thereby enhancing the reliability and comparability of the data collected. Learn more about how to use the Delphi process here.
8. Develop supporting resources – Once we had the measures solidified, we started creating many supporting resources. In this case, we developed training for program leads and a training guide for our client who would be delivering the training. We also wrote a comprehensive reporting guide, where we included our newly defined terms, examples and explicit instructions. These resources ensured that all staff responsible for measurement were equipped with the knowledge and tools needed to successfully implement and maintain the measurement system. Our client didn’t have a need for a dashboard or reporting template, but those would have been other valuable resources.
So, there’s our step-by-step guide to building a shared measurement system. Of course, the specific activities included are dependent on the needs of each case. We found particular value in the current state analysis, field testing and developing that deep familiarity with the client’s strategic documents. Using a participatory approach ensured that the measurement system was not only effective but that the programs were eager to begin using it.