Demonstrating the impact of innovation and systems change approaches is a major challenge.

Measuring impact is made particularly complicated by the very nature of systems change.  An organization that aims at systems change is looking to shift society.  A societal shift is harder to count than, for instance, an increase in existing services.  Further, systems change requires that the societal shifts stick and can resist the inevitable push back from “affected” sectors.

The existence of a solid foundation of societal support, along with strong democratic institutions, are required to make the change real and long-lasting.  The trick then is in how to show success and even how to show the steps along the way.

“Evaluation is a high priority when program outcomes are difficult to observe and knowledge is lacking about how best to achieve results—such as when we collaborate with partners who are working to improve service delivery or effect behavioral change, identify, replicate, or scale innovative models, or catalyze change in systems, policies, or institutions.” The Bill and Melinda Gates Foundation.

Measuring gets even more complicated when organizations work collaboratively, with multiple actors and multiple constituencies.  In collaborations, organizations might be working on separate components of the system and thus need different indicators directly related to their individual work. Then, in addition, the collaboration as a whole needs higher-level indicators of success related to the system itself.

We work with our clients to determine what the best combination of indicators and methodologies would be to satisfy the need for accountability of each of the members of a systems change coalition.  We also guide the process to develop the broader indicators that will determine whether the combined efforts are coming together in the right recipe for the intended, positive impact. We recommend avoiding a one-size-fits-all approach to evaluating impact.

To arrive at these indicators, we often find that the best measures emerge from a planning process that builds off the methodology of outcomes mapping.  An outcomes map should yield a combined dashboard that connects and presents the objectives, actions, and resources in one place, differentiating between outputs and outcomes.  The co-creation process itself is as important as the outcomes maps that result because the results are then owned by the relevant actors – including the organization, partner organizations, and broader stakeholders.  These outcomes maps must then be revisited regularly to adjust for the rapid changes that society is experiencing and the myriad of factors that one cannot control or foresee from the start.

All these components included in the process form a comprehensive picture of what success means for the organization.

“Data and evidence are critical to creating feedback mechanisms in systems. When used with maximum effectiveness, monitoring and evaluation become part of a process of continuous improvement, rather than a one-time or annual event. Systems can regulate themselves thanks to feedback loops, in which timely information is in the hands of decision-makers; the latter can thus push forward when more momentum is needed or pull back when a system is in overdrive. Most importantly, systems entrepreneurs can assess the changing environment, identify unintended consequences in real-time, and alter the course if necessary.” Beyond Organizational Scale: How Social Entrepreneurs Create Systems Change, World Economic Forum.