Today’s Dr.’s in session was led by Kentaro Toyama, UC Berkeley researcher, on “Impact Evaluation for International Development”. Having recently done some data collection for a Global Washington members’ collective impact report, I attended the event as both a Global WA volunteer and as an interested party looking to get more information on the everyday strengths and weaknesses of current methods of evaluation used by global development organizations.
The session, which was the first part of a two-part series, used anonymized real-life examples to demonstrate the strengths and weaknesses of the average impact statement. Toyama used these examples to generate ideas from attendees about the criteria for impact evaluation to produce a credible, concrete assessment of development outcomes attributable to a particular intervention. Many of these criteria highlighted the importance of the current “Gold Standard” of impact evaluation tools, the randomized controlled trial (RCT). Toyama emphasized the importance of this method and its limitations as well as common misconceptions held about it.
In my data collection for Global Washington mentioned above, many members stressed frustration with measuring impact of their programs as they feel that so many of their outcomes are intangible benefits that are hard to quantify and therefore very difficult to measure. The member attendees at today’s session expressed similar concerns around measuring leadership skills, increase in dignity, and empowerment. Toyama agreed with the limitations of current impact evaluation in this area but stressed innovative studies that academics are producing to advance this aspect of the difficult task of measuring impact.
In next week’s session, Toyama will delve more deeply into RCTs as well as the importance of additional or alternative methods of evaluation. Based on the requests of today’s attendees, he plans to address specific situations and cost effective methods of evaluating impact. He will also be using a case study from his own work to highlight his talking points.