Evaluation Roundup - March 2020
New and Noteworthy
The new and noteworthy reads for March 2020 are focused on the new and noteworthy virus – COVID-19. “Noteworthy” is an understatement that does not describe the gravity this virus has and will continue to have on our lives for years to come. As such, here are the latest evaluation resources focused on COVID-19 that we hope will help guide your evaluation practice.
Evaluation’s Yoda, Michael Quinn Patton, shows us the way in a pandemic
Leave it to MQP (@MQuinnP) to be one of the first to show us how this virus is/will impact our evaluation practice. Check out his blog post, Evaluation Implications of the Coronavirus Global Health Pandemic Emergency where he lays out, in a simple and engaging way, 15 implications we need to keep in mind. My favourite is to “Be Proactive”. It seems the tendency right now is to sit, wait and give our clients the time and space they need to deal with this crisis. While I agree evaluation is not top of mind for people right now, MQP hits the nail on the head when he says:
“Don’t wait for them to contact you. Evaluation is the last thing on the minds of people who aren’t evaluators. They won’t be thinking about how the crisis affects evaluations. That’s your job as an evaluator. Get to work doing that job. Adjustments need to be made now, sooner rather than later. Offer help in updating your evaluation. This doesn’t necessarily mean delaying data collection. It may mean accelerating it to get up-to-date information about the effects of the crisis.”
Of course, he also talks a lot about principles relating to his Developmental and Blue Marble evaluation work and how it applies in this situation. In fact, Blue Marble Evaluation held a webinar titled, Blue Marble Evaluation: Evaluation Criteria for Transformation, that discusses evaluating transformation and six criteria for evaluating it. If that is a bit too much for your brain to handle right now then he has also put a video on YouTube for “evaluators’ pandemic comfort music” for you to sing along to.
UNDP Independent Evaluation Office: Six tips for evaluation during crisis
The IEO of UNDP (@UNDP_Evaluation) recently produced and shared a one-page infographic that highlights six tips on evaluating programmes during crises. Like MQP, their first tip is to “rethink evaluation plans” with a focus on getting data to stakeholders as soon as possible. Of course, many of the tips talk about working remotely, be it working as a team or collecting data remotely. They recommend Skype interviews, mobile questionnaires, online surveys, collaboration platforms (Slack and Yammer) and satellite imagery to gather data.
Doing fieldwork in a pandemic
If you really want to dig into collecting date remotely, you need to check out Deborah Lupton’s crowd-sourced Google Doc, Doing Fieldwork in a Pandemic. The document was originally created to turn traditional fieldwork (i.e. face-to-face) methods into a more ‘hands-off’ mode; since, many people have added numerous resources.
Deborah Luton also wrote a blog recently titled Social Research for a COVID and Post-COVID World: An Initial Agenda. In this blog post she talks about her experience doing social research on the HIV/AIDS pandemic and the obesity epidemic; she shares insights on similarities and differences with the current COVID crisis. From there she proposes a list of key social research questions that will generate a better understanding of the current and future impacts of COVID.
Evaluation in a Crisis
Chris Lysy of Fresh Spectrum (@clysy) is an evaluator and evaluation cartoonist and blogger. Recently he wrote about The Evaluation Mindset: Evaluation in a Crisis. Like MQP, Chris talks about the important role evaluators play ensuring decision makers have access to timely information. As he says, “evaluation is an anytime activity”.
Checking in with evaluators
Carolyn Camman (@c_camman) and Brian Hoessler’s (@StrongRoots_SK) produced a podcast (Episode 33: Eval Café Check-in) where they asked sixteen evaluators from around the world (including our own Shelby Corley) how the COVID-19 pandemic is affecting them, what they want people to know, and what is giving them hope during this time. The results – it turns out evaluators are human and not just methodological robots! One of the big themes is how this crisis has brought people together (not physically of course) and ignited our need for connection. The guest evaluators talked about how people are reaching out more now than before when social distancing wasn't the norm. Of course, like all good evaluators they talked about learning and what learning would come from this crisis. They wondered if the increased emphasis on human connection would continue and hoped that it has given us pause to re-assess our values and priorities going forward. Some also talked about the implications for how we work as evaluators and questioned if the new remote ways of working will push us to continue down that path in the future.
Tips for virtual gatherings
Speaking of remote ways of working, evaluators are adapting and incorporating virtual gatherings into how we do our work. For ideas on how best to do that, refer to Elizabeth Diluzio and Laura Zatlin’s AEA blog post Tips + Resources for Virtual Gatherings During This Global Pandemic.
The Spinoff cartoonist is a data wiz with data viz
There have been lots of charts flying around the Internet trying to tell the COVID pandemic story. My favourite data viz so far was posted on a New Zealand online magazine called The Spinoff; it was illustrated by their cartoonist Toby Morris. It is an animated graph that shows how individual discipline really can stop transmission. It is so good I sent it to my baby boomer parents who seem to think staying home does not apply to them.
