Reflexivity in Evaluation
This article is rated as:
Reflectivity vs Reflexivity
Reflective Practice is where a person reflects on what they have learned and how they can apply it or learn from it.
Reflexive Practice is where a person reflects on what they have learned and considers how the implications of their learnings can impact the broader context they work in. The scope to which the person applies their learnings is broader in reflexive practice.
Reflective practice is the first competency domain for the Canadian Evaluation Society (CES) Credentialed Evaluator (CE) Designation. It is a part of the foundation that makes up a sound evaluation practice. Reflective practice involves using a learning mindset to stay up to date on new and best practices, integrating evaluation standards and ethics into practice, providing transparency and a balanced perspective, contributing to the profession of evaluation and using self-awareness and reflective thinking to continually improve practice. This last point is reflexivity, or continually reflecting on how oneself and one’s learnings impact the broader context within which a person works.
While most of us generally understand the concept of reflectivity and how to be reflective (learn, stay up to date, apply professional standards and ethics), this last concept of reflexivity can be elusive. Reflexivity often falls into that category of things we know we should be doing, but aren't really doing, or things we might be doing, but aren't so sure we're doing right.
To try and ease your stress about reflexivity and to help guide your practice, we've compiled our best tips below. To start, we define reflexivity and outline why you should develop a reflexive practice before jumping in to help you become more reflexive.
What is reflexivity
What really is reflexive practice and how do you do it in a practical manner? If we look to the CES Credentialed Evaluator domain, reflective practice is about having a deep understanding of evaluative theory and practice, applying evaluation standards and ethics, and having an awareness of self and reflection on one's practice. In essence, it's about the cycle of learning and growth, both about the field of evaluation and yourself as an evaluator, and using critical insights to improve your practice. Reflexivity can take on many forms, but it is essentially the practice of examining ones' self as an evaluator, how you have been shaped by the evaluative process and how your values and viewpoints have shaped your evaluations.
Why develop a reflexive practice
You might be wondering why you should be concerned with developing a reflexive practice. In evaluation, we are often tasked with defining or providing information for decision-making about the value or merit of an evaluand (e.g. a program or project). We must pay attention to the needs of different stakeholders, outside political influences, and our own biases. Reflexivity gives us the space to process these elements and critically examine these influences on our evaluations. How can you measure value if you are not aware of your own values? Through reflexivity, we learn and grow from our mistakes.
How to be reflexive
Reflexivity sounds like a daunting task. How does one regularly and thoroughly critically appraise ones' self and evaluation practice? In today's busy world where we are being pulled in many directions, here are some strategies to help you develop your own practical reflexive practice.
1. Be reflexive often.
Reflexivity should be a continuous process. To ensure that it doesn't fall to the wayside, carve out time in your calendar and stick to it. Figure out what works best for you — do you prefer to work in larger chunks of time, or to split up your reflexion over the space of a few days or weeks? Being reflexive is something that needs to fit within your current work practices. If you are always rushing out of the office on a Friday afternoon, don't schedule your reflexive time then.
In addition to scheduling reflexive time regularly, include it as part of your evaluation plans. Ensure you include time at the end of every major evaluation phase for some project-specific reflexion time. This practice can help you implement improvements in your current evaluation project and save you from future pitfalls in similar projects. A quick check-in about what you did well, what you could improve on, and what changes you will make with this information counts as reflexion.
Practicing reflexivity in the space between data collection, analysis, and reporting can provide insights into the interpretation of results. If survey response was low, what might it indicate? Were your survey deployment tactics suited to the population? Did your questions resonate with the community? Did you make assumptions about how people would respond to or interpret the questions? How could these assumptions have impacted your results?
2. Be reflexive in a structured manner.
Left to your own devices and without a plan, you can easily use up your whole reflexion time googling how to be reflexive (if that Google search led you to this article, Hello! And welcome to the end of your search.) Set out your questions or focus in advance and stick to them. There are some practical tools to help with this in the tools section at the end of this article.
Bring structure into your reflective practice in a way that makes sense for you. Perhaps it's a weekly set of questions, free drawing time with a focus or intention in mind, or a daily project journal.
3. Be reflexive alone.
Reflexion is about reflecting on your own processes, questioning your attitudes, thought processes, values, assumptions and habitual actions in order to understand our roles in complex situations. Nobody else can do this work for you. Ensure that some of your reflexion is done alone.
4. Be reflexive together.
While you need to be able to think critically about yourself, bringing others into your reflexive practice from time-to-time can help you gain a deeper understanding. Getting feedback from others can challenge your assumptions about yourself. Your coworkers, clients, and colleagues are sources of information to promote learning and growth. Offer to take them out for a coffee, meet up for a walk, or schedule an informal phone call.
If you work as part of a team, consider bringing everyone together for a project debrief. Ask them what worked well in the project, what external facilitators and barriers contributed to your final product, and what internal processes could be improved for next time. You can also examine how you worked together as a team and if there are areas where the team can grow.
5. Record it!
Part of reflexion is looking back at your growth. Recording your thoughts will help you to look back and see patterns over time. It also helps you to be accountable. Find a method of recording your reflexion that works for you. Keep it simple and don't overthink it.
6. Get meta about reflexion.
Don't do this too often, but every once in a while it doesn't hurt to be reflexive about your reflexion. How are your chosen processes and tools working? Are you actually making the time to be reflexive on a regular basis? Are there questions you are avoiding? Have you implemented any of the steps you outlined to make improvements?
Tools and Ideas for Reflexion
At the end of an evaluation, once you have reported on the findings, go back to the data collection tools you use and consider what changes you would make in hindsight. If your evaluation included audio-recorded interviews or focus groups, go back and listen to them again, this time focusing on yourself. How did your involvement in the data collection impact the participants and your interpretations of the findings?
If you are looking for a structured process for reflexion, check out the DATA model:
Describe what is or has been happening in practice.
Analyze the current state of practice; why is this happening in this way?
Theorize why things are occurring.
Act; make a specific action plan to make changes.
Feeling more creative? Carolyn Camman created an adult colouring book with reflexive questions to get your mind going.
Do something with your hands that you are good at but doesn't required a lot of concentration. Doodle, bake, work on a puzzle, or play with playdough while you mull over your reflexive questions. Personally, I've baked dozens of cookies and a couple of pies as a way to carve out time and keep my hands busy while giving my brain space to think.