This article is rated as:

Hey Program Managers, I wonder if you’ve ever needed to decide how to evaluate the programs or services you offer; not just the in-the-weeds details about how to write the evaluation plan and measure the outcomes, but who should be helping you conduct that evaluation.

I work for an evaluation consultancy firm, Three Hive, so I should be espousing the benefits of contracted evaluation. But actually, I think there are some key advantages that come with having an internal evaluator. I’ve been fortunate in my career to have experienced a few different work formats:

• I’ve been a full-time employee evaluator for two different healthcare organizations

• I have been a solo contractor working for multiple clients

• I have been a solo contractor, contracted full-time to a single organization for multiple years

• Now, I work as a consultant for an evaluation firm, as an employee, contracted out to multiple clients.

Let me share some of my insights about the pros and cons of an internal evaluator vs contracted/external evaluation.

Internal evaluators are those who are part of the organization that is implementing or funding the program being evaluated. Usually, this is as an employee, but I’d count my years as a full-time contracted evaluator for one organization as internal as well. External evaluators are those who are independent from the organization being evaluated. External evaluators are usually hired for a specific project, but less commonly could be retainer-style, available to add evaluation expertise at any time to any project.

Pros and Cons of an Internal Evaluator

In my experience, one of the main benefits of being internal is sitting at the table when a project is being designed. Being included right from the start means that evaluation can be integrated into implementation and can inform your project leads along the way. Being included early as an external evaluator can happen, but it’s a rare treat.

On the other hand, as an internal evaluator, I was constantly being asked to take on non-evaluation projects, including leading full projects as the Program Manager, not the evaluator. I also felt a little lonely as the solo evaluator with the company and struggled to advocate for professional development opportunities related specifically to evaluation.

Pros and Cons of an External Evaluator

In my experience one of the biggest benefits of an external evaluator is finding that perfect match – not only someone who is an expert evaluator, but someone who knows your work area, which could be healthcare, community programming, housing, justice, etc. Chances are there’s an evaluator out there that fits your needs! As a contracted evaluator, I value bringing an outside perspective and asking the silly questions.

On the other hand, one of my pet peeves is that notion of “not knowing what you don’t know”.  It can take some time to get to know an organization and to ask the right questions to get you the data and information you need.

How to Choose the Best Option for Your Evaluation Needs

There is no definitive answer to the question of whether to use an internal or external evaluator. The choice depends on several factors, such as:

• Your evaluation needs: Do you operate several programs/grants that require evaluation? Or did you just secure a one-time-limited grant with an evaluation requirement?  Think about how valuable an outside perspective might be or the sensitivity around the project. Conversely, think about workload and where an evaluator could help operationally if projects are not numerous.

• Organizational capacity: Everything from physical space to supervision; how difficult/easy would it be to integrate a new full-time role into your operations?

• Budget! Of course, it goes without saying that both internal and external evaluators cost money. Think about total costs: internal evaluators may have pensions, benefits, vacation, sick time, equipment and software costs to factor in. Contracted evaluators may charge more per hour, but contracts can put hard limits on the budget available.

• The skills needed: Maybe you need an evaluator who knows your field well; maybe a Credentialed Evaluator designation is a requirement; or maybe you need someone with the right networks and connections! It may be a matter of finding the right person (or team) and working on the work arrangement after the fact.

• The “optics” of the project: Often external evaluators are hired when the outcomes of a project may be political or sensitive. That is, when there will be scrutiny on the results, high levels of interest, or when you need to able to ensure reduced bias in your findings and reporting. Consider the ethical and political implications and sensitivities of the evaluation.

Your choice doesn’t necessarily need to be a permanent one. Maybe you start with a contracted evaluator for a specific project but learn over time that you could use evaluation expertise in more areas; you can always look to hire someone or increase the contract with your evaluator.

Internal and external aren’t the only options either. Often, we see someone who has been given the side-of-desk additional responsibility of being the organization’s evaluator. I don’t think this is anyone’s ideal, but we do recognize that sometimes time or budgetary constraints make the less-than-ideal the only option, which is literally why we created our course!  Of course, you can hire part-time, which may mitigate some of the disadvantages of an internal evaluator, or you could look at hybrid models: an internal evaluator supported through external consultants.

If you’re in the market to hire an evaluator, consider your options carefully. A quality evaluator and a quality evaluation can have a significant influence on funding and even operational efficiencies.