In defence of long reports.
This article is rated as:
116, 78, 54, 47 – These are not my golf scores, or my typing speed, or even the number of CDs I owned in the 90s.
These are my report lengths, in pages.
My name is Bonnie and I write long reports.
There I said it. It’s out there. Are your eyes wide? Jaw on the floor? That’s ok. Actually, I’m not all that ashamed. I may be in the minority here, but I think long reports have a place.
Stephanie Evergreen, Ann Emery, and Chris Lysy are all people in the evaluation and reporting field that I follow, respect, and learn from. They are also all people who offer warnings about long reports. Each of these three gurus shares the familiar song about having three versions of a report, usually something like 1 – 3 – 25: here’s Stephanie’s take on it, Ann’s is here and Chris’ here. We even have our own version here at Eval Academy! The idea is that 25 – 30 pages should suffice for your “final report”. This page limit should be enough to get your findings across in a way that remains accessible to your audience(s). A three-page version is for those higher-ups, or peripherally engaged partners who just want the key takeaways, and perhaps aren’t tasked with key decision-making based on findings. And finally, the one-page version may be for the public or to post on a website.
So how do I defend my long page counts? In a few ways:
1. If you don’t write it down, it will get lost. Chris Lysy from Fresh Spectrum has written a lot about reporting. I am a subscriber to his thoughts that these long reports aren’t reports per se but perhaps “documentation”, a place to document the nitty gritty details: What did we do? What did we measure? What did we learn? The purpose of a document, rather than a report, is to build organizational memory or create a historical record. Whatever you want to call it, if you don’t write it down, it will get lost. I’ve definitely had times where I’ve gone into the archives of past projects to recall just how I phrased that question, etc. Long reports(documents) do have value by housing all of this information in one place. Here are some of Chris’ articles sharing more details on the distinction between the two, that I won’t plagiarize here: The two types of report. (freshspectrum.com) In defence of the 200-page report (freshspectrum.com).
2. Design matters. We’ve written a lot at Eval Academy about how to create quality reports. We walk the walk too. I employ lots of principles to make sure each of my 100+ pages are high quality. This includes:
a. Use of white space
b. Use of visuals: images, charts
c. Page breaks
d. Title pages
e. Use of colour to link (or separate) ideas
So yes, my 100+ page report could probably be crammed into 70% of the space but it wouldn’t be fun to read. (I hear you Chris Lysy – see cartoon), but I offer you this: if the report is not a report but a document for knowledge preservation, and your project manager, at the very least, IS going to read it, then good design goes a long way.
3. There’s always the executive summary. I dedicate a fair bit of time to crafting a quality executive summary. It varies in length too, usually from one to five pages. Often, I’ll set the (long) report/document aside for some time, then return to it and try to read it through with fresh-ish eyes while I take notes on what the key takeaways are in each section. I ensure that each point is covered clearly and with appropriate context in the executive summary.
4. I use appendices. Appendices tell a reader “Hey, here’s more information if you’re super keen on this, but we think you can gain the key insights without this additional detail.” I use appendices to give details about survey respondents, response rates, methodologies, frameworks, or approaches. They add length to a report, for sure, but some readers will get to the appendices and feel a little relief that the body of the report was actually much shorter than expected! (Bonus!) Side note: I think most people don’t count appendices in overall report length, but I do, because when a reader opens the document, the first thing they’re going to see is the total page count. They don’t know how many pages are body and how many are appendices.
5. I do actually follow the 1-3-25 method. Kind of. The 100+ page report isn’t the only deliverable I give a client. I’ll pull out the executive summary into a more shareable PDF, and sometimes I’ll create another version that aggregates the summaries of each section, so it gives just a little more detail than an executive summary does, without all the detail. So, in my 116-page example (yes, that’s real), I had a 5-page executive summary, and a 25-page version that included background, summaries for each outcome, recommendations, and limitations. I also always offer to give presentations. No matter how long or how well-designed a report is, some people will just want to hear what was found. This is also your opportunity to facilitate discussions about “now what?” – what can or should be actioned? So, I do subscribe to the many-version method, it’s just that my way is (1 – 5)-25-50+ (or however long it needs to be to get it all down).
As the writer of these long reports, I’ve found added benefit that ultimately makes for a higher-quality evaluation for the client. Long reports help me to reflect on all the data and all the findings – I am better able to make connections and triangulate data when I aim for comprehensiveness. This is especially true if a team has been working on the evaluation; the writer gets the benefit of diving into all of the collected data. As I’m writing up all the results, I’ll reflect on what that means, is there a recommendation to come from it, is this a surprise finding, is this something we want to highlight, etc. Instead, if I aim for brevity, and, as an example, distill our 15 participant interviews into a singular primary theme that “participants were satisfied” it loses value, it loses context and loses the opportunity to learn and do better. Writing the report gives me an extra opportunity to think about the data and the evaluation.
Now, I am a self-professed more-is-more person. I like to read all the details on a subject to be well informed to make a decision. I’m that person searching 30 hotel reviews for my two-night weekend away. I’m that person who gets emailed an 80-page report and carves out time in my calendar to read it. I know there are others like me out there. But, if you’re a less-is-more person, I’ve got you covered there too, with shorter versions of the report, or presentations. I’m not saying that these long reports should be our only report, I’m just trying to defend their place in the world.
Long reports can also serve other roles. Detailed reporting is one way to build capacity. Perhaps a novice evaluator will pick it up and learn about how to implement a certain framework or other evaluators will gain some knowledge about how to ask about demographic diversity, or perhaps other project managers or evaluators will learn what not to do!
Long reports also honour transparency and accountability. There’s no hiding holes in methodology when you detail everything you did in your appendices. There’s no hiding the small sample sizes, poor response rates or missed audiences when you have limited sections or include data tables. There’s no hiding how well each evaluation question or outcome is (or isn’t) addressed.
So, if I have you convinced that long reports have value, here’s a couple of tips if you’re starting down this garden path:
1. It’s easy to get into information overload. Practice plain language wherever possible. Keep sentences short and cut out redundant phrases. Keep your key messages and interpretations front and centre. Long reports are not a license for poor writing or unnecessarily long narrative. It is even more important to be concise, stick within the scope of the evaluation and avoid tangential or meandering content.
2. Plan it out. Ultimately long reports cost money in paying for your time. Don’t get caught without a plan. Maybe you’re following a specific evaluation framework, or plan to set up your report by evaluation question or outcome. Whatever you plan to do, map it out. I’ll often take one of my kids’ pieces of construction paper, a good ol’ Sharpie and map out all the data I have to sections that I envision in the report, so I know ahead of time what goes where. I find this planning step efficient and cost-effective.
3. Know your audience. Will a 100-page report catch them off guard? Do they have the capacity to engage with it? Whenever I know that a report is going to be long, I begin to prepare my audience, even to the point of asking them to have dedicated time set aside for review so as to not derail any timelines. Still, not everyone will read it. It is ok to ask certain people to review only certain sections.
4. Make it easy to review. True for any report, but particularly the long ones, feedback from multiple reviewers can put you in a nightmare world of version control. I’m a big fan of using hyperlinks, not attachments, to mitigate version control disasters. It also helps to set clear deadlines for when you need reviews complete.
So, is anyone brave enough to share their longest report lengths? I’m curious to learn if I’m alone out there in the 100+ page club. I hope I’m not coming across as proud of this, but I’m also not ashamed of it. Projects, and therefore, evaluations are becoming more complex. I am currently working on two projects that are essentially three in one: three mini projects under a larger umbrella. These, I anticipate, will not be short reports. So, I am an advocate of building repositories of knowledge, and of doing reports “for posterity’s sake” but that doesn’t mean they have to be long and boring. Quality reporting is still quality reporting.