Using AI to do an Environmental Scan

This article is rated as:

As we all know by now, using AI (artificial intelligence) tools can greatly enhance your productivity. However, the gains from using AI can only be realized if you can fit it into your workflows. But how do you do that?

This article is going to give you some tips on how to integrate AI tools (specifically ChatGPT, Google Gemini, or Claude 3.5) into the steps you would take to complete an environmental scan. To keep things simple, we're going to modify the steps for completing an environmental scan from another Eval Academy article - How to complete an environmental scan: avoiding the rabbit holes. That article has detailed instructions for completing an environmental scan the conventional way.

Note: In this authors opinion, it is always good to know the slow(er) way of doing things before trying to take any shortcuts. The tips in this guide are suggestions for enhancing your existing workflows.

But first, what is an environmental scan?

“An environmental scan (a.k.a. e-scan) is a tool for collecting information about factors that could affect the future of an organization. It is used to learn about current conditions (social, economic, technological, and political) and plan accordingly for the future.”

– Eval Academy Dictionary, Environmental scan

Why would I want to use AI to do my environmental scan?

In my opinion, environmental scans can be very tedious. They involve time-consuming searching, reading, and systematic cataloguing. Using AI can shorten the amount of time it takes to do all the tedious work involved in an environmental scan! Furthermore, AI tools like  ChatGPT, Google Gemini, or Claude 3.5 are based on large language models (LLMs). LLMs are known to be very useful when you are trying to brainstorm and explore ideas. And I don't know about you, but for me, using AI makes the whole experience of doing an environmental scan a little bit more interesting.

Quick Guide to Using AI in an Environmental Scan

Don’t forget to follow along with our step-by-step guide on How to complete an environmental scan: avoiding the rabbit holes. This article expands on how to integrate AI into those existing steps.

Step one: identify the topics of interest and the purpose of the environmental scan

If you know that you need to do an environmental scan, you probably have a topic in mind. However, you can also use AI to brainstorm and refine your topic ideas. Like I mentioned earlier in this article, AI tools are based on LLMs. They’re known to be great at brainstorming and can help you explore ideas you haven’t even considered.

An example prompt to brainstorm a topic could be: I need to conduct an environmental scan for a project about [idea]. What are some possible topic ideas?

Step two: identify the research question(s)

Sometimes, it isn’t easy to come up with a clear research question. You could call a friend… or you can ask an AI. If you tell the AI tool your topic and ask it to generate some research questions, it can give you a starting point.

An example prompt could be: I am doing an environmental scan on the topic of [topic]. What are some research questions I should be asking?

I asked ChatGPT, “I am doing an environmental scan on the topic of how collective impact initiatives set up measurement systems. What are some research questions I should be asking?” It generated 9 categories of questions. The first two questions were:

• What are the common frameworks or models used for measurement in collective impact initiatives?

• How do collective impact initiatives define and agree upon shared outcomes and indicators?

Not bad, right? I wouldn’t use all the questions ChatGPT generated for me. But now I can filter through the ones it suggested and refine my research questions for the environmental scan.

Step three: identify what environmental scan activities you will complete and where you will look for the information

AND

Step four: for online searches, you'll need to create a list of keywords and search terms that you will use

During steps three and four of the environmental scan, we're going to level up our usage of the AI. In these stages, you can ask the AI to generate a search strategy. In your prompt to generate a search strategy, you can include as many details as you'd like. Anything that you would like the AI to help you generate can be added such as keywords and search terms using Boolean operators,  potential activities, and sources where you could find relevant information.

The search strategy that the AI generates will be more useful if it has more background information on your environmental scan. You can feed this information to the AI manually by giving a prompt that says to generate a search strategy based on X information. If you have been using the AI to brainstorm for your environmental scan up until now and ask it to generate a search strategy using the same conversation thread, that information will also be taken into account. The more information and details you can give the AI in your prompt and thread history, the more likely it is that you’ll be able to generate something usable. Just make sure that you do not feed any sensitive information to the AI, such as personal identifying information or health information.

An example prompt to generate a search strategy could be: Generate a search strategy for an environmental scan on [topic]. The environmental scan should answer the following research question(s): [research questions].

Include potential sources such as Academic databases, websites, publicly available reports, and other similar sources. Also include a list of keywords and search terms using Boolean operators.

[OPTIONAL] Use the information pasted below as background information for generating the search strategy: [additional context information].

I tried using Google Gemini for this prompt, just to try something different: “Generate a search strategy…. on how collective impact initiatives set up measurement systems. The environmental scan… research question(s): a) What are the common frameworks or models used for measurement in collective impact initiatives? and b) How do collective impact initiatives define and agree upon shared outcomes and indicators?” I also attached an image of a chart with sample outcomes and indicators for measuring collective impact as additional context information.

I won’t share the whole search strategy that Google Gemini generated, but I’ll highlight some key parts that I found to be useful. For example, under the heading of “Potential Sources,” it suggested specific websites I could search – and all the suggested sites are known to be thought leaders on the topic of collective impact.

Google Gemini also suggested some potential keywords and search queries using Boolean operators, as I had requested.

You may not get the perfect search strategy on the first try, but the nice thing about these AI tools is that you can continue to refine your searches by giving it additional editing prompts.

Step five: catalogue the information systematically

Once you found your sources, you need to catalogue your information systematically. I would recommend using a table like this Environmental Scan Template in Excel to record your information. You should be able to come up with the categories yourself because they will be determined by what you need to know from the environmental scan. But if you really do get stuck you can always use the AI.

This step is where you will probably see the most amount of time saved by using an AI. Simply upload all your sources into the AI and ask it to catalogue the information according to your categories, preferably in a table format. I like using Claude 3.5 for this step because it can read PDF files and process many sources. However, other AI tools may have similar capabilities depending on the version that is available to you. Again, please do not upload any files that have sensitive information. If your source is not already in an upload file format (e.g., website text), you can just copy and paste the information that you need into an empty Word document and upload that instead.

An example prompt to catalogue the information systematically could look like:
Scan the attached documents and sort the contents into the following categories using a table format that also lists the corresponding source document: [list categories].

[OPTIONAL] Use the following definitions for each of the categories.

[category 1]: [definition]
[category 2]: [definition]

For my own environmental scan on collective impact, I used the following prompt on Claude 3.5: “Scan the… source document: Framework and Models, Shared Outcomes and Indicators, Challenges.

Use the following definitions for each of the categories.
Framework and Models: Theoretical or practical approaches used for measurement in collective impact initiatives.
Shared Outcomes and Indicators: Processes for defining, agreeing upon, and measuring shared goals and outcomes in collective impact initiatives.
Challenges: Common challenges faced in measurement and evaluation of collective impact initiatives.”

I also attached a couple articles on the topic of collective impact and measurement for this example. Depending on the type of AI tool you’re using, whether it’s a free version or a paid version, and the current volume of usage by other people, you may not be able to attach all the articles you want in one prompt. You’ll have to batch them and build on the information in the generated table.

The resulting analysis was a simple, scrollable table with some key points from each article.

After the AI has sorted all the information for you, you should still verify it. This could look like reading all the sources yourself to make sure that it has been categorized properly. This would still take less time than systematically cataloguing the information yourself. Once you verify the information, you can copy paste the AI generated information into an offline Excel table or text document, so that you’ll be able to edit the content as needed.

You can also conduct further analysis with the AI. This could look like asking about any themes that it sees in the information that it has catalogued. You could likewise ask specific questions that you might have about the information. This type of AI analysis would only work if you continued to use the same conversation thread because the AI would already have all the context it needs.

An example prompt to conduct further analysis could look like:
Highlight any key themes found across all documents that would answer the research question(s): [research question(s)].

Using the same conversation thread in Claude 3.5 where I generated my environmental scan table, I entered another prompt about highlighting key themes using the research questions I generated earlier. I said, “Highlight… research question(s): a) What are the common frameworks or models used for measurement in collective impact initiatives? and b) How do collective impact initiatives define and agree upon shared outcomes and indicators?”

The resulting analysis was interesting and relevant, even though it was based on only a couple articles. I would be able to use this to sift through the environmental scan table and delve deeper into my own conclusions.

If you chose to categorize your information manually, you could try copy pasting some of your findings into an AI tool as context if you want to use it to do further analysis. However, you’ll need to enter the findings as text only. As of this writing, the AI tools mentioned in this article struggle with reading Excel files. I am optimistic that this will change in the future with how fast things are developing.

Step six: present the information in a way that is useful to your organization

If you can continue to use the same thread that you used to analyze the information, you can ask the AI to generate a report outline based on that context. Even if you were not using the same thread, you can still ask the AI to generate a report outline for your environmental scan and upload any findings that would give it enough context to generate the outline.

An example prompt could look like: Generate a report outline for an environmental scan based on the previously generated table and key themes. The outline should have no more than [number] headings and is targeted towards [audience]. The main focus of the report should be [main focus].

[OPTIONAL] I want the report to highlight the following findings: [list of key findings].

I continued to use the same conversation thread in Claude 3.5 so that I wouldn’t have to give any additional background information in my prompt. I said, “Generate… The outline should have no more than 5 headings and is targeted towards leaders in the non-profit sector who are part of a collective impact for adult basic education. The main focus… should be making a case for developing a shared measurement strategy for their collective impact.”

I won’t share the entire outline, but I included a screenshot of the first few that were generated. It’s not perfect, but it’s decent for 2 seconds of work.

Other Notes to Consider

If you are conducting a literature review, you can use the same strategies in this article to integrate AI into your workflow. For both a literature review and an environmental scan, make sure you credit the use of the AI tools that you used. You can include it in the method section of your report.

Example credit: This report was analysed with the assistance of OpenAI’s ChatGPT. ChatGPT is a large language model that was used to generate the initial search strategy, initial themes, and an outline for this report.

If you found this information useful, please leave us a comment to let us know!