Stratified Random Sampling in Evaluation
This article is rated as:
In our evaluations, we use varying methods to collect random, representative samples. In most instances, collecting data on all members of a population isn’t feasible (i.e., too expensive and time intensive). Therefore, we rely on sampling methods to make generalizations about our population of interest while minimizing bias.
Random sampling is one of the simplest sampling methodologies used in evaluations. Random sampling treats all members of the population equally; thus, everyone has an equal chance of being sampled. That is, we randomly sample a specified number of individuals from the overall population. For quantitative data collection, see our “easy” and “hard” guides for finding the right sample size.
However, often our evaluations are interested in differences between population characteristics (e.g., gender or ethnicity). While a random sample of sufficient sample size would likely capture individuals falling within the varying levels of these characteristics, it is not guaranteed that each level of these characteristics is sampled. In these cases, we would employ stratified random sampling.
What is Stratified Random Sampling?
Stratified random sampling is a sampling methodology used to capture a representative cross-section of a population. Rather than randomly selecting from a pool of all members of a population (as in random sampling), stratified sampling divides the population of interest into distinct subgroups or strata based on designated characteristics. With the population stratified, a random sample is taken from each of the stratum. This ensures that each subgroup is adequately represented in the final sample.
Types of Stratified Sampling
Stratified random sampling can be split into two variations: (1) Proportionate stratified sampling and (2) Disproportionate stratified sampling.
(1) Proportionate stratified sampling: the size of each sample drawn from each stratum is proportionate to the size of each stratum in the population of interest.
Example
We want a proportionate stratified sample based on participant age group (youth, adult, and senior). Knowing that our population has 40% youth, 50% adult, and 10% senior participants, our stratified sample should reflect these proportions. That is, if we sample 100 individuals, the sample should contain 40 youth, 50 adult, and 10 senior participants.
(2) Disproportionate stratified sampling: the size of each sample drawn from each stratum is not proportionateto the size of each stratum in the population of interest.
Example
Our evaluation wants to better understand Indigenous perspectives related to a given program. However, Indigenous participants are underrepresented within the program accounting for only 20% of all participants. Therefore, to get a better understanding of Indigenous perspectives, if we sample 100 individuals, the sample could contain 50 Indigenous participants and 50 non-Indigenous participants.
Choosing between proportionate and disproportionate stratified sampling depends on the evaluation and the importance of each stratum. Proportionate sampling is effective when we want to maintain the proportionality and representativeness of our population. On the other hand, disproportionate sampling may be more appropriate when certain strata require more in-depth evaluation, particularly for individuals within underrepresented strata.
*Disproportionate stratified sampling may vary depending on the evaluation question. In this example, participants 26 and older are more relevant for the evaluation. Thus, these age groups have larger sample sizes relative to younger age groups, regardless of the actual proportion of each age group within the population as a whole.
Why use Stratified Random Sampling?
Stratified random sampling helps to provide representative samples in our evaluations. By dividing a population into strata and randomly sampling from each stratum, we can better reflect the diversity within our population of interest. Stratified random sampling assists in reducing underrepresentation and overrepresentation within specific groups of our strata, allowing us to better capture important population characteristics that may be missed with a simple random sample.
Particularly, stratified random sampling is beneficial to evaluate the differences within stratum. That is, stratified random sampling allows us to make better comparisons between different population demographics or characteristics relevant to the evaluation that may otherwise be overlooked. Observing group differences across stratum can also promote diversity, equity and inclusion in evaluation as some groups might be heavily represented in an outcome relative to another group.
Limitations of Stratified Random Sampling
Stratified random sampling is not without limitations. These limitations include, but are not limited to:
Misclassification of Strata
While demographic strata, such as age range, may be clearly defined, other strata may be more nuanced. For example, ethnicity may not be clear for all members of a population, with some individuals identifying with one or more ethnic groups.
Time and Cost
When time and cost are limiting factors, splitting a population into appropriate strata while avoiding misclassification can become impractical. Dividing the population into strata and identifying a random sample within stratum require appropriate time and resources for planning and execution that can add to the logistical demands of the overall evaluation.
Wrapping Up
Stratified random sampling can be an effective method to provide comprehensive perspectives about your evaluation population. The key advantage of stratified random sampling lies in its ability to offer a nuanced portrayal of a population, by providing insights from all defined subgroups. The pay-off includes highlighting perspectives of underrepresented groups within the population that may otherwise be overlooked or overshadowed by other overrepresented groups.
While stratified random sampling can be applied to both quantitative and qualitative data collection, it can provide additional support for qualitative data collection, where sample sizes may be limited. For example, think about the time and resources required to conduct a single interview versus having a participant fill out a short survey. We are working on a Stratified Sampling Tool designed specifically for qualitative data collection. Our tool will streamline the qualitative data collection process by providing stratified random samples derived from a defined stratum. Keep an eye out for its release in early 2024.