Standard of Proof recently provided support for a national randomised trial in New Zealand. Interesting in this case, the evaluation randomly allocated “clusters” of people (rather than individuals) into control and treatment groups. Why is this important? Because your sample size needs to be significantly larger in such evaluation designs.
Let’s take an example. A class includes a group of students (ie, a “cluster”). If your evaluation and activity are being delivered across multiple schools, how can you randomise students easily and efficiently? One approach (more) common in clinical trials is to embed a clustered design element in your evaluation; in this example, we would randomise by class – our “cluster” – rather than by student. Although this may be a simpler allocation approach, it may not be as efficient for data collection and analysis.
Clustered data have a pre-existing group structure. Using this class example again, we would reasonably assume that children’s achievement scores are in some way influenced by their cluster (i.e., the teacher, their class, the characteristics of having enrolled in this specific school, etc). This “relatedness” in the cluster will have a significant effect on your sample size requirements. If measuring progress at the student level, what may have been a study requiring 200 participants may now require 2000. Further, the analysis will need to account for the clustering nature of the data. It’s useful to consider such implications prior to data collection and analysis, and plan accordingly.
Such considerations are useful to know up-front. Given the right balance, you can certainly design an efficient and practical randomised trial.
