Neighbourhood Effects Measured with a Counterfactual Design: A Journal Critique
In 2017 I did a postgrad Advanced Statistical Methods course, as it had been 10 years since I had been formally evaluated. As part of my professional development, I like to upgrade my skills in areas that are relevant to the students I work with. As a lot of my work with you pertains to research design, proposals, thesis support as well as research reports, I got a lot out of this course. You can download my journal critique on a Counterfactual Design. It is design to help determine causal inference when using an Observational Method of research outside of the lab. The control group in this design provides 'counterfactual' information; the comparison group where the intervention did not take place can be compared with the observed results with the group which was exposed.
A weakness of this design is that one cannot ever estimate with high accuracy what has occurred in the counterfactual group, as the absence of the intervention in itself, has unpredictable consequences. So, we can never say exactly what might or might not have happened if the intervention were to have taken place.
The paper I review used matched pairs of participants, so that a participant in the intervention group was matched with a participant in the control group, on a bundle of covariates. There is always 'error' or as I prefer to call it 'mystery' (the unknowable) which is what makes life~ Life. The counterfactual design is, I think, an efficient design for data collection and analysis of results, providing one keeps one's feet on the ground when interpreting results. Which should be the case regardless of the design used.
For the particular study of the article critiqued here, I suggested a mixed-method approach~ adding some qualitative elements (e.g., structured interviews) and collecting a broader variety of quantitative covariates (such as level community capacity building, level of spirituality and cultural specific practices).