“Inferential measures such as standard errors and confidence intervals should not be used to describe the variability of characteristics, and significance tests should be avoided in descriptive tables. Also, P values are not an appropriate criterion for selecting which confounders to adjust for in analysis; even small differences in a confounder that has a strong effect on the outcome can be important.”

Vandenbroucke, J. P., Elm, E. V., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., … & Strobe Initiative. (2007). Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Annals of internal medicine, 147(8), W-163.

Should there be p-values in Table 1 of controlled trials (RCT)? P-values were traditionally used to assess for imbalanced baseline covariates in RCTs. However, this practice is increasingly ceasing in favor of qualitative description or no description of baseline imbalances. Randomization distributes known and unknown confounding factors, and any differences seen will be due to chance. Showing p-values can actually give a misleading picture of these differences.

“The authors of this debate are obviously more (Baethge) or less enthusiastic (Stang), with the former advocating the presentation of CIPs [covariate imbalance p values], its careful use as a screening tool, and its interpretation within the context of each study, while the latter emphasizes the dangers of misuse. The aim of this debate was to further trigger the discussion of the role of NHST in biomedical research that uses randomization… For the detection and judgment of imbalances between the study groups, it remains important that descriptive statistics of the groups (categorical characteristics: percentage values; continuous characteristics: eg, mean values and SDs) are presented. Whether a baseline imbalance is meaningful or not depends on subject matter knowledge. For example, it is clinically relevant in a stroke prevention study if 30% diabetics are in one arm of the study and only 15% are diabetics in the other arm, regardless of the p value, as diabetes mellitus is a very relevant risk factor for stroke.”

Stang, A., & Baethge, C. (2018). Imbalance p values for baseline covariates in randomized controlled trials: a last resort for the use of p values? A pro and contra debate. Clinical epidemiology, 531-535.

Are there any alternatives to the use of hypothesis testing for assessing balance? Multiple alternatives have been proposed which avoid hypothesis testing and p-values. These include tools that can assess balance graphically and numerically.

“In any study where all observed covariates were not fully blocked ahead of time, balance should be checked routinely by comparing observed covariate differences between the treated and control groups. Any statistic that is used to evaluate balance should have two key features: (a) it should be a characteristic of the sample and not of some hypothetical population and (b) the sample size should not affect the value of the statistic.”

Imai, K., King, G., & Stuart, E. A. (2008). Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society Series A: Statistics in Society, 171(2), 481-502.

“MatchIt contains several tools to assess balance numerically and graphically. The primary balance assessment function is summary.matchit(), which is called when using summary() on a MatchIt object and produces several tables of balance statistics before and after matching. plot.summary.matchit() generates a Love plot using R’s base graphics system containing the standardized mean differences resulting from a call to summary.matchit() and provides a nice way to display balance visually for inclusion in an article or report. plot.matchit() generates several plots that display different elements of covariate balance, including propensity score overlap and distribution plots of the covariates. These functions together form a suite that can be used to assess and report balance in a variety of ways.”

https://cran.r-project.org/web/packages/MatchIt/vignettes/assessing-balance.html

Directed Acyclic Graphs, or DAGs, are commonly used in Epidemiology to communicate the assumptions that are being made about the structure of the causal network being studied, and can be used to guide our analytic strategy by facilitating the identification of non-causal pathways which could bias the estimates of targeted effect estimands. The following is a list of resources that are useful in learning about DAGs and their applications:

Pearl J. Causal diagrams for empirical research. Biometrika 1995;82:669–88.

Access article on Google Scholar »

In this article, Pearl introduces a framework utilizing DAGs to identify and communicate what assumptions are necessary to produce causal effect estimates from nonexperimental data.

Greenland S, Pearl J, Robins JM. Causal Diagrams for Epidemiologic Research. Epidemiology 1999; 10:37–48.

Access article on Google Scholar »

Greenland, Pearl and Robins provide an introduction to the use of DAGs in Epidemiology. This includes an overview of terminology, application, and a discussion of how this framework expands the traditional epidemiologic criteria for confounding.

Glymour MM. Using causal diagrams to understand common problems in social epidemiology. Methods Soc. Epidemiol., 2006, p. 387–422.

Access article on Google Scholar »

In Chapter 16, Glymour provides an overview for the application of DAGs in Epidemiology with an emphasis on their use in causal inference and in social epidemiology. Glymour also provides discussion of related contemporary issues such as: natural experiments, identification of direct and indirect effects, and the use of indicator variables for handling missing data.

Hernán MA, Robins JM. Graphical Representation of Causal Effects. Causal Inference What If, 2020, p. 69–82.

Access article on website »

Chapter 6 provides an introduction to the use of DAGs in Epidemiology for causal inference, with an emphasis on linking DAGs to the counterfactual framework.

Westreich D, Greenland S. The table 2 fallacy: Presenting and interpreting confounder and modifier coefficients. Am J Epidemiol 2013; 177: 292–8.

Access article on Google Scholar »

Westreich and Greenland use DAGs to introduce and illustrate the Table 2 Fallacy (i.e., the fallacy that all estimated coefficients for a given model can be interpreted as estimating the same type of effect). They also discuss strategies that can be employed to avoid misinterpretations that can result from the Table 2 Fallacy.

Textor J, Hardt J, Knüppel S. DAGitty: A graphical tool for analyzing causal diagrams. Epidemiology 2011;22:745.

Access article on Google Scholar »

DAGitty is a free, user-friendly, online tool that allows you to easily build your DAG. The application is then able to identify the minimally sufficient adjustment set necessary for estimating the total effect or direct effect. It can also identify instrumental variables that are available in the causal network, and it provides a list of testable assumptions based on the causal network (http://www.dagitty.net/dags.html).