metafor SMCC effect size calculation for repeated-measures study with derived outcome variable

Hi everyone,

I am currently working on a meta-analysis and need some advice on calculating an effect size for a repeated-measures study using metafor.

The study I am extracting data from uses a within-subjects design. The outcome reported in the paper is already a calculated difference score (i.e., each participant has a score based on the difference between two measurements). The authors provide the mean and SD of this difference score separately for each condition.

For example:

  • Condition A: M = ..., SD = ...
  • Condition B: M = ..., SD = ...

The same participants completed both conditions.

I am planning to calculate a standardised mean change effect size using:

escalc(measure = "SMCC", ...)

My understanding is that the correlation (ri) required for this calculation should be the correlation between the outcome scores across conditions, i.e. the correlation between the difference scores:

cor((Measure1 - Measure2) in Condition A,
    (Measure1 - Measure2) in Condition B)

However, the authors have only provided correlations for the two individual components that make up the difference score (i.e., the correlation for Measure 1 across conditions and the correlation for Measure 2 across conditions).

I am unsure whether these correlations can be used instead, or whether I specifically need the correlation between the derived difference scores themselves.

Could someone clarify what the appropriate approach would be in this situation?

Many thanks!

The key point is that SMCC must be calculated from the pre/post moments of the outcome that is actually being meta-analysed. If that outcome is derived, first derive its mean, SD at each occasion, and pre/post correlation. Substituting the SDs and correlation of one component will generally give the wrong effect size and sampling variance.

Suppose the outcome is a difference at each occasion: D_pre = A_pre - B_pre and D_post = A_post - B_post. Then:

mean(D_t) = mean(A_t) - mean(B_t)
var(D_t) = var(A_t) + var(B_t) - 2 cov(A_t, B_t)

and:

cov(D_pre, D_post) = cov(A_pre, A_post) - cov(A_pre, B_post) - cov(B_pre, A_post) + cov(B_pre, B_post)

Therefore, r_D = cov(D_pre, D_post) / [sd(D_pre) sd(D_post)].

Once those quantities have been obtained, pass the derived-outcome moments to escalc():

es <- metafor::escalc(
  measure = "SMCC",
  m1i = mean_D_pre,
  sd1i = sd_D_pre,
  m2i = mean_D_post,
  sd2i = sd_D_post,
  ri = r_D,
  ni = n
)

Check the sign of yi against your intended direction of improvement. If necessary, reverse the occasion order or multiply yi by -1 consistently. The sampling variance vi is unchanged when only the sign is reversed.

The important limitation is that the four marginal SDs plus the two within-variable pre/post correlations are not enough. For a difference outcome you also need the within-occasion A-B covariances and the two cross-variable, cross-time covariances. In other words, you need the relevant 4 by 4 covariance/correlation matrix, raw participant-level data, or the SD/correlation of the already-derived outcome reported directly. If these are unavailable, r_D is not identified from the reported summaries. Any assumed value should be handled as a sensitivity analysis, not presented as observed information.

The same logic generalises to a linear composite. If D_t = w'X_t, then var(D_t) = w' Sigma_t w and cov(D_pre, D_post) = w' Sigma_pre,post w. For a ratio, percentage change, product, log-response, or another nonlinear derived outcome, these linear identities do not apply exactly. With raw data, compute the derived score participant by participant. With aggregate data, use a justified delta-method approximation or request the required summary/covariance information.

Finally, if a study contributes several SMCC estimates from the same participants, their sampling errors are dependent. Computing each yi and vi separately does not remove that dependence. The meta-analysis should use an appropriate sampling variance-covariance matrix, or a justified multilevel/robust approach, with sensitivity analysis for unreported correlations.