How to report pairwise comparison with Wald Chi-square in R (following a significant 2-way interaction from glmer)

Thanks for reading my question.

My question is about how to write consistent stats report for both the main and the follow-up (pairwise comparison) from GLMER outcome (2By2 with binomial outcome) in an APA format.

  1. Dataset:
    Two IVs: Condition (2 levels, categorical); Group (3 levels, categorical)
    DV: score (binomial, 1 as correct and 0 as incorrect)

  2. Main Model:

m <- gmler(score ~ Condition*Group + (1+ Condition | id) + (1+ item), data = dataset, family = binomial)
[This guy works after reducing the model complexity from the maximal model]
Anova(m, type = 3)
  1. Main Results:
    We got a significant interaction of Condition×Group, a marginal main effect of Group, and a non-significant main effect of condition (see below).
    The data we reported will be Wald X2. For instance: the Condition×Group interaction was significant (Wald X2(2) = 7.13, p = .028)
               Chisq Df Pr(>Chisq)    
(Intercept) 117.4950  1    < 2e-16 ***
condition          0.0556  1    0.81353    
group         5.9676  2    0.05060 .  
condition:group    7.1374  2    0.02819 * 

4. Follow-up
When we followed up on the Condition×Group interaction, for the differences of each pair of Groups within each level of Condition, we used the suggestion of function "emmeans", suggested from several previous posts (e.g. post 1 and post 2)

Pairwise Model

pairwise <- emmeans(m, list(pairwise ~ group| condition), adjust = "holm")

Pairwise Results

$`pairwise differences of group | condition`
condition = Cued:
 2               estimate    SE  df z.ratio p.value
 group0 - group1  -0.0863 0.115 Inf  -0.751  0.4529
 group0 - group2  -0.4404 0.126 Inf  -3.482  0.0015
 group1 - group2  -0.3541 0.126 Inf  -2.821  0.0096

condition = Uncued:
 2               estimate    SE  df z.ratio p.value
 group0 - group1  -0.1065 0.122 Inf  -0.872  1.0000
 group0 - group2  -0.0575 0.136 Inf  -0.422  1.0000
 group1 - group2   0.0490 0.135 Inf   0.362  1.0000

My Question

For the main results section, we reported the Condition×Group interaction in the format of Wald X2. For the follow-up from emmeans function, we do not have Wald X2 from "emmeans", but we get z, estimate, SE, and p-value.*

How to keep the reporting of the main and the pairwise comparison (follow-up) results consistent, e.g. reporting both interaction and the pairwise with Wald X(2)?

P.S. I got a reviewer comment to keep reporting the main and the follow-up results in the same format, e.g. Wald X(2). Any help would be appreciated if it is possible to run pairwise comparison and generate Wald X2. Or is it not recommended to generate and report Wald X2 for the follow-up.

I'd say Wald z for follow-up, pairwise comparisons is expected after a Chi squared test of overall effects. If you need p-values for this work :blush: I don't think you need to change your test type.

This has direct parallels with something like a two-way ANOVA, where we use F tests for overall effects and t tests for pairwise comparisons. The Chi squared distribution is the asymptotic/limiting distribution for F (F denom df goes to infinity) in the same way a z distribution is related to a t distribution.

Hope that helps!

Thank you for reading my post and giving the suggestion. :slight_smile:
So, that means, Wald z is preferred after a Wlad X(2) (parallel to t after an F test)? The results generated from "emmeans" below is appropriate? (If I understand correctly)

pairwise <- emmeans(m, list(pairwise ~ group| condition), adjust = "holm")

Thank you again- happy holidays!

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.