Normality plot and shapiro test for repeated-measures ANOVA: very nearly significant but not quite...

[eyavuz21]

Hello all,

I have a repeated-measures anova (Two-way), and I want to check for normality; my plots show this:

lmerabsolute <- aov(Proportionofundershoots~ Target*Experiment + (ID/(Target*Experiment)), data=overallundershootproportion)

However, my shapiro test shows this:

shapiro.test(resid(lmerabsolute))

	Shapiro-Wilk normality test

data:  resid(lmerabsolute)
W = 0.98896, p-value = 0.001789

What do I do? I would be so grateful for some advice!

When I arcsine transform the data, my new shapiro test result looks like this:

Shapiro-Wilk normality test

data:  resid(lmerabsolute)
W = 0.98975, p-value = 0.003128

Here is the arcsine transformed normality plot:

So definitely an improvement! I have not managed to get a more significant P-value than this!

[technocrat]

Shapiro-Wilks has a null hypothesis that the data are drawn from a population that is normally distributed. The alternative hypothesis is that they are drawn from a population that has a different distribution.

Whenever doing a statistical test and checking a metric such as p-value the starting point is deciding on a threshold decision boundary: If the outcome is x the null hypothesis is rejected or failed to be rejected.

For p-value, the probability that the result is simply random, the conventional \alpha is 0.05. (I call this the laugh test.) Where greater confidence is needed that might be dropped to 0.01 or even 0.001. In any event, after setting the threshold it's bad form to change it to be able to claim that the result is significant (statistic's most unfortunate turn of phase).

In your example, it makes sense to look for p-value < 0.001789 if you have previously selected \alpha = 0.001 or less, but not otherwise. It's been said

If you torture the data long enough, it will confess anything.

and contortions to find the lowest possible p-value are termed p-value hacking, a blot upon the escutcheon.