# What statistics to use?

Announcements
#1
I've done a 2x2 ANOVA and found a non-significant interaction between variables A and B. Doing t-tests between Ai and Bi are significant whereas Aii and Bii are not. I realise I cannot infer an interaction between the two variables but can I still use the t-tests to report a specific relationship?
0
4 years ago
#2
(Original post by Pali138)
I've done a 2x2 ANOVA and found a non-significant interaction between variables A and B. Doing t-tests between Ai and Bi are significant whereas Aii and Bii are not. I realise I cannot infer an interaction between the two variables but can I still use the t-tests to report a specific relationship?
I don't fully get what you mean here.

In general, what you look for in a t-test (a mean difference) is what you get out of the main effect from an ANOVA.
If you have two different groups (e.g. the effect of a drug between a placebo and a active condition, and the effect of gender of participant on outcome).
An ANOVA will tell you if the main effects of gender and drug condition are significant, and if there's an interaction (e.g. does the drug work better in males or females).

You could report a t-test for gender and drug group on outcomes which is equally valid but you cannot infer any interaction effect. This is even if you run a t-test for the males and females seperately - just because you may find a significant effect in the males and not the females doesn't mean that there is a significant interaction by gender (you could have p=.04 for males and p=.06 for females, which are very similar results...)
0
#3
Okay using this example, does this make sense? When comparing gender and drug condition, the ANOVA interaction returns as non-significant.

There is in males, a significant difference between active and placebo (p = 0.4) whereas p = 0.6 for females is non-significant so therefore its likely due to chance; no matter how close the p values actually are?

Therefore, the gender and drug interaction cannot predict the outcome of the patient, however the results of the t-tests suggest when the active drug is given to a male there is a significant difference in outcome. This effect is not observed in females.
0
4 years ago
#4
(Original post by Pali138)
Okay using this example, does this make sense? When comparing gender and drug condition, the ANOVA interaction returns as non-significant.

There is in males, a significant difference between active and placebo (p = 0.4) whereas p = 0.6 for females is non-significant so therefore its likely due to chance; no matter how close the p values actually are?

Therefore, the gender and drug interaction cannot predict the outcome of the patient, however the results of the t-tests suggest when the active drug is given to a male there is a significant difference in outcome. This effect is not observed in females.
Do you mean that your ANOVA main effect is not significant and the interaction is not significant, but the t-test for one subgroup (e.g. males) is significant?

This paper will clarify my above point: http://www.stat.columbia.edu/~gelman...ed/signif4.pdf in summary, significance does not equal no effect - you can get a non-significant result just because your sample is tiny.

The main outcome of your analysis should include all your participants - you get into dangerous territory when you start running analyses separately in groups (as you can run this for various groups such as gender/age/ethnicity just to fish for a statistically significant result).

In the above case if your main effect is not significant but your t-test for a subgroup (e.g. males) is significant, then its worth reporting but also your results are probably quite spurious if they're edging on p=.05. I'd report it as a "tentative/suggestive" result.

If your main effect is significant then doing the t-tests for each group adds little value except for providing descriptive stats for each subgroup.
0
#5
Right okay, thank you so much!
0
X

new posts
Back
to top
Latest

### Oops, nobody has postedin the last few hours.

Why not re-start the conversation?

see more

### Poll

Join the discussion

They're better than I expected (20)
31.75%
They're what I expected (19)
30.16%
They're worse than what I expected (24)
38.1%