Competition - post a photo you've taken of nature >>

What statistics to use?

I've done a 2x2 ANOVA and found a non-significant interaction between variables A and B. Doing t-tests between Ai and Bi are significant whereas Aii and Bii are not. I realise I cannot infer an interaction between the two variables but can I still use the t-tests to report a specific relationship?

Reply 1

iammichealjackson

Original post by Pali138

I don't fully get what you mean here.

In general, what you look for in a t-test (a mean difference) is what you get out of the main effect from an ANOVA.
If you have two different groups (e.g. the effect of a drug between a placebo and a active condition, and the effect of gender of participant on outcome).
An ANOVA will tell you if the main effects of gender and drug condition are significant, and if there's an interaction (e.g. does the drug work better in males or females).

You could report a t-test for gender and drug group on outcomes which is equally valid but you cannot infer any interaction effect. This is even if you run a t-test for the males and females seperately - just because you may find a significant effect in the males and not the females doesn't mean that there is a significant interaction by gender (you could have p=.04 for males and p=.06 for females, which are very similar results...)

Reply 2

Pali138

Okay using this example, does this make sense? When comparing gender and drug condition, the ANOVA interaction returns as non-significant.

There is in males, a significant difference between active and placebo (p = 0.4) whereas p = 0.6 for females is non-significant so therefore its likely due to chance; no matter how close the p values actually are?

Therefore, the gender and drug interaction cannot predict the outcome of the patient, however the results of the t-tests suggest when the active drug is given to a male there is a significant difference in outcome. This effect is not observed in females.

(edited 6 years ago)

Reply 3

iammichealjackson

Original post by Pali138

Do you mean that your ANOVA main effect is not significant and the interaction is not significant, but the t-test for one subgroup (e.g. males) is significant?

This paper will clarify my above point: http://www.stat.columbia.edu/~gelman/research/published/signif4.pdf in summary, significance does not equal no effect - you can get a non-significant result just because your sample is tiny.

The main outcome of your analysis should include all your participants - you get into dangerous territory when you start running analyses separately in groups (as you can run this for various groups such as gender/age/ethnicity just to fish for a statistically significant result).

In the above case if your main effect is not significant but your t-test for a subgroup (e.g. males) is significant, then its worth reporting but also your results are probably quite spurious if they're edging on p=.05. I'd report it as a "tentative/suggestive" result.

If your main effect is significant then doing the t-tests for each group adds little value except for providing descriptive stats for each subgroup.

What statistics to use?

Quick Reply

Related discussions

Latest

Trending

Trending

Articles for you