Which statistical test should I use
Answer the questions
What do you want to find out?
What are you measuring in each group?
How many groups are you comparing?
Are the two sets of measurements from the same subjects?
Are the same subjects measured in every condition?
How are the categories organised?
Does any cell of the table expect fewer than five cases?
What kind are the two variables?
Does the cloud of points look like a straight line, without extreme outliers?
What kind is the variable you want to predict?
How many variables are you predicting from?
One sample t test
It asks whether the mean of your group differs from a value you already knew.
Check that the data are roughly symmetric. If they are not, use the Wilcoxon signed rank test.
Independent samples t test
It asks whether two separate groups have different means.
Check that each group is roughly symmetric. If they are not, use the Mann Whitney U test.
Paired t test
It asks whether the same subjects changed between the two measurements.
Check that the differences are roughly symmetric. If they are not, use the Wilcoxon signed rank test.
One way ANOVA
It asks whether at least one of three or more groups differs from the others.
It tells you that some group differs, not which one, so a post hoc comparison comes next. If the data are not symmetric, use the Kruskal Wallis test.
Repeated measures ANOVA
It asks whether the same subjects changed across three or more conditions.
Check sphericity, and correct it if it fails. The rank based alternative is the Friedman test.
Chi square goodness of fit
It asks whether the counts you observed match the proportions you expected.
With only two categories and few cases, the exact binomial test is the safer choice.
Chi square test of independence
It asks whether two categorical variables are associated.
It says there is an association, not how strong. Report an effect size such as Cramer's V alongside it.
Fisher's exact test
It answers the same question as chi square, but stays valid when the counts are small.
Use it whenever an expected count falls below five, which is exactly where chi square stops being trustworthy.
McNemar test
It asks whether the same subjects changed category between two moments.
Only the subjects who changed carry information; the ones who stayed the same do not enter the test.
Pearson correlation
It measures how close two numeric variables are to a straight line relationship.
Correlation is not causation, and a single outlier can create or destroy it. Plot the points before trusting the number.
Spearman correlation
It measures whether one variable rises as the other rises, without demanding a straight line.
It works on ranks, so it resists outliers and also fits ordered scales.
Simple linear regression
It draws the line that best predicts a number from one other number.
Look at the residuals: they should have no pattern. A curve in them means the straight line is the wrong shape.
Multiple linear regression
It predicts a number from several other variables at once.
Check that the predictors are not saying the same thing as each other, which inflates the coefficients and makes them unstable.
Logistic regression
It predicts the probability of a yes or no outcome.
Its coefficients are odds ratios, not differences in probability, and reading one as the other is the usual mistake.
Multinomial logistic regression
It predicts which of several unordered categories an observation falls into.
Every result is read against a reference category, so say which one you chose.
Ordinal logistic regression
It predicts a position on an ordered scale, keeping the order in the model.
It assumes the effect is the same across every cut of the scale. Test that assumption before reporting.
Poisson regression
It predicts how many times something happens in a period or a place.
It assumes the variance equals the mean. When the variance is much bigger, use negative binomial regression instead.
How do I know whether my data are normal enough for a t test?
Plot them first. A histogram or a box plot answers the question faster than any test, and what matters is not perfect normality but whether the shape is roughly symmetric and free of extreme outliers.
The t test also gets more forgiving as the sample grows, so a mild skew with a hundred cases is rarely a problem while the same skew with eight cases is. When in doubt, run the rank based alternative as well: if both agree, the choice did not matter.
What is the difference between a parametric and a rank based test?
A parametric test works on the values themselves and assumes a shape for them, usually something close to a normal curve. A rank based test throws the values away and keeps only their order, which is why outliers stop mattering.
The price is that the rank based test answers a slightly different question and has less power when the assumption of the parametric one really does hold. Neither is a safer default; they are answers to different situations.
The test came out significant. Does that mean the effect is big?
No. A p value answers whether the result is hard to explain by chance, and with a large enough sample even a tiny difference becomes hard to explain by chance.
Report an effect size next to it, and a confidence interval if you can. The size tells the reader whether the difference matters; the p value only says it is probably not noise.
Does this replace asking someone who knows statistics?
No, and it is not trying to. It points at the choice that fits the shape of your question, and it tells you which assumption to check before using it.
Real data bring things a flowchart cannot see: missing values, subjects measured more than once, groups of very different sizes, variables that were chosen after looking at the results. Any of those can change the answer.
Reactions
0
0 Comments
Be the first to comment