Answer:
A statistically significant relationship is one that is large enough to be unlikely to have occurred in the sample if there's no relationship in the population. The issue of whether a result is unlikely to happen by chance is an important one in establishing cause-and-effect relationships from experimental data. If an experiment is well planned, randomization makes the various treatment groups similar to each other at the beginning of the experiment except for the luck of the draw that determines who gets into which group. Then, if subjects are treated the same during the experiment (e.g. via double blinding), there can be two possible explanations for differences seen: 1) the treatment(s) had an effect or 2) differences are due to the luck of the draw. Thus, showing that random chance is a poor explanation for a relationship seen in the sample provides important evidence that the treatment had an effect.
The issue of statistical significance is also applied to observational studies - but in that case, there are many possible explanations for seeing an observed relationship, so a finding of significance cannot help in establishing a cause-and-effect relationship. For example, an explanatory variable may be associated with the response because:
Changes in the explanatory variable cause changes in the response;
Changes in the response variable cause changes in the explanatory variable;
Changes in the explanatory variable contribute, along with other variables, to changes in the response;
A confounding variable or a common cause affects both the explanatory and response variables;
Both variables have changed together over time or space; or
The association may be the result of coincidence (the only issue on this list that is addressed by statistical significance).
Remember the key lesson: correlation demonstrates association - but the association is not the same as causation, even with a finding of significance.
« Previous
Next »
Lesson
Welcome to STAT 100!
1: Statistics: Benefits, Risks, and Measurements
2: Characteristics of Good Sample Surveys and Comparative Studies
3: Getting the Big Picture and Summaries
4: Bell-Shaped Curves and Statistical Pictures
5: Relationships Between Measurement Variables
5.1 - Graphs for Two Different Measurement Variables
5.2 - Correlation & Significance
5.3 - Key Caveats with Correlations
5.4 - Regression
5.5 - Two Warnings about Regression
5.6 - Test Yourself!
5.7 - Have Fun With It!
6: Relationships Between Categorical Variables
7: Understanding Uncertainty
8: The Diversity of Samples
9: Confidence Intervals
10: Hypothesis Testing
11: Significance Testing Caveats & Ethics of Experiments
Resources
References
Reviews
