The Design of Experiments

Content sourced from Wikipedia, licensed under CC BY-SA 3.0.

The Design of Experiments — a short, easy guide

What is it?
The Design of Experiments is a 1935 book by Ronald A. Fisher about how to plan and analyze scientific experiments. It introduced ideas that are still the foundation of modern statistics and experimental practice.

Key ideas Fisher introduced
- Randomization: randomly assigning treatments to subjects or plots to make fair comparisons and avoid bias.
- Replication: repeating experiments or treatments to get a clearer picture of what actually happened.
- Blocking: grouping similar units together to reduce the effect of unknown differences on the results.
- The null hypothesis: the starting assumption that there is no real effect or difference, which helps scientists decide if their results are due to chance.
- Factorial designs: testing several factors at once to see how they interact, rather than changing one thing at a time.

The Lady tasting tea and the null hypothesis
One famous example in the book is the Lady tasting tea. Fisher asked whether a woman could tell if milk was poured into the cup before tea or after. He set up eight cups to test this idea and used it to illustrate how the null hypothesis works: if the woman did no better than random guessing, there would be no reason to doubt the null hypothesis. If she did well, the experiment could challenge the idea that there was no real ability. This story helped formalize the concept of testing hypotheses with probability.

Why it matters
- Before this book, many experiments were poorly designed and hard to trust. Fisher showed how careful planning, randomization, and proper analysis could separate real effects from natural variation.
- It gave researchers a practical framework for designing experiments and judging their results, which improved research in agriculture, psychology, medicine, and many other fields.

Impact and legacy
- The book helped turn experimental work into a disciplined statistical practice. It popularized randomization, error estimation, and the use of statistics to draw valid conclusions from experiments.
- It influenced later researchers and the development of methods that are still standard today, including more sophisticated ways to handle variability and to compare multiple factors at once.
- While it sparked important advances, it also sparked debates about statistical methods. Some critics argued that certain uses of p-values and significance testing can be misleading, and discussions about the best way to reason from data continue in science.

In short, The Design of Experiments laid the groundwork for how scientists plan, run, and interpret experiments, making scientific testing more reliable and broadly applicable across disciplines.