Block 3: Confidence intervals and coverage
Week 11 · Uncertainty I · Imai §7.1.3 · about 4 min
NoteHow this works
Run each cell with Run Code, look at the output, then answer the question at the end of the block. Every cell is editable; Start Over resets it. Errors are information: read them.
A 95% confidence interval is: estimate \(\pm\) 1.96 \(\times\) standard error.
1. Poll: 99%, 95% and 90% intervals
A poll of n = 1,000 finds a sample proportion \(\bar{x} = 0.6\).
2. Coverage: how often does the 95% interval contain the truth?
The true proportion is p = 0.6. For each sample size we draw 5,000 samples, build the 95% interval each time and check whether it contains p.
Conclusion: Is the coverage of the "95%" interval exactly 95%?
Coverage is about 0.90, 0.94, 0.95 and 0.95 for n = 10, 50, 100, 1000. The normal approximation needs enough data: with n = 10 the interval is too narrow. The 99% interval is the widest (0.560 to 0.640): higher confidence costs precision.