Adv Physiol Educ Journal of Neurophysiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Advan. Physiol. Edu. 33: 87-90, 2009; doi:10.1152/advan.00006.2009
1043-4046/09 $8.00
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Material
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Curran-Everett, D.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Curran-Everett, D.
ADV PHYSIOL EDUC 33:87-90, 2009
© 2009 American Physiological Society

STAYING CURRENT

Explorations in statistics: confidence intervals

Douglas Curran-Everett

Division of Biostatistics and Bioinformatics, National Jewish Health, and Department of Biostatistics and Informatics and Department of Physiology and Biophysics, School of Medicine, University of Colorado Denver, Denver, Colorado

Address for reprint requests and other correspondence: D. Curran-Everett, Div. of Biostatistics and Bioinformatics, M222, National Jewish Health, 1400 Jackson St., Denver, CO 80206 (e-mail: EverettD{at}NJHealth.org)

Abstract

Learning about statistics is a lot like learning about science: the learning is more meaningful if you can actively explore. This third installment of Explorations in Statistics investigates confidence intervals. A confidence interval is a range that we expect, with some level of confidence, to include the true value of a population parameter such as the mean. A confidence interval provides the same statistical information as the P value from a hypothesis test, but it circumvents the drawbacks of that hypothesis test. Even more important, a confidence interval focuses our attention on the scientific importance of some experimental result.

Key words: estimation; R; software

THIS THIRD ARTICLE in Explorations in Statistics (see Refs. 11 and 12) provides an opportunity to explore confidence intervals. A confidence interval estimates our uncertainty about the true value of some population parameter.1 For example, when we construct a confidence interval for the mean of some population, we expect, with some level of confidence, that the true value of the population mean will fall within that interval. A confidence interval provides the same statistical information as the P value from a hypothesis test, it circumvents the drawbacks inherent to that hypothesis test, and it provides information about scientific importance. The routine reporting of confidence intervals is recommended (1–3, 9, 13, 15, 18), but the meaning of a confidence interval is difficult to understand (7, 16). To be blunt, the meaning of a confidence interval is nearly impossible to understand unless you observe the development of its underlying concept. In this exploration, we will.

A Brief History of Confidence Intervals

Unlike hypothesis tests whose origins can be traced to 1279 (25), confidence intervals are a recent development: Jerzy Neyman derived them in the 1930s (20–22). There would be a 50-year lag before medical journals advocated the use of confidence intervals (4, 5, 17, 18). It would be just 10 years before George Snedecor added confidence intervals to his historic Statistical Methods (24).2 In 1913, 6 years before Fisher went to Rothamsted Station (11, 26), Snedecor arrived at Iowa State College as an assistant professor of mathematics (6, 8, 10, 19). In his courses, Snedecor derived examples based on agricultural and biological data from researchers at Iowa State. These collaborations led Snedecor to create the Mathematics Statistical Service (1927) and then the Statistical Laboratory (1933) at what is now Iowa State University. The hallmark of Statistical Methods is its focus on the application of statistical methods to actual scientific problems and data.

R: Basic Operations

In the inaugural article (12) of this series, I summarized the freeware package R (23) and outlined its installation. For this exploration, there is just one additional step: download the script Advances_Statistics_Code_CI.R3 to your Advances folder (see Ref. 12).

If you use a Mac, highlight the commands in Advances_Statistics_Code_CI.R you want to submit and then press Formula (command key+enter). If you use a PC, highlight the commands you want to submit, right-click, and then click Run line or selection. Or, highlight the commands you want to submit and then press Ctrl+R.

The Simulation: Observations and Sample Statistics

For these explorations (11, 12), we drew a total of 1000 random samples–each with 9 observations–from our population, a standard normal distribution with mean µ = 0 and standard deviation {sigma} = 1 (see Ref. 12, Fig. 2). These were the observations–the data–for samples 1, 2, and 1000: > # Sample Observations

Formula


Figure 2
View larger version (9K):
[in this window]
[in a new window]

  		Fig. 2. Theoretical distribution of the sample mean for n observations. The interval [µ – a, µ + a] covers 100(1 – {alpha})% of the possible sample means (see Eq. 1). Compared to the distribution of population values, the theoretical distribution of the sample mean is narrower by a factor of 1/n and taller by factor of n (see Fig. 1 in Ref. 13).

 
Each time we drew a random sample, we calculated the sample statistics listed in Table 1. These were the statistics for samples 1, 2, and 1000:

Formula

The commands in lines 35–63 of Advances_Statistics_Code_ CI.R generate the observations and compute the sample statistics. These commands are identical to those in the first two scripts (11, 12).


View this table:
[in this window]
[in a new window]

  		Table 1. Sample statistics calculated for each random sample

 
With these 1000 sets of sample statistics, we are ready to explore confidence intervals.

Confidence Intervals

When we began these explorations, we wanted in part to estimate µ = 0, the mean of our population. In the first iteration of our simulation, the sample mean y = 0.797 estimated the population mean. In the second iteration, the sample mean 0.517 estimated the population mean. All told, we have 1000 sample estimates of the population mean: 900 of them are between –0.523 and +0.552 (Fig. 1). 4 We can generalize from this empirical distribution of sample means to the theoretical distribution of the sample mean, a normal distribution with mean µ and standard deviation {sigma}/Formula (12, 15), where n is the number of observations in the sample.


Figure 1
View larger version (9K):
[in this window]
[in a new window]

  		Fig. 1. Empirical (black) and theoretical (gray) distributions of the sample mean for 9 observations. The empirical distribution is composed of 1000 sample means. The empirical standard deviation of the 1000 sample means, 0.326, is near the theoretical value of 1/3. [Reprinted from Ref. 12.]

 
In the theoretical distribution of the sample mean (Fig. 2), 100(1 – {alpha})% of the possible sample means are covered by the interval

Formula

where the allowance a is

Formula 1(1)

In Eq. 1, z{alpha}/2 is the 100[1 – ({alpha}/2)]th percentile from the standard normal distribution, and SD{Formula 1} is the standard deviation of the sample means, {sigma}/Formula 1. The standard deviation of the distribution of the sample mean is also called the standard error of the sample mean SE{Formula 1}.

Suppose we want the interval [µ – a, µ + a] to cover 90% of the possible sample means for 9 observations. In this situation, µ = 0, {alpha} = 0.10, and z{alpha}/2 = 1.645. Because we defined the population standard deviation {sigma} to be 1 (see Ref. 12),

Formula 1

and the resulting allowance a is

Formula 1

Therefore, the interval [–0.548, +0.548] covers 90% of the sample means for 9 observations. This theoretical interval agrees with the empirical interval of [–0.523, +0.552].

When we calculate the interval [µ – a, µ + a], we use the population mean µ to learn about possible values of the sample mean Formula 1. Is this what we do when we calculate a confidence interval for the mean of some population? Sadly, no. When we calculate a confidence interval, we use the sample mean Formula 1 to learn about possible values of the population mean µ. Happily, we can use the interval [µ – a, µ + a] to derive a confidence interval.

First, we write the interval [µ – a, µ + a] as the probability expression

Formula 1

What does this expression mean in words? It means that the probability is 1 – {alpha} that a sample mean is covered by–lies within–the interval [µ – a, µ + a] (see Fig. 2). Then, we rearrange the joint inequality portion of the expression to get

Formula 1

In this form, the interval

Formula 2(2)

is called the 100(1 – {alpha})% confidence interval for the population mean µ. Now we have what we want.

In an actual experiment, we do not know the population standard deviation {sigma}. Therefore, we use the sample standard deviation s to estimate the population standard deviation {sigma} and s/Formula 2 to estimate the standard error of the sample mean. In addition, when we calculate a 100(1 – {alpha})% confidence interval for some population mean µ, we handle our uncertainty about the actual value of {sigma} by replacing z{alpha}/2 (Eq. 1) with t{alpha}/2,v, the 100[1 – ({alpha}/2)]th percentile from a Student t distribution with v = n – 1 degrees of freedom. As a result, the allowance we apply to the sample mean to obtain the 100(1 – {alpha})% confidence interval (Eq. 2) becomes

Formula 2

where SE{Formula 2} = s/Formula 2.5 This allowance is bigger than the allowance in Eq. 1: we are more uncertain about the value of the population mean µ. This happens because if v < {infty}, then t{alpha}/2,v > z{alpha}./2 for all values of {alpha}.

Suppose we want to calculate a confidence interval for the population mean µ = 0 using the observations 0.422, 1.103,..., 1.825 of the first sample. The mean and standard deviation of these 9 observations are Formula 2 = 0.797 and s = 0.702, and the estimated standard error of the mean is

Formula 2

Because n = 9, there are v = 8 degrees of freedom. If we want a 90% confidence interval, then {alpha} = 0.10, t{alpha}/2,v = 1.860, and the allowance a = 1.860 x 0.234 = 0.435. Therefore, the 90% confidence interval is

Formula 2

What does this expression mean in words? We can declare, with 90% confidence, that the population mean is included in the interval [0.36, 1.23]. Because 0 is outside this interval, we can state, with 90% confidence, that 0 is not a plausible value of the population mean. This inference is consistent with our second exploration in which we rejected the null hypothesis H0: µ = 0 and concluded that the sample observations were consistent with having come from a population that had a mean other than 0 (see Ref. 12).

But now we have a problem: a single confidence interval either does or does not include the true value of some population parameter. In a real experiment, we do not know which outcome has occurred. So the question is, where does the notion of confidence in a confidence interval come from? The answer: not from a single confidence interval but from a theoretical process of calculating a whole bunch of confidence intervals. For these explorations, we drew a total of 1000 random samples. Each time we drew a random sample, we calculated its mean and standard deviation. Because the population from which we drew the samples was distributed over a range of possible values, the sample means (see Fig. 1) and standard deviations (see Ref. 12, Fig. 3) varied among our 1000 samples. Therefore, we calculated 1000 different confidence intervals. We expect about 100(1 – {alpha})% of these confidence intervals to include the actual value of the population mean (Fig. 3). This is the underlying concept of confidence in a confidence interval. The next question is, how do we use a confidence interval to help us make an inference about scientific importance?


Figure 3
View larger version (9K):
[in this window]
[in a new window]

  		Fig. 3. Confidence intervals for the initial 100 samples of 9 observations. It is because of random sampling that the position and length of the confidence intervals vary from sample to sample. About 90 of these intervals–the actual number will vary–are expected to cover the population mean of 0. In this simulation, 84 of the confidence intervals cover 0; the 16 exceptions are highlighted (numbered black lines). To generate this data graphic, highlight and submit the lines of code from Figure 3: first line to Figure 3: last line.

 
In a manner similar to Ref. 14, suppose you find three articles in Physiological Genomics that investigated independently the impact of three different drugs on the expression of some gene. Suppose also that a fractional change of 0.25 (25%) results in an altered phenotype. Each study involved a sample of 9 subjects, and each reported a 90% confidence interval for the fractional change in expression of the gene. For each drug, these are the sample mean Formula 2, sample standard deviation s, P value, and 90% confidence interval:

Formula 2

How do you interpret these results?

Drug A increased expression by 80%, a change that differed from 0 (P = 0.005). The confidence interval suggests the true impact of drug A is probably a 36–123% increase in expression, a change that is scientifically meaningful. Drug A produced a convincing change of scientific importance.

Drug B increased expression by 1%, a change that also differed from 0 (P = 0.005). The confidence interval suggests the true impact of drug B is probably a 0.4–1% increase in expression, a change that is scientifically trivial but quite precise. Drug B produced a convincing change of no scientific importance.

Drug C increased expression by 80%, a change consistent with 0 (P = 0.14). The confidence interval suggests the true impact of drug C could range from a 51% decrease to a 210% increase in expression. Either would be scientifically meaningful. Because it is relatively long, the confidence interval for drug C is an imprecise estimate of the true impact of drug C on expression of the gene. Drug C bears further study using a larger sample size.

Note that the scientific importance of the upper and lower bounds of a confidence interval depends on scientific context.

Summary

As this exploration has demonstrated, a confidence interval is a range that we expect, with some level of confidence, to include the true value of a population parameter such as the mean. For example, when we construct a confidence interval for the mean of some population, we assign numerical limits to the expected discrepancy between the sample mean Formula 2 and the population mean µ. A confidence interval is useful because it focuses our attention away from a singularly statistical P value and toward the scientific importance of some experimental result.

In the next installment of this series, we will explore bootstrapping, a statistical technique even more recent than confidence intervals. Bootstrapping gives us an approach we can use to assess whether the inferences we make from hypothesis tests and confidence intervals are justified.

Footnotes

1 A parameter is a numerical constant: for example, the population mean. Back

2 Snedecor published the early editions of Statistical Methods in 1937, 1938, and 1940. William Cochran contributed a chapter to the 1956 edition and helped author the 1967, 1980, and 1989 editions. Back

3 This file is available through the Supplemental Material link for this article at the Advances in Physiology Education website. Back

4 The command in line 75 of Advances_Statistics_Code_CI.R returns these values. Your values will differ slightly. Back

5 The standard error of the sample mean SE{Formula 2} is identical to the standard deviation of the theoretical distribution of the sample mean SD{Formula 2} in Eq. 1. Back

Received for publication January 28, 2009. Accepted for publication March 6, 2009.

REFERENCES

  1. Altman DG. Statistics in medical journals: developments in the 1980s. Stat Med 10: 1897–1913, 1991.[Web of Science][Medline]
  2. Altman DG. Statistics in medical journals: some recent trends. Stat Med 19: 3275–3289, 2000.[CrossRef][Web of Science][Medline]
  3. Altman DG, Machin D, Bryant TN, Gardner MJ. Statistics with Confidence. Bristol: BMJ Books, 2000.
  4. Altman DG, Gore SM, Gardner MJ, Pocock SJ. Statistical guidelines for contributors to medical journals. Br Med J 286: 1489–1493, 1983.[Free Full Text]
  5. Bailar JC III, Mosteller F. Guidelines for statistical reporting in articles for medical journals. Ann Intern Med 108: 266–273, 1988.[Abstract/Free Full Text]
  6. Bancroft TA. George W. Snedecor, pioneer statistician, 1881–1974. Am Stat 28: 108–109, 1974.
  7. Belia S, Fidler F, Williams J, Cumming G. Researchers misunderstand confidence intervals and standard error bars. Psychol Methods 10: 389–396, 2005.[CrossRef][Web of Science][Medline]
  8. Cochran WG. George W Snedecor. J R Stat Soc Ser A 137: 456–457, 1974.
  9. Council of Science Editors, Style Manual Subcommittee. Scientific Style and Format: the CSE Manual for Authors, Editors, and Publishers (7th ed.). Reston, VA: Rockefeller Univ. Press, 2006.
  10. Cox GM, Homeyer PG. Professional and personal glimpses of George W. Snedecor. Biometrics 31: 265–301, 1975.[CrossRef][Web of Science]
  11. Curran-Everett D. Explorations in statistics: hypothesis tests and P values. Adv Physiol Educ; doi:10.1152/advan.90218.2008.[Abstract/Free Full Text]
  12. Curran-Everett D. Explorations in statistics: standard deviations and standard errors. Adv Physiol Educ 32: 203–208, 2008.[Abstract/Free Full Text]
  13. Curran-Everett D, Benos DJ. Guidelines for reporting statistics in journals published by the American Physiological Society. Am J Physiol Endocrinol Metab 287: E189–E191, 2004.[Free Full Text]
  14. Curran-Everett D, Benos DJ. Last Word on Perspectives "Guidelines for reporting statistics in journals published by the American Physiological Society: the sequel". Adv Physiol Educ 31: 306–307, 2007.[Free Full Text]
  15. Curran-Everett D, Taylor S, Kafadar K. Fundamental concepts in statistics: elucidation and illustration. J Appl Physiol 85: 775–786, 1998.[Abstract/Free Full Text]
  16. Fidler F, Thomason N, Cumming G, Finch S, Leeman J. Editors can lead researchers to confidence intervals, but can't make them think: statistical reform lessons from medicine. Psychol Sci 15: 119–126, 2004.[CrossRef][Web of Science][Medline]
  17. Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J 292: 746–750, 1986.[Abstract/Free Full Text]
  18. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Ann Intern Med 108: 258–265, 1988.[Abstract/Free Full Text]
  19. Kempthorne O. George W. Snedecor, 1881–1974. Int Stat Rev 42: 319–323, 1974.
  20. Neyman J. On the two different aspects of the representative method. J R Stat Soc Ser A 97: 558–625, 1934.
  21. Neyman J. On the problem of confidence intervals. Ann Math Stat 6: 111–116, 1935.[CrossRef]
  22. Neyman J. Outline of a theory of statistical estimation based on the classical theory of probability. Philos Trans R Soc Lond A 236: 333–380, 1937.[Free Full Text]
  23. R Development Core Team. R: a Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing, 2008; http://www.R-project.org.
  24. Snedecor GW. Statistical Methods Applied to Experiments in Agriculture and Biology (4th ed.). Ames, IA: Iowa State College Press, 1946.
  25. Stigler SM. Statistics on the Table: the History of Statistical Concepts and Methods. Cambridge, MA: Harvard Univ. Press, 1999.
  26. Yates F, Mather K. Ronald Aylmer Fisher, 1890–1962. Biogr Mem Fellows R Soc Lond 9: 91–120, 1963.[CrossRef]



This article has been cited by other articles:


Home page
 Adv. Physiol. Educ.Home page
D. Curran-Everett
Explorations in statistics: the bootstrap
Advan Physiol Educ, December 1, 2009; 33(4): 286 - 292.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Supplemental Material
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Curran-Everett, D.
Right arrow Search for Related Content
PubMed
Right arrow Articles by Curran-Everett, D.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Visit Other APS Journals Online
Copyright © 2009 by the American Physiological Society.