One of the most important aspects of good research is proper study design; no type or amount of statistical analysis can make up for a poorly designed study. In longitudinal research one of the most important aspects of study design is making sure that the temporal sequence of events is understood and recorded correctly. Errors in this arena can lead to misclassification of exposure time and differential follow-up between comparison groups. It is therefore crucial to understand time at risk in studies and correctly and fairly apply rules of follow-up to study subjects.
In experimental longitudinal research (e.g. clinical trials) understanding what constitutes time at risk can be quite straightforward. In the prototypic design, patients are recruited at diagnosis or progression of a specific disease and then are randomized to receive an intervention or placebo. Supposing that a patient meets the diagnostic criteria for inclusion in the trial, the time at risk usually begins at randomization (i.e. at the start of therapy). Following such a simple procedure ensures that the survival times in each group are unbiased, and thus, validly comparable.1
But what about observational studies, where the treatments are not randomized and the data are often historical in nature? Because there is no experimental control over who received which therapies and when, researchers must be more careful about defining study cohorts and periods at risk, lest immortal time gets included in the follow-up period, biasing results and potentially invalidating inferences.
So what is immortal time, and how does it creep into observational longitudinal research? Simply put, immortal time is any amount of time during which, by study design, a study subject could not die. The immortal time bias (ITB) occurs when immortal time is not recognized and is differentially and systematically included in the follow-up for some subjects but not others. We will illustrate the concept with some examples we have recently observed in various peer-reviewed publications.
In a study of Hollywood actors and actresses, Redelmeier and Singh2 found that those who won an Oscar tended to live longer than those who did not. A subsequent reanalysis of the data by Sylvestre et al.3 found that immortal time was introduced for Oscar winners in the original study because the authors considered start points for follow-up such as birth or release of a performer’s first film. In so doing, the authors counted immortal time in the analysis for the Oscar winners: the winners were guaranteed to survive at least as long as it takes to win an Oscar, whereas the non-winners had no such guarantee. The corrected analysis found no significant difference in survival times between Oscar-winners and non-winners, meaning the original results were purely an artifact of ITB.
Another example of ITB can be found in a study of the life expectancy of the mentally ill in England.4 The authors reported life expectancies at birth for the mentally ill. The problem is that a diagnosis of mental illness typically does not occur until at least adolescence/young adulthood, sometimes not until well into the early 20s for diagnoses such as schizophrenia. This means “patients with mental illness” are guaranteed to have lived at least long enough to receive a diagnosis. The authors attempted to correct for this immortal time bias by assuming that mortality rates for mentally ill patients prior to diagnosis would be those of the general population. The effect of this choice is unclear however; there is no way to know whether or not people who eventually might develop diagnoses of mental illness have mortality rates that are greater or less than those of the general population. Unless the mortality rates are the same as the general population rates the only thing we can know for sure is that the resulting life expectancy estimates are biased, albeit perhaps only marginally so. The authors repeated this potentially biased analysis in a more recent article dealing with the life expectancy of patients with personality disorders.5
Two more examples are found in studies of the mortality of NBA basketball players.6-7 In the first study, Lawler & Lawler examine survival between left- and right-handed basketball players, including computing life expectancies.6 The paper is not explicit on the age to which those estimates apply, but given that the estimates of life expectancy are in the high 70-year range, it is reasonable to assume they are estimates of life expectancy at birth, meaning that immortal time from birth until age at NBA selection is included. In the second paper, Lawler et al. compare survival curves for NBA players with those of age- and race-matched males in the general population.7 Here the survival curves start at birth, and again the period from birth up until entry into the NBA is immortal time. This can be seen clearly in their Figure 3 (reproduced below), where the general population survival curves begin to decline immediately, while the NBA curves remain at 100% until age 20 or later. Because life expectancy is the area under the respective survival curves, any life expectancy estimated for NBA players from such curves would be biased upward because of inclusion of immortal time, just as with life expectancies based on handedness.
ITB errors are undoubtedly embarrassing for those who commit them. However, these errors can sometimes have damaging long-term consequences beyond tarnishing one’s reputation. In clinical practice ITB errors have the potential to misinform clinical decision making when they are committed in the context of evaluating observational evidence for treatment efficacy of drugs and therapies. For example, ITB errors were discovered in a study of statins for the prevention of progression of diabetes8 and in a study of multidisciplinary care for chronic kidney disease.9 In each of these cases a biased estimate of outcome frequencies generated a biased assessment of treatment efficacy. This had the potential of encouraging widespread use of ineffective or even inferior therapies.
Another example of the damaging consequences of ITB errors is found in a study of children with neurological disabilities, including cerebral palsy (CP), published in 1998 by Plioplys et al.10 While the errors in that paper have been covered in detail in a three-part series beginning here, a brief restatement of the errors is illustrative. The authors conducted 10 years of follow-up on a group of children in three different age groups, those <1 year, > 1year, and >15 years. Careful inspection of the survival curves in the paper reveals that all three groups have survival curves that span up to age 34 – in a study with only 10 years of follow-up. When computing the survival for the youngest two groups the authors apparently included all children in the computation, even those who were in the older groups when the study observation period started. This is a classic ITB error: we know that the children who were 15 at the time the study began had to have survived to that age in order to be observed, making all years lived before age 15 immortal time. As a consequence the Plioplys et al. paper reported 8-year survival probabilities that were much higher than had been previously reported. In spite of these errors this paper continues to be cited in litigation as evidence that better quality care can result in normal or near-normal life expectancies for children with even the most severe disabilities.
ITB errors are regrettably common in the scientific literature. Fortunately they are easy to prevent with a modicum of diligence in study planning and execution. Unfortunately they can be difficult to recognize once published, and have the potential to misinform medical and policy decisions when they go undetected. As scientists and consumers of the knowledge that survival studies provide it is up to all of us to examine the evidence carefully, ask the appropriate questions, and be willing to be honest about errors if and when we commit them or publish them.
References:
- Friedman LM, Furberg CD, DeMets DL. Fundamentals of Clinical Trials, 4th Edition. New York: Springer; 2010.
- Redelmeier DA, Singh SM. Survival in Academy Award-winning actors and actresses. Ann Intern Med. 2001 May 15;134(10):955-62.
- Sylvestre M, Huszti E, Hanley JA. Do Oscar Winners Live Longer than Less Successful Peers? A Reanalysis of the Evidence. Ann Intern Med. 2006 Sep 5;145(5):361-3.
- Chang CK, Hayes RD, Perera G, Broadbent MT, Fernandes AC, Lee WE, Hotopf M, Stewart R. Life expectancy at birth for people with serious mental illness and other major disorders from a secondary mental health care case register in London. PLoS One. 2011;6(5):e19590.
- Fok ML, Hayes RD, Chang CK, Stewart R, Callard FJ, Moran P. Life expectancy at birth and all-cause mortality among people with personality disorder. J Psychosom Res. 2012 Aug;73(2):104-7.
- Lawler TP, Lawler FH. Left-handedness in professional basketball: prevalence, performance, and survival. Percept Mot Skills. 2011 Dec;113(3):815-24.
- Lawler T, Lawler F, Gibson J, Murray R. Does the African-American-white mortality gap persist after playing professional basketball? A 59-year historical cohort study. Ann Epidemiol. 2012 Jun;22(6):406-12.
- Lévesque LE, Hanley JA, Kezouh A, Suissa S. Problem of immortal time bias in cohort studies: example using statins for preventing progression of diabetes. BMJ. 2010 Mar 12;340:b5087.
- Shariff SZ, Cuerden MS, Jain AK, Garg AX. The secret of immortal time bias in epidemiologic studies. J Am Soc Nephrol. 2008 May;19(5):841-3.
- Plioplys AV, Kasnicka I, Lewis S, Moller D. Survival rates among children with severe neurologic disabilities. South Med J. 1998 Feb;91(2):161-72.
Author
Related posts:
- Losing sleep over denominators, Part III: Explaining the error and the bias in Plioplys 1998
- Losing sleep over denominators, Part II: The survival curves in Plioplys et al. 1998 cannot be right
- Losing sleep over denominators, Part I: An introduction to the problem of Plioplys 1998
- Are doctors experts on life expectancy?