In statistics , censoring is a condition in which the value of a measurement or observation is only partially known.
52-424: KME may refer to: Kaplan–Meier estimator , estimates the fraction of patients living for a certain amount of time after treatment. KME Smart , Iot platform, Turkish company. Kappa Mu Epsilon Kilmore East railway station , Australia KME Group , Italian company Kovatch Mobile Equipment Corp VEB Kombinat Mikroelektronik Erfurt [ de ] ,
104-974: A ) {\displaystyle Pr(a<x\leqslant b)=F(b)-F(a)} , where F ( x ) {\displaystyle F(x)} is the CDF of the probability distribution, and the two special cases are: For continuous probability distributions: P r ( a < x ⩽ b ) = P r ( a < x < b ) {\displaystyle Pr(a<x\leqslant b)=Pr(a<x<b)} Suppose we are interested in survival times, T 1 , T 2 , . . . , T n {\displaystyle T_{1},T_{2},...,T_{n}} , but we don't observe T i {\displaystyle T_{i}} for all i {\displaystyle i} . Instead, we observe When T i > U i , U i {\displaystyle T_{i}>U_{i},U_{i}}
156-511: A "plug-in estimator" where each q ( s ) {\displaystyle q(s)} is estimated based on the data and the estimator of S ( t ) {\displaystyle S(t)} is obtained as a product of these estimates. It remains to specify how q ( s ) = 1 − Prob ( τ = s ∣ τ ≥ s ) {\displaystyle q(s)=1-\operatorname {Prob} (\tau =s\mid \tau \geq s)}
208-475: A better use of all the data. This is what the Kaplan–Meier estimator accomplishes. Note that the naive estimator cannot be improved when censoring does not take place; so whether an improvement is possible critically hinges upon whether censoring is in place. By elementary calculations, where the second to last equality used that τ {\displaystyle \tau } is integer valued and for
260-400: A certain amount of time after treatment. In other fields, Kaplan–Meier estimators may be used to measure the length of time people remain unemployed after a job loss, the time-to-failure of machine parts, or how long fleshy fruits remain on plants before they are removed by frugivores . The estimator is named after Edward L. Kaplan and Paul Meier , who each submitted similar manuscripts to
312-640: A conglomerate of microelectronic design and development facilities around the VEB Mikroelektronik "Karl Marx" Erfurt (MME) in the former East Germany Topics referred to by the same term [REDACTED] This disambiguation page lists articles associated with the title KME . If an internal link led you here, you may wish to change the link to point directly to the intended article. Retrieved from " https://en.wikipedia.org/w/index.php?title=KME&oldid=1250826421 " Category : Disambiguation pages Hidden categories: Short description
364-480: A function of parameters in an assumed model. To incorporate censored data points in the likelihood the censored data points are represented by the probability of the censored data points as a function of the model parameters given a model, i.e. a function of CDF(s) instead of the density or probability mass. The most general censoring case is interval censoring: P r ( a < x ⩽ b ) = F ( b ) − F (
416-529: A given range: values in the population outside the range are never seen or never recorded if they are seen. Note that in statistics, truncation is not the same as rounding . Interval censoring can occur when observing a value requires follow-ups or inspections. Left and right censoring are special cases of interval censoring, with the beginning of the interval at zero or the end at infinity, respectively. Estimation methods for using left-censored data vary, and not all methods of estimation may be applicable to, or
468-525: A patient withdraws from a study, is lost to follow-up , or is alive without event occurrence at last follow-up. On the plot, small vertical tick-marks state individual patients whose survival times have been right-censored. When no truncation or censoring occurs, the Kaplan–Meier curve is the complement of the empirical distribution function . In medical statistics , a typical application might involve grouping patients into categories, for instance, those with Gene A profile and those with Gene B profile. In
520-414: A similar reasoning that lead to the construction of the naive estimator above, we arrive at the estimator (think of estimating the numerator and denominator separately in the definition of the "hazard rate" Prob ( τ = s | τ ≥ s ) {\displaystyle \operatorname {Prob} (\tau =s|\tau \geq s)} ). The Kaplan–Meier estimator
572-409: A situation could occur if the individual withdrew from the study at age 75, or if the individual is currently alive at the age of 75. Censoring also occurs when a value occurs outside the range of a measuring instrument . For example, a bathroom scale might only measure up to 140 kg. If a 160 kg individual is weighed using the scale, the observer would only know that the individual's weight
SECTION 10
#1732852356022624-446: Is a statistic , and several estimators are used to approximate its variance . One of the most common estimators is Greenwood's formula: where d i {\displaystyle d_{i}} is the number of cases and n i {\displaystyle n_{i}} is the total number of observations, for t i < t {\displaystyle t_{i}<t} . Greenwood's formula
676-424: Is a fixed, deterministic integer, the censoring time of event j {\displaystyle j} and τ ~ j = min ( τ j , c j ) {\displaystyle {\tilde {\tau }}_{j}=\min(\tau _{j},c_{j})} . In particular, the information available about the timing of event j {\displaystyle j}
728-487: Is at least 140 kg. The problem of censored data, in which the observed value of some variable is partially known, is related to the problem of missing data , where the observed value of some variable is unknown. Censoring should not be confused with the related idea truncation . With censoring, observations result either in knowing the exact value that applies, or in knowing that the value lies within an interval . With truncation, observations never result in values outside
780-471: Is called the censoring time . If the censoring times are all known constants, then the likelihood is where f ( u i ) {\displaystyle f(u_{i})} = the probability density function evaluated at u i {\displaystyle u_{i}} , and S ( u i ) {\displaystyle S(u_{i})} = the probability that T i {\displaystyle T_{i}}
832-1194: Is derived by noting that probability of getting d i {\displaystyle d_{i}} failures out of n i {\displaystyle n_{i}} cases follows a binomial distribution with failure probability h i {\displaystyle h_{i}} . As a result for maximum likelihood hazard rate h ^ i = d i / n i {\displaystyle {\widehat {h}}_{i}=d_{i}/n_{i}} we have E ( h ^ i ) = h i {\displaystyle E\left({\widehat {h}}_{i}\right)=h_{i}} and Var ( h ^ i ) = h i ( 1 − h i ) / n i {\displaystyle \operatorname {Var} \left({\widehat {h}}_{i}\right)=h_{i}(1-h_{i})/n_{i}} . To avoid dealing with multiplicative probabilities we compute variance of logarithm of S ^ ( t ) {\displaystyle {\widehat {S}}(t)} and will use
884-399: Is different from Wikidata All article disambiguation pages All disambiguation pages Kaplan%E2%80%93Meier estimator The Kaplan–Meier estimator , also known as the product limit estimator , is a non-parametric statistic used to estimate the survival function from lifetime data. In medical research, it is often used to measure the fraction of patients living for
936-498: Is given by: with t i {\displaystyle t_{i}} a time when at least one event happened, d i the number of events (e.g., deaths) that happened at time t i {\displaystyle t_{i}} , and n i {\displaystyle n_{i}} the individuals known to have survived (have not yet had an event or been censored) up to time t i {\displaystyle t_{i}} . A plot of
988-798: Is greater than u i {\displaystyle u_{i}} , called the survival function . This can be simplified by defining the hazard function , the instantaneous force of mortality, as so Then For the exponential distribution , this becomes even simpler, because the hazard rate, λ {\displaystyle \lambda } , is constant, and S ( u ) = exp ( − λ u ) {\displaystyle S(u)=\exp(-\lambda u)} . Then: where k = ∑ δ i {\displaystyle k=\sum {\delta _{i}}} . From this we easily compute λ ^ {\displaystyle {\hat {\lambda }}} ,
1040-428: Is large, which, through S ( t ) = 1 − Prob ( τ ≤ t ) {\displaystyle S(t)=1-\operatorname {Prob} (\tau \leq t)} means that S ( t ) {\displaystyle S(t)} must be small. However, this information is ignored by this naive estimator. The question is then whether there exists an estimator that makes
1092-761: Is not random and so neither is m ( t ) {\displaystyle m(t)} . Furthermore, ( X k ) k ∈ C ( t ) {\displaystyle (X_{k})_{k\in C(t)}} is a sequence of independent, identically distributed Bernoulli random variables with common parameter S ( t ) = Prob ( τ ≥ t ) {\displaystyle S(t)=\operatorname {Prob} (\tau \geq t)} . Assuming that m ( t ) > 0 {\displaystyle m(t)>0} , this suggests to estimate S ( t ) {\displaystyle S(t)} using where
SECTION 20
#17328523560221144-872: Is small, which happens, by definition, when a lot of the events are censored. A particularly unpleasant property of this estimator, that suggests that perhaps it is not the "best" estimator, is that it ignores all the observations whose censoring time precedes t {\displaystyle t} . Intuitively, these observations still contain information about S ( t ) {\displaystyle S(t)} : For example, when for many events with c k < t {\displaystyle c_{k}<t} , τ k < c k {\displaystyle \tau _{k}<c_{k}} also holds, we can infer that events often happen early, which implies that Prob ( τ ≤ t ) {\displaystyle \operatorname {Prob} (\tau \leq t)}
1196-515: Is sometimes called hazard , or mortality rates . However, before doing this it is worthwhile to consider a naive estimator. To understand the power of the Kaplan–Meier estimator, it is worthwhile to first describe a naive estimator of the survival function. Fix k ∈ [ n ] := { 1 , … , n } {\displaystyle k\in [n]:=\{1,\dots ,n\}} and let t > 0 {\displaystyle t>0} . A basic argument shows that
1248-480: Is that of τ {\displaystyle \tau } : τ j {\displaystyle \tau _{j}} is the random time when some event j {\displaystyle j} happened. The data available for estimating S {\displaystyle S} is not ( τ j ) j = 1 , … , n {\displaystyle (\tau _{j})_{j=1,\dots ,n}} , but
1300-745: Is the number of known deaths at time s {\displaystyle s} , while n ( s ) = | { 1 ≤ k ≤ n : τ ~ k ≥ s } | {\displaystyle n(s)=|\{1\leq k\leq n\,:\,{\tilde {\tau }}_{k}\geq s\}|} is the number of those persons who are alive (and not being censored) at time s − 1 {\displaystyle s-1} . Note that if d ( s ) = 0 {\displaystyle d(s)=0} , q ^ ( s ) = 1 {\displaystyle {\hat {q}}(s)=1} . This implies that we can leave out from
1352-583: Is then given by The form of the estimator stated at the beginning of the article can be obtained by some further algebra. For this, write q ^ ( s ) = 1 − d ( s ) / n ( s ) {\displaystyle {\hat {q}}(s)=1-d(s)/n(s)} where, using the actuarial science terminology, d ( s ) = | { 1 ≤ k ≤ n : τ k = s } | {\displaystyle d(s)=|\{1\leq k\leq n\,:\,\tau _{k}=s\}|}
1404-996: Is to be estimated. By Proposition 1, for any k ∈ [ n ] {\displaystyle k\in [n]} such that c k ≥ s {\displaystyle c_{k}\geq s} , Prob ( τ = s ) = Prob ( τ ~ k = s ) {\displaystyle \operatorname {Prob} (\tau =s)=\operatorname {Prob} ({\tilde {\tau }}_{k}=s)} and Prob ( τ ≥ s ) = Prob ( τ ~ k ≥ s ) {\displaystyle \operatorname {Prob} (\tau \geq s)=\operatorname {Prob} ({\tilde {\tau }}_{k}\geq s)} both hold. Hence, for any k ∈ [ n ] {\displaystyle k\in [n]} such that c k ≥ s {\displaystyle c_{k}\geq s} , By
1456-458: Is used to denote maximum likelihood estimation. Given this result, we can write: More generally (for continuous as well as discrete survival distributions), the Kaplan-Meier estimator may be interpreted as a nonparametric maximum likelihood estimator. The Kaplan–Meier estimator is one of the most frequently used methods of survival analysis. The estimate may be useful to examine recovery rates,
1508-414: Is whether the event happened before the fixed time c j {\displaystyle c_{j}} and if so, then the actual time of the event is also available. The challenge is to estimate S ( t ) {\displaystyle S(t)} given this data. Two derivations of the Kaplan–Meier estimator are shown. Both are based on rewriting the survival function in terms of what
1560-542: The Journal of the American Statistical Association . The journal editor, John Tukey , convinced them to combine their work into one paper, which has been cited more than 34,000 times since its publication in 1958. The estimator of the survival function S ( t ) {\displaystyle S(t)} (the probability that life is longer than t {\displaystyle t} )
1612-530: The Cox proportional hazards test . Other statistics that may be of use with this estimator are pointwise confidence intervals, the Hall-Wellner band and the equal-precision band. Censoring (statistics) For example, suppose a study is conducted to measure the impact of a drug on mortality rate . In such a study, it may be known that an individual's age at death is at least 75 years (but may be more). Such
KME - Misplaced Pages Continue
1664-534: The Kaplan–Meier estimator for estimating censored costs was Quesenberry et al. (1989), however this approach was found to be invalid by Lin et al. unless all patients accumulated costs with a common deterministic rate function over time, they proposed an alternative estimation technique known as the Lin estimator. Reliability testing often consists of conducting a test on an item (under specified conditions) to determine
1716-408: The delta method to convert it back to the original variance: using martingale central limit theorem , it can be shown that the variance of the sum in the following equation is equal to the sum of variances: as a result we can write: using the delta method once more: as desired. In some cases, one may wish to compare different Kaplan–Meier curves. This can be done by the log rank test , and
1768-405: The maximum likelihood estimate (MLE) of λ {\displaystyle \lambda } , as follows: Then We set this to 0 and solve for λ {\displaystyle \lambda } to get: Equivalently, the mean time to failure is: This differs from the standard MLE for the exponential distribution in that the any censored observations are considered only in
1820-460: The Kaplan–Meier estimator is a series of declining horizontal steps which, with a large enough sample size, approaches the true survival function for that population. The value of the survival function between successive distinct sampled observations ("clicks") is assumed to be constant. An important advantage of the Kaplan–Meier curve is that the method can take into account some types of censored data , particularly right-censoring , which occurs if
1872-450: The beginning of the article: As opposed to the naive estimator, this estimator can be seen to use the available information more effectively: In the special case mentioned beforehand, when there are many early events recorded, the estimator will multiply many terms with a value below one and will thus take into account that the survival probability cannot be large. Kaplan–Meier estimator can be derived from maximum likelihood estimation of
1924-420: The discrete hazard function . More specifically given d i {\displaystyle d_{i}} as the number of events and n i {\displaystyle n_{i}} the total individuals at risk at time t i {\displaystyle t_{i}} , discrete hazard rate h i {\displaystyle h_{i}} can be defined as
1976-394: The events for which the outcome was not censored before time t {\displaystyle t} . Let m ( t ) = | C ( t ) | {\displaystyle m(t)=|C(t)|} be the number of elements in C ( t ) {\displaystyle C(t)} . Note that the set C ( t ) {\displaystyle C(t)}
2028-689: The following proposition holds: Let k {\displaystyle k} be such that c k ≥ t {\displaystyle c_{k}\geq t} . It follows from the above proposition that Let X k = I ( τ ~ k ≥ t ) {\displaystyle X_{k}=\mathbb {I} ({\tilde {\tau }}_{k}\geq t)} and consider only those k ∈ C ( t ) := { k : c k ≥ t } {\displaystyle k\in C(t):=\{k\,:\,c_{k}\geq t\}} , i.e.
2080-457: The graph, patients with Gene B die much quicker than those with Gene A. After two years, about 80% of the Gene A patients survive, but less than half of patients with Gene B. To generate a Kaplan–Meier estimator, at least two pieces of data are required for each patient (or each subject): the status at last observation (event occurrence or right-censored), and the time to event (or time to censoring). If
2132-630: The last line we introduced By a recursive expansion of the equality S ( t ) = q ( t ) S ( t − 1 ) {\displaystyle S(t)=q(t)S(t-1)} , we get Note that here q ( 0 ) = 1 − Prob ( τ = 0 ∣ τ > − 1 ) = 1 − Prob ( τ = 0 ) {\displaystyle q(0)=1-\operatorname {Prob} (\tau =0\mid \tau >-1)=1-\operatorname {Prob} (\tau =0)} . The Kaplan–Meier estimator can be seen as
KME - Misplaced Pages Continue
2184-504: The list of pairs ( ( τ ~ j , c j ) ) j = 1 , … , n {\displaystyle (\,({\tilde {\tau }}_{j},c_{j})\,)_{j=1,\dots ,n}} where for j ∈ [ n ] := { 1 , 2 , … , n } {\displaystyle j\in [n]:=\{1,2,\dots ,n\}} , c j ≥ 0 {\displaystyle c_{j}\geq 0}
2236-519: The most reliable, for all data sets. A common misconception with time interval data is to class as left censored intervals where the start time is unknown. In these cases we have a lower bound on the time interval , thus the data is right censored (despite the fact that the missing start point is to the left of the known interval when viewed as a timeline!). Special techniques may be used to handle censored data. Tests with specific failure times are coded as actual failures; censored data are coded for
2288-472: The probability of an individual with an event at time t i {\displaystyle t_{i}} . Then survival rate can be defined as: and the likelihood function for the hazard function up to time t i {\displaystyle t_{i}} is: therefore the log likelihood will be: finding the maximum of log likelihood with respect to h i {\displaystyle h_{i}} yields: where hat
2340-520: The probability of death, and the effectiveness of treatment. It is limited in its ability to estimate survival adjusted for covariates ; parametric survival models and the Cox proportional hazards model may be useful to estimate covariate-adjusted survival. The Kaplan-Meier estimator is directly related to the Nelson-Aalen estimator and both maximize the empirical likelihood . The Kaplan–Meier estimator
2392-422: The product defining S ^ ( t ) {\displaystyle {\hat {S}}(t)} all those terms where d ( s ) = 0 {\displaystyle d(s)=0} . Then, letting 0 ≤ t 1 < t 2 < ⋯ < t m {\displaystyle 0\leq t_{1}<t_{2}<\dots <t_{m}} be
2444-530: The second equality follows because τ ~ k ≥ t {\displaystyle {\tilde {\tau }}_{k}\geq t} implies c k ≥ t {\displaystyle c_{k}\geq t} , while the last equality is simply a change of notation. The quality of this estimate is governed by the size of m ( t ) {\displaystyle m(t)} . This can be problematic when m ( t ) {\displaystyle m(t)}
2496-400: The survival functions between two or more groups are to be compared, then a third piece of data is required: the group assignment of each subject. Let τ ≥ 0 {\displaystyle \tau \geq 0} be a random variable as the time that passes between the start of the possible exposure period, t 0 {\displaystyle t_{0}} , and
2548-411: The time it takes for a failure to occur. An analysis of the data from replicate tests includes both the times-to-failure for the items that failed and the time-of-test-termination for those that did not fail. An earlier model for censored regression , the tobit model , was proposed by James Tobin in 1958. The likelihood is the probability or probability density of what was observed, viewed as
2600-591: The time that the event of interest takes place, t 1 {\displaystyle t_{1}} . As indicated above, the goal is to estimate the survival function S {\displaystyle S} underlying τ {\displaystyle \tau } . Recall that this function is defined as Let τ 1 , … , τ n ≥ 0 {\displaystyle \tau _{1},\dots ,\tau _{n}\geq 0} be independent, identically distributed random variables, whose common distribution
2652-429: The times s {\displaystyle s} when d ( s ) > 0 {\displaystyle d(s)>0} , d i = d ( t i ) {\displaystyle d_{i}=d(t_{i})} and n i = n ( t i ) {\displaystyle n_{i}=n(t_{i})} , we arrive at the form of the Kaplan–Meier estimator given at
SECTION 50
#17328523560222704-447: The type of censoring and the known interval or limit. Special software programs (often reliability oriented) can conduct a maximum likelihood estimation for summary statistics, confidence intervals, etc. One of the earliest attempts to analyse a statistical problem involving censored data was Daniel Bernoulli 's 1766 analysis of smallpox morbidity and mortality data to demonstrate the efficacy of vaccination . An early paper to use
#21978