Misplaced Pages

#47952

124-432: In probability theory , a probability density function ( PDF ), density function , or density of an absolutely continuous random variable , is a function whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can be interpreted as providing a relative likelihood that the value of the random variable would be equal to that sample. Probability density

248-810: A k μ ( S k ∩ B ) . {\displaystyle \int _{B}s\,\mathrm {d} \mu =\int 1_{B}\,s\,\mathrm {d} \mu =\sum _{k}a_{k}\,\mu (S_{k}\cap B).} Let f be a non-negative measurable function on E , which we allow to attain the value +∞ , in other words, f takes non-negative values in the extended real number line . We define ∫ E f d μ = sup { ∫ E s d μ : 0 ≤ s ≤ f ,   s   simple } . {\displaystyle \int _{E}f\,d\mu =\sup \left\{\,\int _{E}s\,d\mu :0\leq s\leq f,\ s\ {\text{simple}}\,\right\}.} We need to show this integral coincides with

372-424: A k 1 S k {\displaystyle \sum _{k}a_{k}1_{S_{k}}} where the coefficients a k are real numbers and S k are disjoint measurable sets, is called a measurable simple function . We extend the integral by linearity to non-negative measurable simple functions. When the coefficients a k are positive, we set ∫ ( ∑ k

496-440: A k 1 S k ) d μ = ∑ k a k ∫ 1 S k d μ = ∑ k a k μ ( S k ) {\displaystyle \int \left(\sum _{k}a_{k}1_{S_{k}}\right)\,d\mu =\sum _{k}a_{k}\int 1_{S_{k}}\,d\mu =\sum _{k}a_{k}\,\mu (S_{k})} whether this sum

620-670: A ≤ X ≤ b ] = ∫ a b f X ( x ) d x . {\displaystyle \Pr[a\leq X\leq b]=\int _{a}^{b}f_{X}(x)\,dx.} Hence, if F X {\displaystyle F_{X}} is the cumulative distribution function of X {\displaystyle X} , then: F X ( x ) = ∫ − ∞ x f X ( u ) d u , {\displaystyle F_{X}(x)=\int _{-\infty }^{x}f_{X}(u)\,du,} and (if f X {\displaystyle f_{X}}

744-709: A counting measure over the set of all possible outcomes. Densities for absolutely continuous distributions are usually defined as this derivative with respect to the Lebesgue measure . If a theorem can be proved in this general setting, it holds for both discrete and continuous distributions as well as others; separate proofs are not required for discrete and continuous distributions. Certain random variables occur very often in probability theory because they well describe many natural or physical processes. Their distributions, therefore, have gained special importance in probability theory. Some fundamental discrete distributions are

868-465: A measure P {\displaystyle P\,} defined on F {\displaystyle {\mathcal {F}}\,} is called a probability measure if P ( Ω ) = 1. {\displaystyle P(\Omega )=1.\,} If F {\displaystyle {\mathcal {F}}\,} is the Borel σ-algebra on the set of real numbers, then there

992-465: A monotonic function , then the resulting density function is f Y ( y ) = f X ( g − 1 ( y ) ) | d d y ( g − 1 ( y ) ) | . {\displaystyle f_{Y}(y)=f_{X}{\big (}g^{-1}(y){\big )}\left|{\frac {d}{dy}}{\big (}g^{-1}(y){\big )}\right|.} Here g denotes

1116-467: A sequence of independent and identically distributed random variables X k {\displaystyle X_{k}} converges towards their common expectation (expected value) μ {\displaystyle \mu } , provided that the expectation of | X k | {\displaystyle |X_{k}|} is finite. It is in the different forms of convergence of random variables that separates

1240-502: A book on the subject in 1657. In the 19th century, what is considered the classical definition of probability was completed by Pierre Laplace . Initially, probability theory mainly considered discrete events, and its methods were mainly combinatorial . Eventually, analytical considerations compelled the incorporation of continuous variables into the theory. This culminated in modern probability theory, on foundations laid by Andrey Nikolaevich Kolmogorov . Kolmogorov combined

1364-730: A collapsed random variable with probability density function p Z ( z ) = δ ( z ) {\displaystyle p_{Z}(z)=\delta (z)} (i.e., a constant equal to zero). Let the random vector X ~ {\displaystyle {\tilde {X}}} and the transform H {\displaystyle H} be defined as H ( Z , X ) = [ Z + V ( X ) X ] = [ Y X ~ ] . {\displaystyle H(Z,X)={\begin{bmatrix}Z+V(X)\\X\end{bmatrix}}={\begin{bmatrix}Y\\{\tilde {X}}\end{bmatrix}}.} It

SECTION 10

#1732877035048

1488-636: A continuous sample space. Classical definition : The classical definition breaks down when confronted with the continuous case. See Bertrand's paradox . Modern definition : If the sample space of a random variable X is the set of real numbers ( R {\displaystyle \mathbb {R} } ) or a subset thereof, then a function called the cumulative distribution function ( CDF ) F {\displaystyle F\,} exists, defined by F ( x ) = P ( X ≤ x ) {\displaystyle F(x)=P(X\leq x)\,} . That is, F ( x ) returns

1612-489: A countable set), while the PDF is used in the context of continuous random variables. Suppose bacteria of a certain species typically live 20 to 30 hours. The probability that a bacterium lives exactly 5 hours is equal to zero. A lot of bacteria live for approximately 5 hours, but there is no chance that any given bacterium dies at exactly 5.00... hours. However, the probability that the bacterium dies between 5 hours and 5.01 hours

1736-665: A density function: the distributions of discrete random variables do not; nor does the Cantor distribution , even though it has no discrete component, i.e., does not assign positive probability to any individual point. A distribution has a density function if and only if its cumulative distribution function F ( x ) is absolutely continuous . In this case: F is almost everywhere differentiable , and its derivative can be used as probability density: d d x F ( x ) = f ( x ) . {\displaystyle {\frac {d}{dx}}F(x)=f(x).} If

1860-437: A differentiable function and X {\displaystyle X} be a random vector taking values in R n {\displaystyle \mathbb {R} ^{n}} , f X {\displaystyle f_{X}} be the probability density function of X {\displaystyle X} and δ ( ⋅ ) {\displaystyle \delta (\cdot )} be

1984-487: A discrete variable can take n different values among real numbers, then the associated probability density function is: f ( t ) = ∑ i = 1 n p i δ ( t − x i ) , {\displaystyle f(t)=\sum _{i=1}^{n}p_{i}\,\delta (t-x_{i}),} where x 1 , … , x n {\displaystyle x_{1},\ldots ,x_{n}} are

2108-607: A firm foundation. The Riemann integral —proposed by Bernhard Riemann (1826–1866)—is a broadly successful attempt to provide such a foundation. Riemann's definition starts with the construction of a sequence of easily calculated areas that converge to the integral of a given function. This definition is successful in the sense that it gives the expected answer for many already-solved problems, and gives useful results for many other problems. However, Riemann integration does not interact well with taking limits of sequences of functions, making such limiting processes difficult to analyze. This

2232-513: A function with respect to a general measure , as introduced by Lebesgue, or the specific case of integration of a function defined on a sub-domain of the real line with respect to the Lebesgue measure . The integral of a positive real function f between boundaries a and b can be interpreted as the area under the graph of f , between a and b . This notion of area fits some functions, mainly piecewise continuous functions, including elementary functions , for example polynomials . However,

2356-407: A given distribution, the parameters are constants, and terms in a density function that contain only parameters, but not variables, are part of the normalization factor of a distribution (the multiplicative factor that ensures that the area under the density—the probability of something in the domain occurring— equals 1). This normalization factor is outside the kernel of the distribution. Since

2480-479: A joint density are all independent from each other if and only if f X 1 , … , X n ( x 1 , … , x n ) = f X 1 ( x 1 ) ⋯ f X n ( x n ) . {\displaystyle f_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})=f_{X_{1}}(x_{1})\cdots f_{X_{n}}(x_{n}).} If

2604-408: A layer identifies a set of intervals in the domain of f , which, taken together, is defined to be the preimage of the lower bound of that layer, under the simple function. In this way, the partitioning of the range of f implies a partitioning of its domain. The integral of a simple function is found by summing, over these (not necessarily connected) subsets of the domain, the product of the measure of

SECTION 20

#1732877035048

2728-456: A mix, for example, the Cantor distribution has no positive probability for any single point, neither does it have a density. The modern approach to probability theory solves these problems using measure theory to define the probability space : Given any set Ω {\displaystyle \Omega \,} (also called sample space ) and a σ-algebra F {\displaystyle {\mathcal {F}}\,} on it,

2852-408: A probability distribution admits a density, then the probability of every one-point set { a } is zero; the same holds for finite and countable sets. Two probability densities f and g represent the same probability distribution precisely if they differ only on a set of Lebesgue measure zero . In the field of statistical physics , a non-formal reformulation of the relation above between

2976-629: A random fashion). Although it is not possible to perfectly predict random events, much can be said about their behavior. Two major results in probability theory describing such behaviour are the law of large numbers and the central limit theorem . As a mathematical foundation for statistics , probability theory is essential to many human activities that involve quantitative analysis of data. Methods of probability theory also apply to descriptions of complex systems given only partial knowledge of their state, as in statistical mechanics or sequential estimation . A great discovery of twentieth-century physics

3100-489: A random value from a normal distribution with probability 1/2. It can still be studied to some extent by considering it to have a PDF of ( δ [ x ] + φ ( x ) ) / 2 {\displaystyle (\delta [x]+\varphi (x))/2} , where δ [ x ] {\displaystyle \delta [x]} is the Dirac delta function . Other distributions may not even be

3224-471: A random variable X is given and its distribution admits a probability density function f , then the expected value of X (if the expected value exists) can be calculated as E ⁡ [ X ] = ∫ − ∞ ∞ x f ( x ) d x . {\displaystyle \operatorname {E} [X]=\int _{-\infty }^{\infty }x\,f(x)\,dx.} Not every probability distribution has

3348-410: A random variable (or vector) X is given as f X ( x ) , it is possible (but often not necessary; see below) to calculate the probability density function of some variable Y = g ( X ) . This is also called a "change of variable" and is in practice used to generate a random variable of arbitrary shape f g ( X ) = f Y using a known (for instance, uniform) random number generator. It

3472-407: A real number uniformly at random from the unit interval, the probability of picking a rational number should be zero. Lebesgue summarized his approach to integration in a letter to Paul Montel : I have to pay a certain sum, which I have collected in my pocket. I take the bills and coins out of my pocket and give them to the creditor in the order I find them until I have reached the total sum. This

3596-718: A reference for a continuous random variable). Furthermore, when it does exist, the density is almost unique, meaning that any two such densities coincide almost everywhere . Unlike a probability, a probability density function can take on values greater than one; for example, the continuous uniform distribution on the interval [0, 1/2] has probability density f ( x ) = 2 for 0 ≤ x ≤ 1/2 and f ( x ) = 0 elsewhere. The standard normal distribution has probability density f ( x ) = 1 2 π e − x 2 / 2 . {\displaystyle f(x)={\frac {1}{\sqrt {2\pi }}}\,e^{-x^{2}/2}.} If

3720-470: A theory of measurable functions and integrals on these functions. One approach to constructing the Lebesgue integral is to make use of so-called simple functions : finite, real linear combinations of indicator functions . Simple functions that lie directly underneath a given function f can be constructed by partitioning the range of f into a finite number of layers. The intersection of the graph of f with

3844-490: A value to the integral of the indicator function 1 S of a measurable set S consistent with the given measure μ , the only reasonable choice is to set: ∫ 1 S d μ = μ ( S ) . {\displaystyle \int 1_{S}\,d\mu =\mu (S).} Notice that the result may be equal to +∞ , unless μ is a finite measure. A finite linear combination of indicator functions ∑ k

Probability density function - Misplaced Pages Continue

3968-457: A very pathological function into one that is "nice" from the point of view of integration, and thus let such pathological functions be integrated. Folland (1999) summarizes the difference between the Riemann and Lebesgue approaches thus: "to compute the Riemann integral of f , one partitions the domain [ a , b ] into subintervals", while in the Lebesgue integral, "one is in effect partitioning

4092-976: A whole, often called joint probability density function . This density function is defined as a function of the n variables, such that, for any domain D in the n -dimensional space of the values of the variables X 1 , ..., X n , the probability that a realisation of the set variables falls inside the domain D is Pr ( X 1 , … , X n ∈ D ) = ∫ D f X 1 , … , X n ( x 1 , … , x n ) d x 1 ⋯ d x n . {\displaystyle \Pr \left(X_{1},\ldots ,X_{n}\in D\right)=\int _{D}f_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})\,dx_{1}\cdots dx_{n}.} If F ( x 1 , ..., x n ) = Pr( X 1 ≤ x 1 , ..., X n ≤ x n )

4216-410: Is Pr ( X > 0 , Y > 0 ) = ∫ 0 ∞ ∫ 0 ∞ f X , Y ( x , y ) d x d y . {\displaystyle \Pr \left(X>0,Y>0\right)=\int _{0}^{\infty }\int _{0}^{\infty }f_{X,Y}(x,y)\,dx\,dy.} If the probability density function of

4340-399: Is axiomatic . This means that a measure is any function μ defined on a certain class X of subsets of a set E , which satisfies a certain list of properties. These properties can be shown to hold in many different cases. We start with a measure space ( E , X , μ ) where E is a set , X is a σ-algebra of subsets of E , and μ is a (non- negative ) measure on E defined on

4464-485: Is a unique probability measure on F {\displaystyle {\mathcal {F}}\,} for any CDF, and vice versa. The measure corresponding to a CDF is said to be induced by the CDF. This measure coincides with the pmf for discrete variables and PDF for continuous variables, making the measure-theoretic approach free of fallacies. The probability of a set E {\displaystyle E\,} in

4588-431: Is actually impossible to assign a length to all subsets of R in a way that preserves some natural additivity and translation invariance properties. This suggests that picking out a suitable class of measurable subsets is an essential prerequisite. The Riemann integral uses the notion of length explicitly. Indeed, the element of calculation for the Riemann integral is the rectangle [ a , b ] × [ c , d ] , whose area

4712-549: Is an upper triangular matrix with ones on the main diagonal, therefore its determinant is 1. Applying the change of variable theorem from the previous section we obtain that f Y , X ( y , x ) = f X ( x ) δ ( y − V ( x ) ) , {\displaystyle f_{Y,X}(y,x)=f_{X}(\mathbf {x} )\delta {\big (}y-V(\mathbf {x} ){\big )},} which if marginalized over x {\displaystyle x} leads to

4836-456: Is attached, which satisfies the following properties: That is, the probability function f ( x ) lies between zero and one for every value of x in the sample space Ω , and the sum of f ( x ) over all values x in the sample space Ω is equal to 1. An event is defined as any subset E {\displaystyle E\,} of the sample space Ω {\displaystyle \Omega \,} . The probability of

4960-429: Is calculated to be ( b − a )( d − c ) . The quantity b − a is the length of the base of the rectangle and d − c is the height of the rectangle. Riemann could only use planar rectangles to approximate the area under the curve, because there was no adequate theory for measuring more general sets. In the development of the theory in most modern textbooks (after 1950), the approach to measure and integration

5084-940: Is clear that H {\displaystyle H} is a bijective mapping, and the Jacobian of H − 1 {\displaystyle H^{-1}} is given by: d H − 1 ( y , x ~ ) d y d x ~ = [ 1 − d V ( x ~ ) d x ~ 0 n × 1 I n × n ] , {\displaystyle {\frac {dH^{-1}(y,{\tilde {\mathbf {x} }})}{dy\,d{\tilde {\mathbf {x} }}}}={\begin{bmatrix}1&-{\frac {dV({\tilde {\mathbf {x} }})}{d{\tilde {\mathbf {x} }}}}\\\mathbf {0} _{n\times 1}&\mathbf {I} _{n\times n}\end{bmatrix}},} which

Probability density function - Misplaced Pages Continue

5208-514: Is closed under algebraic operations, but more importantly it is closed under various kinds of point-wise sequential limits : sup k ∈ N f k , lim inf k ∈ N f k , lim sup k ∈ N f k {\displaystyle \sup _{k\in \mathbb {N} }f_{k},\quad \liminf _{k\in \mathbb {N} }f_{k},\quad \limsup _{k\in \mathbb {N} }f_{k}} are measurable if

5332-456: Is continuous at x {\displaystyle x} ) f X ( x ) = d d x F X ( x ) . {\displaystyle f_{X}(x)={\frac {d}{dx}}F_{X}(x).} Intuitively, one can think of f X ( x ) d x {\displaystyle f_{X}(x)\,dx} as being the probability of X {\displaystyle X} falling within

5456-488: Is finite or +∞. A simple function can be written in different ways as a linear combination of indicator functions, but the integral will be the same by the additivity of measures. Some care is needed when defining the integral of a real-valued simple function, to avoid the undefined expression ∞ − ∞ : one assumes that the representation f = ∑ k a k 1 S k {\displaystyle f=\sum _{k}a_{k}1_{S_{k}}}

5580-469: Is given by the sum of the probabilities of the events. The probability that any one of the events {1,6}, {3}, or {2,4} will occur is 5/6. This is the same as saying that the probability of event {1,2,3,4,6} is 5/6. This event encompasses the possibility of any number except five being rolled. The mutually exclusive event {5} has a probability of 1/6, and the event {1,2,3,4,5,6} has a probability of 1, that is, absolute certainty. When doing calculations using

5704-486: Is important, for instance, in the study of Fourier series , Fourier transforms , and other topics. The Lebesgue integral describes better how and when it is possible to take limits under the integral sign (via the monotone convergence theorem and dominated convergence theorem ). While the Riemann integral considers the area under a curve as made out of vertical rectangles, the Lebesgue definition considers horizontal slabs that are not necessarily just rectangles, and so it

5828-509: Is measurable if the pre-image of every interval of the form ( t , ∞) is in X : { x ∣ f ( x ) > t } ∈ X ∀ t ∈ R . {\displaystyle \{x\,\mid \,f(x)>t\}\in X\quad \forall t\in \mathbb {R} .} We can show that this is equivalent to requiring that the pre-image of any Borel subset of R be in X . The set of measurable functions

5952-458: Is more flexible. For this reason, the Lebesgue definition makes it possible to calculate integrals for a broader class of functions. For example, the Dirichlet function, which is 1 where its argument is rational and 0 otherwise, has a Lebesgue integral, but does not have a Riemann integral. Furthermore, the Lebesgue integral of this function is zero, which agrees with the intuition that when picking

6076-449: Is more general than the Riemann integral , which it largely replaced in mathematical analysis since the first half of the 20th century. It can accommodate functions with discontinuities arising in many applications that are pathological from the perspective of the Riemann integral. The Lebesgue integral also has generally better analytical properties. For instance, under mild conditions, it

6200-501: Is not necessarily a density) then the n variables in the set are all independent from each other, and the marginal probability density function of each of them is given by f X i ( x i ) = f i ( x i ) ∫ f i ( x ) d x . {\displaystyle f_{X_{i}}(x_{i})={\frac {f_{i}(x_{i})}{\int f_{i}(x)\,dx}}.} This elementary example illustrates

6324-402: Is possible to exchange limits and Lebesgue integration, while the conditions for doing this with a Riemann integral are comparatively baroque. Furthermore, the Lebesgue integral can be generalized in a straightforward way to more general spaces, measure spaces , such as those that arise in probability theory . The term Lebesgue integration can mean either the general theory of integration of

SECTION 50

#1732877035048

6448-408: Is possible to represent certain discrete random variables as well as random variables involving both a continuous and a discrete part with a generalized probability density function using the Dirac delta function . (This is not possible with a probability density function in the sense defined above, it may be done with a distribution .) For example, consider a binary discrete random variable having

6572-414: Is quantifiable. Suppose the answer is 0.02 (i.e., 2%). Then, the probability that the bacterium dies between 5 hours and 5.001 hours should be about 0.002, since this time interval is one-tenth as long as the previous. The probability that the bacterium dies between 5 hours and 5.0001 hours should be about 0.0002, and so on. In this example, the ratio (probability of living during an interval) / (duration of

6696-474: Is such that μ( S k ) < ∞ whenever a k ≠ 0 . Then the above formula for the integral of f makes sense, and the result does not depend upon the particular representation of f satisfying the assumptions. If B is a measurable subset of E and s is a measurable simple function one defines ∫ B s d μ = ∫ 1 B s d μ = ∑ k

6820-900: Is tempting to think that in order to find the expected value E( g ( X )) , one must first find the probability density f g ( X ) of the new random variable Y = g ( X ) . However, rather than computing E ⁡ ( g ( X ) ) = ∫ − ∞ ∞ y f g ( X ) ( y ) d y , {\displaystyle \operatorname {E} {\big (}g(X){\big )}=\int _{-\infty }^{\infty }yf_{g(X)}(y)\,dy,} one may find instead E ⁡ ( g ( X ) ) = ∫ − ∞ ∞ g ( x ) f X ( x ) d x . {\displaystyle \operatorname {E} {\big (}g(X){\big )}=\int _{-\infty }^{\infty }g(x)f_{X}(x)\,dx.} The values of

6944-737: Is the Radon–Nikodym derivative : f = d X ∗ P d μ . {\displaystyle f={\frac {dX_{*}P}{d\mu }}.} That is, f is any measurable function with the property that: Pr [ X ∈ A ] = ∫ X − 1 A d P = ∫ A f d μ {\displaystyle \Pr[X\in A]=\int _{X^{-1}A}\,dP=\int _{A}f\,d\mu } for any measurable set A ∈ A . {\displaystyle A\in {\mathcal {A}}.} In

7068-534: Is the cumulative distribution function of the vector ( X 1 , ..., X n ) , then the joint probability density function can be computed as a partial derivative f ( x ) = ∂ n F ∂ x 1 ⋯ ∂ x n | x {\displaystyle f(x)=\left.{\frac {\partial ^{n}F}{\partial x_{1}\cdots \partial x_{n}}}\right|_{x}} For i = 1, 2, ..., n , let f X i ( x i ) be

7192-430: Is the probability per unit length, in other words, while the absolute likelihood for a continuous random variable to take on any particular value is 0 (since there is an infinite set of possible values to begin with), the value of the PDF at two different samples can be used to infer, in any particular draw of the random variable, how much more likely it is that the random variable would be close to one sample compared to

7316-427: Is the Riemann integral. But I can proceed differently. After I have taken all the money out of my pocket I order the bills and coins according to identical values and then I pay the several heaps one after the other to the creditor. This is my integral. The insight is that one should be able to rearrange the values of a function freely, while preserving the value of the integral. This process of rearrangement can convert

7440-976: Is the number of solutions in x for the equation g ( x ) = y {\displaystyle g(x)=y} , and g k − 1 ( y ) {\displaystyle g_{k}^{-1}(y)} are these solutions. Suppose x is an n -dimensional random variable with joint density f . If y = G ( x ) , where G is a bijective , differentiable function , then y has density p Y : p Y ( y ) = f ( G − 1 ( y ) ) | det [ d G − 1 ( z ) d z | z = y ] | {\displaystyle p_{Y}(\mathbf {y} )=f{\Bigl (}G^{-1}(\mathbf {y} ){\Bigr )}\left|\det \left[\left.{\frac {dG^{-1}(\mathbf {z} )}{d\mathbf {z} }}\right|_{\mathbf {z} =\mathbf {y} }\right]\right|} with

7564-459: Is the probability that the bacterium dies in that window. A probability density function is most commonly associated with absolutely continuous univariate distributions . A random variable X {\displaystyle X} has density f X {\displaystyle f_{X}} , where f X {\displaystyle f_{X}} is a non-negative Lebesgue-integrable function, if: Pr [

SECTION 60

#1732877035048

7688-524: Is the probability that the bacterium dies within an infinitesimal window of time around 5 hours, where dt is the duration of this window. For example, the probability that it lives longer than 5 hours, but shorter than (5 hours + 1 nanosecond), is (2 hour)×(1 nanosecond) ≈ 6 × 10 (using the unit conversion 3.6 × 10 nanoseconds = 1 hour). There is a probability density function f with f (5 hours) = 2 hour. The integral of f over any window of time (not only infinitesimal windows but also large windows)

7812-490: The Borel sets as measurable subsets) has as probability distribution the pushforward measure X ∗ P on ( X , A ) {\displaystyle ({\mathcal {X}},{\mathcal {A}})} : the density of X {\displaystyle X} with respect to a reference measure μ {\displaystyle \mu } on ( X , A ) {\displaystyle ({\mathcal {X}},{\mathcal {A}})}

7936-671: The Dirac delta function. It is possible to use the formulas above to determine f Y {\displaystyle f_{Y}} , the probability density function of Y = V ( X ) {\displaystyle Y=V(X)} , which will be given by f Y ( y ) = ∫ R n f X ( x ) δ ( y − V ( x ) ) d x . {\displaystyle f_{Y}(y)=\int _{\mathbb {R} ^{n}}f_{X}(\mathbf {x} )\delta {\big (}y-V(\mathbf {x} ){\big )}\,d\mathbf {x} .} This result leads to

8060-532: The Generalized Central Limit Theorem (GCLT). Lebesgue integration In mathematics , the integral of a non-negative function of a single variable can be regarded, in the simplest case, as the area between the graph of that function and the X axis. The Lebesgue integral , named after French mathematician Henri Lebesgue , is one way to make this concept rigorous and to extend it to more general functions. The Lebesgue integral

8184-471: The Rademacher distribution —that is, taking −1 or 1 for values, with probability 1 ⁄ 2 each. The density of probability associated with this variable is: f ( t ) = 1 2 ( δ ( t + 1 ) + δ ( t − 1 ) ) . {\displaystyle f(t)={\frac {1}{2}}(\delta (t+1)+\delta (t-1)).} More generally, if

8308-414: The continuous univariate case above , the reference measure is the Lebesgue measure . The probability mass function of a discrete random variable is the density with respect to the counting measure over the sample space (usually the set of integers , or some subset thereof). It is not possible to define a density with reference to an arbitrary measure (e.g. one can not choose the counting measure as

8432-424: The discrete uniform , Bernoulli , binomial , negative binomial , Poisson and geometric distributions . Important continuous distributions include the continuous uniform , normal , exponential , gamma and beta distributions . In probability theory, there are several notions of convergence for random variables . They are listed below in the order of strength, i.e., any subsequent notion of convergence in

8556-699: The identity function . This does not always work. For example, when flipping a coin the two possible outcomes are "heads" and "tails". In this example, the random variable X could assign to the outcome "heads" the number "0" ( X ( heads ) = 0 {\textstyle X({\text{heads}})=0} ) and to the outcome "tails" the number "1" ( X ( tails ) = 1 {\displaystyle X({\text{tails}})=1} ). Discrete probability theory deals with events that occur in countable sample spaces. Examples: Throwing dice , experiments with decks of cards , random walk , and tossing coins . Classical definition : Initially

8680-1357: The inverse function . This follows from the fact that the probability contained in a differential area must be invariant under change of variables. That is, | f Y ( y ) d y | = | f X ( x ) d x | , {\displaystyle \left|f_{Y}(y)\,dy\right|=\left|f_{X}(x)\,dx\right|,} or f Y ( y ) = | d x d y | f X ( x ) = | d d y ( x ) | f X ( x ) = | d d y ( g − 1 ( y ) ) | f X ( g − 1 ( y ) ) = | ( g − 1 ) ′ ( y ) | ⋅ f X ( g − 1 ( y ) ) . {\displaystyle f_{Y}(y)=\left|{\frac {dx}{dy}}\right|f_{X}(x)=\left|{\frac {d}{dy}}(x)\right|f_{X}(x)=\left|{\frac {d}{dy}}{\big (}g^{-1}(y){\big )}\right|f_{X}{\big (}g^{-1}(y){\big )}={\left|\left(g^{-1}\right)'(y)\right|}\cdot f_{X}{\big (}g^{-1}(y){\big )}.} For functions that are not monotonic,

8804-1349: The law of the unconscious statistician : E Y ⁡ [ Y ] = ∫ R y f Y ( y ) d y = ∫ R y ∫ R n f X ( x ) δ ( y − V ( x ) ) d x d y = ∫ R n ∫ R y f X ( x ) δ ( y − V ( x ) ) d y d x = ∫ R n V ( x ) f X ( x ) d x = E X ⁡ [ V ( X ) ] . {\displaystyle \operatorname {E} _{Y}[Y]=\int _{\mathbb {R} }yf_{Y}(y)\,dy=\int _{\mathbb {R} }y\int _{\mathbb {R} ^{n}}f_{X}(\mathbf {x} )\delta {\big (}y-V(\mathbf {x} ){\big )}\,d\mathbf {x} \,dy=\int _{{\mathbb {R} }^{n}}\int _{\mathbb {R} }yf_{X}(\mathbf {x} )\delta {\big (}y-V(\mathbf {x} ){\big )}\,dy\,d\mathbf {x} =\int _{\mathbb {R} ^{n}}V(\mathbf {x} )f_{X}(\mathbf {x} )\,d\mathbf {x} =\operatorname {E} _{X}[V(X)].} Proof: Let Z {\displaystyle Z} be

8928-552: The mean , variance , and kurtosis ), starting from the formulas given for a continuous distribution of the probability. It is common for probability density functions (and probability mass functions ) to be parametrized—that is, to be characterized by unspecified parameters . For example, the normal distribution is parametrized in terms of the mean and the variance , denoted by μ {\displaystyle \mu } and σ 2 {\displaystyle \sigma ^{2}} respectively, giving

9052-485: The probability distribution is defined as a function over general sets of values or it may refer to the cumulative distribution function , or it may be a probability mass function (PMF) rather than the density. "Density function" itself is also used for the probability mass function, leading to further confusion. In general though, the PMF is used in the context of discrete random variables (random variables that take values on

9176-883: The weak and the strong law of large numbers It follows from the LLN that if an event of probability p is observed repeatedly during independent experiments, the ratio of the observed frequency of that event to the total number of repetitions converges towards p . For example, if Y 1 , Y 2 , . . . {\displaystyle Y_{1},Y_{2},...\,} are independent Bernoulli random variables taking values 1 with probability p and 0 with probability 1- p , then E ( Y i ) = p {\displaystyle {\textrm {E}}(Y_{i})=p} for all i , so that Y ¯ n {\displaystyle {\bar {Y}}_{n}} converges to p almost surely . The central limit theorem (CLT) explains

9300-556: The 6 have even numbers and each face has the same probability of appearing. Modern definition : The modern definition starts with a finite or countable set called the sample space , which relates to the set of all possible outcomes in classical sense, denoted by Ω {\displaystyle \Omega } . It is then assumed that for each element x ∈ Ω {\displaystyle x\in \Omega \,} , an intrinsic "probability" value f ( x ) {\displaystyle f(x)\,}

9424-434: The above definition of multidimensional probability density functions in the simple case of a function of a set of two variables. Let us call R → {\displaystyle {\vec {R}}} a 2-dimensional random vector of coordinates ( X , Y ) : the probability to obtain R → {\displaystyle {\vec {R}}} in the quarter plane of positive x and y

9548-451: The areas of these horizontal slabs. From this perspective, a key difference with the Riemann integral is that the "slabs" are no longer rectangular (cartesian products of two intervals), but instead are cartesian products of a measurable set with an interval. An equivalent way to introduce the Lebesgue integral is to use so-called simple functions , which generalize the step functions of Riemann integration. Consider, for example, determining

9672-418: The basic theorems about the Lebesgue integral. Measure theory was initially created to provide a useful abstraction of the notion of length of subsets of the real line—and, more generally, area and volume of subsets of Euclidean spaces. In particular, it provided a systematic answer to the question of which subsets of R have a length. As later set theory developments showed (see non-measurable set ), it

9796-414: The cumulative COVID-19 case count from a graph of smoothed cases each day (right). One can think of the Lebesgue integral either in terms of slabs or simple functions . Intuitively, the area under a simple function can be partitioned into slabs based on the (finite) collection of values in the range of a simple function (a real interval). Conversely, the (finite) collection of slabs in the undergraph of

9920-410: The derivative gives us the CDF back again, then the random variable X is said to have a probability density function ( PDF ) or simply density f ( x ) = d F ( x ) d x . {\displaystyle f(x)={\frac {dF(x)}{dx}}\,.} For a set E ⊆ R {\displaystyle E\subseteq \mathbb {R} } ,

10044-542: The derivative of the cumulative distribution function and the probability density function is generally used as the definition of the probability density function. This alternate definition is the following: If dt is an infinitely small number, the probability that X is included within the interval ( t , t + dt ) is equal to f ( t ) dt , or: Pr ( t < X < t + d t ) = f ( t ) d t . {\displaystyle \Pr(t<X<t+dt)=f(t)\,dt.} It

10168-1022: The desired probability density function. The probability density function of the sum of two independent random variables U and V , each of which has a probability density function, is the convolution of their separate density functions: f U + V ( x ) = ∫ − ∞ ∞ f U ( y ) f V ( x − y ) d y = ( f U ∗ f V ) ( x ) {\displaystyle f_{U+V}(x)=\int _{-\infty }^{\infty }f_{U}(y)f_{V}(x-y)\,dy=\left(f_{U}*f_{V}\right)(x)} Probability theory Central subjects in probability theory include discrete and continuous random variables , probability distributions , and stochastic processes (which provide mathematical abstractions of non-deterministic or uncertain processes or measured quantities that may either be single occurrences or evolve over time in

10292-1815: The differential regarded as the Jacobian of the inverse of G (⋅) , evaluated at y . For example, in the 2-dimensional case x = ( x 1 , x 2 ) , suppose the transform G is given as y 1 = G 1 ( x 1 , x 2 ) , y 2 = G 2 ( x 1 , x 2 ) with inverses x 1 = G 1 ( y 1 , y 2 ) , x 2 = G 2 ( y 1 , y 2 ) . The joint distribution for y  = ( y 1 , y 2 ) has density p Y 1 , Y 2 ( y 1 , y 2 ) = f X 1 , X 2 ( G 1 − 1 ( y 1 , y 2 ) , G 2 − 1 ( y 1 , y 2 ) ) | ∂ G 1 − 1 ∂ y 1 ∂ G 2 − 1 ∂ y 2 − ∂ G 1 − 1 ∂ y 2 ∂ G 2 − 1 ∂ y 1 | . {\displaystyle p_{Y_{1},Y_{2}}(y_{1},y_{2})=f_{X_{1},X_{2}}{\big (}G_{1}^{-1}(y_{1},y_{2}),G_{2}^{-1}(y_{1},y_{2}){\big )}\left\vert {\frac {\partial G_{1}^{-1}}{\partial y_{1}}}{\frac {\partial G_{2}^{-1}}{\partial y_{2}}}-{\frac {\partial G_{1}^{-1}}{\partial y_{2}}}{\frac {\partial G_{2}^{-1}}{\partial y_{1}}}\right\vert .} Let V : R n → R {\displaystyle V:\mathbb {R} ^{n}\to \mathbb {R} } be

10416-426: The discrete values accessible to the variable and p 1 , … , p n {\displaystyle p_{1},\ldots ,p_{n}} are the probabilities associated with these values. This substantially unifies the treatment of discrete and continuous probability distributions. The above expression allows for determining statistical characteristics of such a discrete variable (such as

10540-490: The discrete, continuous, a mix of the two, and more. Consider an experiment that can produce a number of outcomes. The set of all outcomes is called the sample space of the experiment. The power set of the sample space (or equivalently, the event space) is formed by considering all different collections of possible results. For example, rolling an honest die produces one of six possible results. One collection of possible results corresponds to getting an odd number. Thus,

10664-771: The distribution function of f {\displaystyle f} as the "width of a slab", i.e., F ( y ) = μ { x | f ( x ) > y } . {\displaystyle F(y)=\mu \{x|f(x)>y\}.} Then F ( y ) {\displaystyle F(y)} is monotone decreasing and non-negative, and therefore has an (improper) Riemann integral over ( 0 , ∞ ) {\displaystyle (0,\infty )} . The Lebesgue integral can then be defined by ∫ f d μ = ∫ 0 ∞ F ( y ) d y {\displaystyle \int f\,d\mu =\int _{0}^{\infty }F(y)\,dy} where

10788-453: The event E {\displaystyle E\,} is defined as So, the probability of the entire sample space is 1, and the probability of the null event is 0. The function f ( x ) {\displaystyle f(x)\,} mapping a point in the sample space to the "probability" value is called a probability mass function abbreviated as pmf . Continuous probability theory deals with events that occur in

10912-427: The event made up of all possible results (in our example, the event {1,2,3,4,5,6}) be assigned a value of one. To qualify as a probability distribution , the assignment of values must satisfy the requirement that if you look at a collection of mutually exclusive events (events that contain no common results, e.g., the events {1,6}, {3}, and {2,4} are all mutually exclusive), the probability that any of these events occurs

11036-463: The family of densities f ( x ; μ , σ 2 ) = 1 σ 2 π e − 1 2 ( x − μ σ ) 2 . {\displaystyle f(x;\mu ,\sigma ^{2})={\frac {1}{\sigma {\sqrt {2\pi }}}}e^{-{\frac {1}{2}}\left({\frac {x-\mu }{\sigma }}\right)^{2}}.} Different values of

11160-401: The foundations of probability theory, but instead emerges from these foundations as a theorem. Since it links theoretically derived probabilities to their actual frequency of occurrence in the real world, the law of large numbers is considered as a pillar in the history of statistical theory and has had widespread influence. The law of large numbers (LLN) states that the sample average of

11284-422: The function can be rearranged after a finite repartitioning to be the undergraph of a simple function. The slabs viewpoint makes it easy to define the Lebesgue integral, in terms of basic calculus. Suppose that f {\displaystyle f} is a (Lebesgue measurable) function, taking non-negative values (possibly including + ∞ {\displaystyle +\infty } ). Define

11408-487: The graphs of other functions, for example the Dirichlet function , don't fit well with the notion of area. Graphs like the one of the latter, raise the question: for which class of functions does "area under the curve" make sense? The answer to this question has great theoretical importance. As part of a general movement toward rigor in mathematics in the nineteenth century, mathematicians attempted to put integral calculus on

11532-496: The identification in Distribution theory of measures with distributions of order 0 , or with Radon measures , one can also use a dual pair notation and write the integral with respect to μ in the form ⟨ μ , f ⟩ . {\displaystyle \langle \mu ,f\rangle .} The theory of the Lebesgue integral requires a theory of measurable sets and measures on these sets, as well as

11656-554: The infinitesimal interval [ x , x + d x ] {\displaystyle [x,x+dx]} . ( This definition may be extended to any probability distribution using the measure-theoretic definition of probability . ) A random variable X {\displaystyle X} with values in a measurable space ( X , A ) {\displaystyle ({\mathcal {X}},{\mathcal {A}})} (usually R n {\displaystyle \mathbb {R} ^{n}} with

11780-434: The integral on the right is an ordinary improper Riemann integral, of a non-negative function (interpreted appropriately as + ∞ {\displaystyle +\infty } if F ( y ) = + ∞ {\displaystyle F(y)=+\infty } on a neighborhood of 0). Most textbooks, however, emphasize the simple functions viewpoint, because it is then more straightforward to prove

11904-407: The interval) is approximately constant, and equal to 2 per hour (or 2 hour). For example, there is 0.02 probability of dying in the 0.01-hour interval between 5 and 5.01 hours, and (0.02 probability / 0.01 hours) = 2 hour. This quantity 2 hour is called the probability density for dying at around 5 hours. Therefore, the probability that the bacterium dies at 5 hours can be written as (2 hour) dt . This

12028-530: The joint probability density function of a vector of n random variables can be factored into a product of n functions of one variable f X 1 , … , X n ( x 1 , … , x n ) = f 1 ( x 1 ) ⋯ f n ( x n ) , {\displaystyle f_{X_{1},\ldots ,X_{n}}(x_{1},\ldots ,x_{n})=f_{1}(x_{1})\cdots f_{n}(x_{n}),} (where each f i

12152-406: The list implies convergence according to all of the preceding notions. As the names indicate, weak convergence is weaker than strong convergence. In fact, strong convergence implies convergence in probability, and convergence in probability implies weak convergence. The reverse statements are not always true. Common intuition suggests that if a fair coin is tossed many times, then roughly half of

12276-433: The measure-theoretic treatment of probability is that it unifies the discrete and the continuous cases, and makes the difference a question of which measure is used. Furthermore, it covers distributions that are neither discrete nor continuous nor mixtures of the two. An example of such distributions could be a mix of discrete and continuous distributions—for example, a random variable that is 0 with probability 1/2, and takes

12400-540: The notion of sample space , introduced by Richard von Mises , and measure theory and presented his axiom system for probability theory in 1933. This became the mostly undisputed axiomatic basis for modern probability theory; but, alternatives exist, such as the adoption of finite rather than countable additivity by Bruno de Finetti . Most introductions to probability theory treat discrete probability distributions and continuous probability distributions separately. The measure theory-based treatment of probability covers

12524-611: The original sequence ( f k ) , where k ∈ N , consists of measurable functions. There are several approaches for defining an integral for measurable real-valued functions f defined on E , and several notations are used to denote such an integral. ∫ E f d μ = ∫ E f ( x ) d μ ( x ) = ∫ E f ( x ) μ ( d x ) . {\displaystyle \int _{E}f\,d\mu =\int _{E}f(x)\,d\mu (x)=\int _{E}f(x)\,\mu (dx).} Following

12648-408: The other sample. More precisely, the PDF is used to specify the probability of the random variable falling within a particular range of values , as opposed to taking on any one value. This probability is given by the integral of this variable's PDF over that range—that is, it is given by the area under the density function but above the horizontal axis and between the lowest and greatest values of

12772-411: The outcomes of an experiment, it is necessary that all those elementary events have a number assigned to them. This is done using a random variable . A random variable is a function that assigns to each elementary event in the sample space a real number . This function is usually denoted by a capital letter. In the case of a die, the assignment of a number to certain elementary events can be done using

12896-404: The parameters are constants, reparametrizing a density in terms of different parameters to give a characterization of a different random variable in the family, means simply substituting the new parameter values into the formula in place of the old ones. For continuous random variables X 1 , ..., X n , it is also possible to define a probability density function associated to the set as

13020-417: The parameters describe different distributions of different random variables on the same sample space (the same set of all possible values of the variable); this sample space is the domain of the family of random variables that this family of distributions describes. A given set of parameters describes a single distribution within the family sharing the functional form of the density. From the perspective of

13144-501: The preceding one, defined on the set of simple functions, when E is a segment [ a , b ] . There is also the question of whether this corresponds in any way to a Riemann notion of integration. It is possible to prove that the answer to both questions is yes. We have defined the integral of f for any non-negative extended real-valued measurable function on E . For some functions, this integral ∫ E f d μ {\textstyle \int _{E}f\,d\mu }

13268-800: The probability density function associated with variable X i alone. This is called the marginal density function, and can be deduced from the probability density associated with the random variables X 1 , ..., X n by integrating over all values of the other n − 1 variables: f X i ( x i ) = ∫ f ( x 1 , … , x n ) d x 1 ⋯ d x i − 1 d x i + 1 ⋯ d x n . {\displaystyle f_{X_{i}}(x_{i})=\int f(x_{1},\ldots ,x_{n})\,dx_{1}\cdots dx_{i-1}\,dx_{i+1}\cdots dx_{n}.} Continuous random variables X 1 , ..., X n admitting

13392-475: The probability density function for y is ∑ k = 1 n ( y ) | d d y g k − 1 ( y ) | ⋅ f X ( g k − 1 ( y ) ) , {\displaystyle \sum _{k=1}^{n(y)}\left|{\frac {d}{dy}}g_{k}^{-1}(y)\right|\cdot f_{X}{\big (}g_{k}^{-1}(y){\big )},} where n ( y )

13516-470: The probability of an event to occur was defined as the number of cases favorable for the event, over the number of total outcomes possible in an equiprobable sample space: see Classical definition of probability . For example, if the event is "occurrence of an even number when a dice is rolled", the probability is given by 3 6 = 1 2 {\displaystyle {\tfrac {3}{6}}={\tfrac {1}{2}}} , since 3 faces out of

13640-678: The probability of the random variable X being in E {\displaystyle E\,} is In case the PDF exists, this can be written as Whereas the PDF exists only for continuous random variables, the CDF exists for all random variables (including discrete random variables) that take values in R . {\displaystyle \mathbb {R} \,.} These concepts can be generalized for multidimensional cases on R n {\displaystyle \mathbb {R} ^{n}} and other continuous sample spaces. The utility of

13764-450: The probability that X will be less than or equal to x . The CDF necessarily satisfies the following properties. The random variable X {\displaystyle X} is said to have a continuous probability distribution if the corresponding CDF F {\displaystyle F} is continuous. If F {\displaystyle F\,} is absolutely continuous , i.e., its derivative exists and integrating

13888-427: The range of f ." For the Riemann integral, the domain is partitioned into intervals, and bars are constructed to meet the height of the graph. The areas of these bars are added together, and this approximates the integral, in effect by summing areas of the form f ( x ) dx where f ( x ) is the height of a rectangle and dx is its width. For the Lebesgue integral, the range is partitioned into intervals, and so

14012-413: The range. The probability density function is nonnegative everywhere, and the area under the entire curve is equal to 1. The terms probability distribution function and probability function have also sometimes been used to denote the probability density function. However, this use is not standard among probabilists and statisticians. In other sources, "probability distribution function" may be used when

14136-481: The region under the graph is partitioned into horizontal "slabs" (which may not be connected sets). The area of a small horizontal "slab" under the graph of f , of height dy , is equal to the measure of the slab's width times dy : μ ( { x ∣ f ( x ) > y } ) d y . {\displaystyle \mu \left(\{x\mid f(x)>y\}\right)\,dy.} The Lebesgue integral may then be defined by adding up

14260-560: The sequence of random variables converges in distribution to a standard normal random variable. For some classes of random variables, the classic central limit theorem works rather fast, as illustrated in the Berry–Esseen theorem . For example, the distributions with finite first, second, and third moment from the exponential family ; on the other hand, for some random variables of the heavy tail and fat tail variety, it works very slowly or may not work at all: in such cases one may use

14384-480: The sets of X . For example, E can be Euclidean n -space R or some Lebesgue measurable subset of it, X is the σ-algebra of all Lebesgue measurable subsets of E , and μ is the Lebesgue measure. In the mathematical theory of probability, we confine our study to a probability measure  μ , which satisfies μ ( E ) = 1 . Lebesgue's theory defines integrals for a class of functions called measurable functions . A real-valued function f on E

14508-477: The subset and its image under the simple function (the lower bound of the corresponding layer); intuitively, this product is the sum of the areas of all bars of the same height. The integral of a non-negative general measurable function is then defined as an appropriate supremum of approximations by simple functions, and the integral of a (not necessarily positive) measurable function is the difference of two integrals of non-negative measurable functions. To assign

14632-400: The subset {1,3,5} is an element of the power set of the sample space of dice rolls. These collections are called events . In this case, {1,3,5} is the event that the die falls on some odd number. If the results that actually occur fall in a given event, that event is said to have occurred. Probability is a way of assigning every "event" a value between zero and one, with the requirement that

14756-544: The theory of stochastic processes . For example, to study Brownian motion , probability is defined on a space of functions. When it is convenient to work with a dominating measure, the Radon-Nikodym theorem is used to define a density as the Radon-Nikodym derivative of the probability distribution of interest with respect to this dominating measure. Discrete densities are usually defined as this derivative with respect to

14880-407: The time it will turn up heads , and the other half it will turn up tails . Furthermore, the more often the coin is tossed, the more likely it should be that the ratio of the number of heads to the number of tails will approach unity. Modern probability theory provides a formal version of this intuitive idea, known as the law of large numbers . This law is remarkable because it is not assumed in

15004-427: The two integrals are the same in all cases in which both X and g ( X ) actually have probability density functions. It is not necessary that g be a one-to-one function . In some cases the latter integral is computed much more easily than the former. See Law of the unconscious statistician . Let g : R → R {\displaystyle g:\mathbb {R} \to \mathbb {R} } be

15128-756: The ubiquitous occurrence of the normal distribution in nature, and this theorem, according to David Williams, "is one of the great results of mathematics." The theorem states that the average of many independent and identically distributed random variables with finite variance tends towards a normal distribution irrespective of the distribution followed by the original random variables. Formally, let X 1 , X 2 , … {\displaystyle X_{1},X_{2},\dots \,} be independent random variables with mean μ {\displaystyle \mu } and variance σ 2 > 0. {\displaystyle \sigma ^{2}>0.\,} Then

15252-561: The σ-algebra F {\displaystyle {\mathcal {F}}\,} is defined as where the integration is with respect to the measure μ F {\displaystyle \mu _{F}\,} induced by F . {\displaystyle F\,.} Along with providing better understanding and unification of discrete and continuous probabilities, measure-theoretic treatment also allows us to work on probabilities outside R n {\displaystyle \mathbb {R} ^{n}} , as in

15376-403: Was the probabilistic nature of physical phenomena at atomic scales, described in quantum mechanics . The modern mathematical theory of probability has its roots in attempts to analyze games of chance by Gerolamo Cardano in the sixteenth century, and by Pierre de Fermat and Blaise Pascal in the seventeenth century (for example the " problem of points "). Christiaan Huygens published

#47952