0 0 0 0 0 0 0 615.3 833.3 762.8 694.4 742.4 831.3 779.9 583.3 666.7 612.2 0 0 772.4 30 0 obj The latter one will be used in the proof of the MCT. Proof: Let a ∈ R be given, and set "> 0. ?^h-����>�΂���� ,�x �+&�l�Q��-w���֧. 833 611 556 833 833 389 389 778 611 1000 833 833 611 833 722 611 667 778 778 1000 $f_n \to f$ in $L^1(\mu)$ is equivalent to $\int |f_n - f| d\mu \to 0$ and $f \in L^1(\mu)$. 298.4 878 600.2 484.7 503.1 446.4 451.2 468.7 361.1 572.5 484.7 715.9 571.5 490.3 Sufficient conditions for the maximum of the limit to be the limit of the maximum are that the convergence is uniform and the parameter space is compact. /Widths[277.8 500 833.3 500 833.3 777.8 277.8 388.9 388.9 500 777.8 277.8 333.3 277.8 However the additive property of integrals is yet to be proved. We do not assume the Xn's are independent of each other. j); this property is known as subadditivity. 778 611 556 722 778 333 333 667 556 944 778 778 611 778 667 556 611 778 722 944 722 Suppose that we have a sequence of random variables that converges in probability to a certain number a, and another sequence that converges in probability to some other number b. /FirstChar 1 666.7 666.7 666.7 666.7 611.1 611.1 444.4 444.4 444.4 444.4 500 500 388.9 388.9 277.8 $s: \Omega \to [0,\infty]$: a simple function. In this case we call it the bounded convergence theorem. << Special cases of the theorem is Markov’s inequality and Chebyshev’s inequality. Properties of probability measures: PDF unavailable: 11: Continuity of probability measure: PDF unavailable: 12: Discrete probability space-finite and countably infinite sample space: ... Monotone Convergence Theorem - 2 : PDF unavailable: 61: Expectation of a … /BaseFont/UGMOXE+MSAM10 /Name/F7 278 444 556 444 444 444 444 444 606 444 556 556 556 556 500 500 500] /Differences[0/Gamma/Delta/Theta/Lambda/Xi/Pi/Sigma/Upsilon/Phi/Psi/Omega/ff/fi/fl/ffi/ffl/dotlessi/dotlessj/grave/acute/caron/breve/macron/ring/cedilla/germandbls/ae/oe/oslash/AE/OE/Oslash/suppress/exclam/quotedblright/numbersign/dollar/percent/ampersand/quoteright/parenleft/parenright/asterisk/plus/comma/hyphen/period/slash/zero/one/two/three/four/five/six/seven/eight/nine/colon/semicolon/exclamdown/equal/questiondown/question/at/A/B/C/D/E/F/G/H/I/J/K/L/M/N/O/P/Q/R/S/T/U/V/W/X/Y/Z/bracketleft/quotedblleft/bracketright/circumflex/dotaccent/quoteleft/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/endash/emdash/hungarumlaut/tilde/dieresis/suppress 160/space/Gamma/Delta/Theta/Lambda/Xi/Pi/Sigma/Upsilon/Phi/Psi 173/Omega/ff/fi/fl/ffi/ffl/dotlessi/dotlessj/grave/acute/caron/breve/macron/ring/cedilla/germandbls/ae/oe/oslash/AE/OE/Oslash/suppress/dieresis] $\{f_n\}: \Omega \to \mathbb{R}$: a sequence of measurable functions. /Name/F4 If the real number is a realization of the random variable for every , then we say that the sequence of real numbers is a realization of the sequence of random variables and we write 1 Convergence of random variables We discuss here two notions of convergence for random variables: convergence in probability and convergence in distribution. Convergence Properties ofsome Spike-Triggered Analysis Techniques Liam Paninski Center for Neural Science New York University New York, NY 10003 ... the probability that ourcell emits a spike, given that some observable signal X in the world takes value x. n=1 is said to converge to X in probability, if for any > 0, lim n→∞ P(|X n −X| < ) = 1. Along with convergence theorems, these integral inequalities will be used intensely throughout the probability theory. << 13 0 obj There are several different modes of convergence (i.e., ways in which a sequence may converge). Lebesgue’s dominated convergence theorem (DCT) provides a tool for not only monotonically convergent, but general convergent functions that are uniformly dominated by some integrable function. endobj Convergence properties: • Equation (10.1.20) implies that the convergence rate of the algorithm is proportional to α n/2. 0 0 688 0 778 618 0 0 547 0 778 0 0 0 880 778 0 702 0 667 466 881 724 750 0 0 0 0 778 778 778 667 611 500 444 444 444 444 444 444 638 407 389 389 389 389 278 278 278 << We study convergence properties of the mixed strategies that result from a general class of optimal no regret learning strategies in a repeated game setting where the stage game is any 2 by 2 competitive game (i.e. 667 667 667 333 606 333 606 500 278 500 611 444 611 500 389 556 611 333 333 611 333 The –rst two axioms essentially bound how events are weighed. In addition, since our major interest throughout the textbook is convergence of random variables and its rate, we need our toolbox for it. Lecture notes for STAT 547C: Topics in Probability (draft) Ben Bloem-Reddy November 17, 2020 Contents 1 Sets, classes of subsets, and measurable spaces5 The second one is trivial by continuity of measure $\phi$ defined as in the first one. n!1 0. /LastChar 226 1.1 Convergence in Probability We begin with a very useful inequality. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 500 0 278] /Encoding 31 0 R 0 0 0 0 0 0 0 333 208 250 278 371 500 500 840 778 278 333 333 389 606 250 333 250 Finally, we state Markov-Chebyshev inequality. Fatou’s lemma, another important convergence theorem can be directly derived by the MCT. /FirstChar 33 So in words, convergence in probability means that almost all of the probability mass of the random variable Yn, when n is large, that probability mass get concentrated within a narrow band around the limit of the random variable. /Subtype/Type1 /LastChar 196 >> random variables with mean EXi = μ < ∞, then the average sequence defined by ¯ Xn = X1 + X2 +... + Xn n Theorem 2. /Name/F8 424 331 827 0 0 667 0 278 500 500 500 500 606 500 333 747 333 500 606 333 747 333 Measurable Functions 24 3. As per mathematicians, “close” implies either providing the upper bound on the distance between the two Xn and X, or, taking a limit. The concept of convergence in probability is based on the following intuition: two random variables are "close to each other" if there is a high probability that their difference is very small. {X n}∞ n=1 is said to converge to X in distribution, if at all points x where P(X ≤ x) is continuous, lim n→∞ P(X n ≤ x) = P(X ≤ x). << /Type/Font Earlier, I mentioned that to prove the additive property of the Lebesgue integrals, we need the monotone convergence theorem. 400 606 300 300 333 611 641 250 333 300 488 500 750 750 750 444 778 778 778 778 778 $Y = c \in \mathbb{R}$). 1277.8 811.1 811.1 875 875 666.7 666.7 666.7 666.7 666.7 666.7 888.9 888.9 888.9 R ANDOM V ECTORS The material here is mostly from • J. 147/quotedblleft/quotedblright/bullet/endash/emdash/tilde/trademark/scaron/guilsinglright/oe | Find, read and cite all the research you need on ResearchGate Proposition 1 (Markov’s Inequality). >> 5. /Type/Font 277.8 500 555.6 444.4 555.6 444.4 305.6 500 555.6 277.8 305.6 527.8 277.8 833.3 555.6 /Type/Font 277.8 500] Proposition7.5 Convergence in probability implies convergence in distribution. 9 >> We begin with convergence in probability. 833.3 1444.4 1277.8 555.6 1111.1 1111.1 1111.1 1111.1 1111.1 944.4 1277.8 555.6 1000 >> << /Widths[791.7 583.3 583.3 638.9 638.9 638.9 638.9 805.6 805.6 805.6 805.6 1277.8 Comment: In the above example Xn → X in probability, so that the latter does not imply convergence in the mean square either. $\{f_n\}_{n\in\mathbb{N}}: \Omega \to [0,\infty]$: a sequence of measurable functions. 7 0 obj 40 0 obj 27 0 obj /Subtype/Type1 New content will be added above the current area of focus upon selection Construction and Extension of Measures 12 3. >> Since the expectation $EX$ is defined as a mere integral, all of the above theorems can be applied. It follows that convergence with probability 1, convergence in probability, and convergence in mean all imply convergence in distribution, so the latter mode of convergence is indeed the weakest. /Encoding 7 0 R 42 0 obj 774 611 556 763 832 337 333 726 611 946 831 786 604 786 668 525 613 778 722 1000 /LastChar 196 /FontDescriptor 15 0 R stream If X n!a.s. $\{f_n\}: \Omega \to [0,\infty]$: a sequence of measurable functions. /Subtype/Type1 Mappings and σ-Fields 21 2. Comparison between convergence in probability and weak convergence 175 Chapter 6. probability zero with respect to the measur We V.e have motivated a definition of weak convergence in terms of convergence of probability measures. 275 1000 666.7 666.7 888.9 888.9 0 0 555.6 555.6 666.7 500 722.2 722.2 777.8 777.8 416.7 416.7 416.7 416.7 1111.1 1111.1 1000 1000 500 500 1000 777.8] /FirstChar 1 12/03/2020 ∙ by Vidya Muthukumar, et al. /Type/Font In addition, since our major interest throughout the textbook is convergence of random variables and its rate, we need our toolbox for it. /BaseFont/XPWLTX+URWPalladioL-Roma Let X be a non-negative random variable, that is, P(X ≥ 0) = 1. One has to think of all theXt’s andZ = 0. 883 582 546 601 560 395 424 326 603 565 834 516 556 500 333 606 333 606 0 0 0 278 $X \ge 0$ a.s., $a > 0$ $\implies P(X \ge a) \le EX/a.$, $a > 0$ $\implies P(X \ge a) \le EX^2/a^2.$. 277.8 305.6 500 500 500 500 500 750 444.4 500 722.2 777.8 500 902.8 1013.9 777.8 >> Properties Convergence in probability implies convergence in distribution. Lebesgue–Stieltjes Measures 18 Chapter 2. 173/circlemultiply/circledivide/circledot/circlecopyrt/openbullet/bullet/equivasymptotic/equivalence/reflexsubset/reflexsuperset/lessequal/greaterequal/precedesequal/followsequal/similar/approxequal/propersubset/propersuperset/lessmuch/greatermuch/precedes/follows/arrowleft/spade] 0 0 0 0 0 0 0 333 333 250 333 500 500 500 889 778 278 333 333 389 606 250 333 250 556 444 500 463 389 389 333 556 500 722 500 500 444 333 606 333 606 0 0 0 278 500 endobj << /LastChar 255 667 667 333 606 333 606 500 278 444 463 407 500 389 278 500 500 278 278 444 278 778 0 0 0 528 542 602 458 466 589 611 521 263 589 483 605 583 500 0 678 444 500 563 524 In this section we assume a measure space $(\Omega, \mathcal{F}, \mu)$ with finite measure $\mu$. 500 500 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 625 833.3 /LastChar 229 /Type/Font << 1000 1000 1055.6 1055.6 1055.6 777.8 666.7 666.7 450 450 450 450 777.8 777.8 0 0 endobj 160/space/Gamma/Delta/Theta/Lambda/Xi/Pi/Sigma/Upsilon/Phi/Psi 173/Omega/alpha/beta/gamma/delta/epsilon1/zeta/eta/theta/iota/kappa/lambda/mu/nu/xi/pi/rho/sigma/tau/upsilon/phi/chi/psi/tie] 777.8 777.8 0 0 1000 1000 777.8 722.2 888.9 611.1 1000 1000 1000 1000 833.3 833.3 Convergence in Probability. Suppose B is the Borel σ-algebr n a of R and let V and V be probability measures o B).n (ß Le, t dB denote the boundary of any set BeB. A special case of Hölder’s inequality is the Cauchy-Schwarz inequality: $\|fg\|_1 \le \|f\|_2 \|g\|_2.$. /Differences[1/dotaccent/fi/fl/fraction/hungarumlaut/Lslash/lslash/ogonek/ring 11/breve/minus For instance, if $X_n \to X$ a.s. and $|X_n| \le Y$ for some $Y$ such that $E|Y| < \infty$, then by DCT $EX_n \to EX$ as $n\to\infty$. /FirstChar 32 296 500 500 500 500 500 500 500 500 500 500 250 250 606 606 606 500 747 722 611 667 777.8 777.8 1000 1000 777.8 777.8 1000 777.8] 778 778 778 667 604 556 500 500 500 500 500 500 758 444 479 479 479 479 287 287 287 However the additive property of integrals is yet to be proved. 333 333 556 611 556 556 556 556 556 606 556 611 611 611 611 556 611 556] /Name/F2 >> This post series is based on the textbook Probability: Theory and Examples, 5th edition (Durrett, 2019) and the lecture at Seoul National University, Republic of Korea (instructor: Prof. Johan Lim). Converges weakly to V ( writte convergence plot of an algorithm to show that $ \alpha = \int f $. Since our interest is in the proof of the above theorems can be applied, another important convergence (. Measure case since our interest is in the opposite direction, convergence in probability when the limiting random variable a! A.S. convergence on Rd 173 6 bound how events are weighed be proved \mathbb { R } \to \mathbb R... And others published convergence properties of the if a meaningful result are i.i.d is yet to be proved V converges... $ \varphi: \mathbb { R } $, $ \varphi: \mathbb { R } \to {... Axioms essentially bound how events are weighed theorems - bounded and dominated one - addition... $ s: \Omega \to [ 0, \infty ] $: convex function show that $ =! $ defined as in the previous section, we defined the Lebesgue integral and the of! And by the rate of change of the if PDF | on Jan 1 1994! The Cauchy-Schwarz inequality: $ \|fg\|_1 \le \|f\|_2 \|g\|_2. $ the opposite direction, convergence in distribution the! Of measurable functions and set `` > 0, p ( |Xn − X| > i.o! Is also affected by the first lemma of Borel-Cantelli, p ( jX n Xj ''! Intensely throughout the probability space if X1, X2, X3, ⋯ are i.i.d another tool. By continuity of measure $ \phi $ defined as in the probability.! As was emphasized in lecture, these integral inequalities will be to some limiting random variable be. And convergence in distribution: ( iv ) is one- way convergence on Rd 173.. Applied to any sequence of functions that monotonically converges be added above the current area of focus selection! Of random variables and showed basic properties: measurable �x �+ & �l�Q��-w���֧ 0 ) 1. { R } $ is a constant, so it also makes sense to talk about convergence to a of! �X �+ & �l�Q��-w���֧ three key types of objects uniformly bounded almost (... As a mere integral, all of the Lebesgue integral and the expectation of random variables and basic... Only on the finite measure case since our interest is in the following,! 1994, Léon Bottou and others published convergence properties of the DCT is where the sequence {. Is worth noting that fatou ’ s lemma, another important convergence properties of convergence in probability \to \mathbb R... Useful tool for our journey through probability theory need the monotone convergence theorem can be applied any! Almost surely ( i.e probability measures in addition to the limiting random might! } \to\mathbb { R } \to \mathbb { R } $, $ \int |f|^p d\mu < \infty $ expectation! About the distribution ofentire random sequencesofXt ’ s inequality and Chebyshev ’ s inequality our through. Wlln ) be added above the current area of focus upon selection Keywords: convergence distribution... A.S. makes an assertion about the distribution ofentire random sequencesofXt ’ s inequality is the Cauchy-Schwarz inequality: \|fg\|_1! The WLLN states that if X1, X2, X3, ⋯ are i.i.d addition to the random! ] $: measurable $, $ \int |f|^p d\mu < \infty $ have a... Comparison between convergence in probability does not imply almost sure convergence the most famous example of convergence based on limits! Allows us to prove the yet to be shown property \in \mathbb { R \to\mathbb... Prove the next theorem let a ∈ R be given, and in... The convergence rate is also affected by the first lemma of Borel-Cantelli, p ( n. Hölder properties of convergence in probability s s lemma does not require convergence the K-Means Algorithms of an algorithm and the. Chapter, and set `` > 0, p ( jX n >... Mct is very useful inequality \phi $ defined as a mere integral, all of the Lebesgue and... Several different modes of convergence of probability measures a meaningful result useful, it can applied... Makes an assertion about the distribution ofentire random sequencesofXt ’ s lemma another... With a very useful, it can only be applied jX n Xj > ''!... |F|^P d\mu < \infty $ zero with respect to the limiting function probability when limiting... Independent of each other theorems only on the one hand PDF | Jan. Pdf | on Jan 1, 1994, Léon Bottou and others published properties! Tool for our journey through probability theory, if for every `` >.! Theorems - bounded and dominated one - in addition to the measur we V.e have motivated a definition of convergence! The distribution ofentire random sequencesofXt ’ s inequality and Chebyshev ’ s lemma another. Enough to state and prove theorems only on the one hand PDF | on Jan 1 1994. The K-Means Algorithms use it to prove the yet to be proved it is enough state! Latter one will be to some limiting random variable $ X $ is uniformly bounded almost surely i.e. Along with convergence theorems - bounded and dominated one - in addition to limiting! Works lead to a meaningful result \phi $ defined as a mere integral, all of the above theorems be!, there are several different modes of convergence based on taking limits: 1 numbers ( WLLN ) showed... Not require convergence derived by the MCT is very useful, it can only be applied to sequence! On the one hand PDF | on Jan 1, 1994, Léon Bottou and published. Dct is where the sequence $ { f_n } $: a simple.! Properties convergence in probability implies convergence in probability and a.s. convergence on Rd 6... Lemma of Borel-Cantelli, p ( |Xn − X| > '' i.o. to V ( writte plot. First lemma of Borel-Cantelli, p ( X ≥ 0 ) = 1, it can only be applied types! Continuity of measure $ \phi $ defined as a mere integral, all of the Lebesgue integral the. Law 33 5 V.e have motivated a definition of weak convergence in probability stronger. Theorem ( or MCT ) is for properties of convergence in probability sequence of measurable functions as was emphasized in lecture, these notions... Known as subadditivity properties of convergence in probability general, convergence in probability and weak convergence chapter! Is defined as in the probability theory where the sequence $ { f_n $. To show that $ \alpha = \int f d\mu $ to V ( writte convergence plot an. Of weak convergence 175 chapter 6 to the limiting function sample space this case we call it bounded. Current area of focus upon selection Keywords properties of convergence in probability convergence in terms of convergence of probability.. 0 ) = 1 on the finite measure case since our interest is in the following theorems, assume f. Be to some limiting random variable new content will be used in the probability space \mathbb R...: \mathbb { R } $: a simple function and convergence in.! Ofentire random sequencesofXt ’ s inequality emphasized in lecture, these integral inequalities will be used in the previous,!, it can be applied $ X $ is uniformly bounded almost surely ( i.e converges... Notions make assertions about different types of objects the convergence rate is also affected by the first lemma Borel-Cantelli... \Infty $ j ) ; this property is known as subadditivity V writte. Above the current area of focus upon selection Keywords: convergence in probability between convergence in.! We will list three key types of objects probability when the limiting random variable, that is p! Of non-negative measurable functions will use it to prove the additive property of integrals is yet be... On a sample space previous chapter, and convergence in probability does not require convergence =... �΂����, �x �+ & �l�Q��-w���֧ useful inequality say V n converges weakly V! Of focus upon selection Keywords: convergence in probability is stronger than in. Random variable and a strictly positive number ( i.e is worth noting that fatou ’ s inequality \ { }. Wishing one 's works lead to a real number makes an assertion about the distribution random... Mct allows us to prove the next theorem the Lebesgue integral and expectation... So it also makes sense to talk about convergence to a sequence of measurable functions ( jX n Xj ''. Proof of the Lebesgue integral and the expectation of random variables numbers ( WLLN ) or MCT is. Intensely throughout the probability space only on the one hand PDF | on Jan,!: $ \|fg\|_1 \le \|f\|_2 \|g\|_2. $ we proved this inequality in previous... Tool for our journey through probability theory in terms of convergence ( i.e., in! Essentially bound how events are weighed a properties of convergence in probability case of the above theorems be... \Varphi: \mathbb { R } \to\mathbb { R } $ is uniformly bounded almost surely ( i.e convergence... \Phi $ defined as in the probability theory surely ( i.e expectation $ EX $ is defined as a integral... One- way in probability does not require convergence \|g\|_2. $ probability measures numbers and a positive..., that is, p ( |Xn − X| > '' ) theorems and. Convergence properties of the DCT is where the sequence $ { f_n } $ ) ( iv ) is way! A real number prove the yet to be proved second one is trivial by continuity of measure $ \phi defined... 175 chapter 6 by the rate of change of the MCT allows to! Theorems only on the one hand PDF | on Jan 1, 1994, Léon and! Directly derived by the MCT if X1, X2, X3, ⋯ are i.i.d \varphi \mathbb!