Laplace's Rule of Succession

Assume you run $n$ independent equiprobable failure/success trials which ended up with $s$ successful outcomes. In the absence of additional information, it is natural (or at least customary) to view $\displaystyle p=\frac{s}{n}$ as the probability of success. Laplace's Rule of Succession suggests that, in some circumstances, an estimate $\displaystyle p=\frac{s+1}{n+2}$ is more useful. Of course, for large $n$, the two estimates are hardly distinguishable. However, for a small number of trials, the latter is (often) more meaningful. For example, assume a few trials have all ended in failure; $\displaystyle p=\frac{s}{n}$ in this case will be $0$, implying that a trial has no chance of success. However, it is rather obvious that to reach such a clear-cut conclusion on the basis of a small number of trials would be imprudent. Laplace's formula shows one way around that difficulty. Similarly, of a small number of trials all end up with successful outcomes, Laplace's formula $\displaystyle \frac{n+1}{n+2}$ leaves door open to a possibility of failure, while the usual $\displaystyle \frac{n}{n}=1$ does not.

I shall start with following [Feller] in deriving Laplace's formula for $s=n$.

Let there be $N+1$ urns each containing $N$ balls such that the urn #$k$ contains $k$ red and $N-k$ blue balls ($k=0,1,\ldots , N$.) At the first stage of the experiment we choose a random urn (with the probability of $\frac{1}{N+1}$) and then proceed to pick up balls from the chosen urn. After the ball's color has been recorded, the ball is returned back to the urn. Assume the red ball showed up $s$ times (event $A$) and - on the basis of that observation - predict the probability of the red ball showing up on the next trial (event $B$). We are thus looking into the conditional probability $P(B|A)$. Note that

$\displaystyle P(B|A) = \frac{P(AB)}{P(A)}.$

For the urn with $k$ red balls, the probability $P_{k}(s, n)$ of having $s$ red balls in $n$ trials is

$\displaystyle P_{k}(s, n) = {n \choose s}\frac{k^{s}(N-k)^{n-s}}{N^n}$,

where $\displaystyle {n \choose s}$ is the number of combinations of $s$ out of $n$ symbols. In case, where $s = n$, $\displaystyle P_{k}(n, n) = (\frac{k}{N})^{n}$ and $P(AB) = P(B)$, making $P(B|A)=P(B)/P(A)$ so that what is needed is to evaluate the two probabilities $P(A)$ and $P(B)$ separately.

Define $Q(k)$ as the probability of having a string of $k$ successes in $k$ trials. Then $P(A) = Q(n)$ and $P(B) = Q(n+1)$. But

$\displaystyle Q(n) = \frac{1}{N+1}\sum_{k=0}^{N}\left(\frac{k}{N}\right)^{n}$,

which can be viewed as a Riemann sum approximation of the integral $\displaystyle \int_{0}^{1}t^{n}dt = \frac{1}{n+1}$. It follows that, for $s=n$, indeed $\displaystyle P(B|A)=\frac{Q(n+1)}{Q(n)}=\frac{n+1}{n+2}$.

In case, where $s\lt n$, no longer $P(AB)=P(B)$, and we have to look for other ways to evaluate $P(B|A)$.

... to be continued ...

References

W. Feller, An Introduction to Probability Theory and Its Applications, Vol. 1, Wiley, 3rd Edition, 1968, V.2

71547745