Keep the likelihood, posterior and prediction target separate. The temperature example will distinguish an unknown mean from measurement noise.

Given $\theta$, the model tells us which data are plausible. Given data, inference asks which values of $\theta$ remain plausible. This alone is not a claim of causal identification.

Bring: Conditional probability, sums and a Gaussian distribution.

The same joint probability can be factored in two orders:

A coin is either fair ($\theta=0.5$) or head-biased ($\theta=0.8$), with equal prior probabilities. We observe two heads, conditionally independently.

For the observed sequence $D=(H,H,T)$, conditional independence gives

For continuous $\theta$, $p(\theta)$ is a density: interval areas are probabilities and $P(\theta=\theta_0)=0$. A density may exceed 1. This differs from the two discrete candidates on the previous slide.

Assume $Y_i\mid\theta$ are independent Bernoulli variables. Let $S=\sum_{i=1}^nY_i$.

For $0<S<n$, taking logs turns the product into a sum:

If $S=0$ or $S=n$, the maximum is at the boundary, $0$ or $1$; there is no interior derivative-zero solution. Here $S$ is sufficient for $\theta$: the order adds no information under this model.

Use a density supported on the possible coin biases, $0<\theta<1$:

$B(\alpha,\beta)$ makes the density integrate to 1. The parameters act like pseudo-counts in the update; they need not represent actual past flips. Uniform $\mathrm{Beta}(1,1)$ is still a choice of prior.

Start with a Beta prior and observe $S$ heads in $n$ tosses:

The posterior after one batch becomes the prior before the next batch:

The previous picture uses Normal observations with known noise and a flat prior on the mean. The endpoints coincide in that special case; the interpretations still differ.

For the coin model,

On average, yes, in the following precise sense. The average is over possible datasets under the joint model:

The posterior answers which coin biases are plausible. A prediction asks what might happen next.

Let $K$ count heads in $m$ future tosses. All of them are driven by the same unknown $\theta$, so the honest prediction averages over everything $\theta$ might be:

For our coin the posterior was $\mathrm{Beta}(4,3)$, so the batch of $m$ future tosses is $\mathrm{BetaBinomial}(m,4,3)$ — while plugging in the posterior mean $\hat\theta=4/7$ would give $\mathrm{Binomial}(m,4/7)$.

For the coin: Bernoulli/Binomial + Beta → updated Beta → predictions of future heads.

A prior is conjugate to a likelihood when the posterior stays in the prior's family. We can recognise and normalise the result analytically.

Conjugacy is algebraic convenience. It does not establish model fit or make every posterior mean a simple average of prior and sample means. Unknown Normal mean and variance require an appropriate joint prior.

Let $Y_i$ count events in equal observation windows, with a common unknown rate $\lambda$:

We use $a,b$ for Gamma to distinguish them from the coin's Beta parameters. A Gamma scale would be $1/b$, not $b$.

For conditionally independent, equal-exposure counts,

The observations

14, 13, 7, 10, 15, 15, 2, 13, 13, 11, 10, 13, 5, 13, 9, 12, 9, 12, 8, 7.

Twenty districts: $n=20$, total $\sum_i y_i=211$, average $\bar y=10.55$.

The model treats districts as comparable units with one common rate. Unequal areas, search effort or spatial dependence require a richer model or exposure adjustment.

Prior $\mathrm{Gamma}(a,b)=\mathrm{Gamma}(4,\,0.2)$; data $n=20$ districts with $\sum_i y_i=211$, so $\bar y=10.55$.

Return to $\lambda\mid D\sim\mathrm{Gamma}(215,20.2)$. For one comparable new district, $\tilde Y\mid\lambda\sim\mathrm{Poisson}(\lambda)$.

The predictive distribution is Negative Binomial, a Poisson–Gamma mixture. Its probability mass function and parameter convention are in the appendix.

Let $Y_i\mid\theta\sim\mathcal N(\theta,\sigma^2)$ independently with $\sigma^2$ known, and $\theta\sim\mathcal N(\mu_0,\tau_0^2)$.

Only the shape in $\theta$ matters, so every factor without a $\theta$ in it goes into the constant:

Reading $A$ and $B/A$ back gives the posterior in the form worth remembering.

Prior $\theta\sim\mathcal N(20,2^2)$; measurement noise has standard deviation $3$; four readings average $23$.

Step 1 — state each source as a precision, since that is the currency that adds:

Step 3 — weight each mean by the precision that carries it. Dividing through by the total precision $25/36$ turns the formula into an ordinary weighted average:

The true temperature $\theta$ has prior $\mathcal N(20,4)$. One sensor reading is $y=22$, with $y\mid\theta\sim\mathcal N(\theta,1)$. The second parameter here is variance.

Predict: Will the posterior mean be closer to 20 or to 22? Will its variance equal the variance of a new reading?

Keep the prior and the observed value 22. Change the sensor variance from 1 to 4, for both this and the next reading. Find the posterior mean, posterior variance and predictive variance.

The updated mean parameter is $\theta\mid D\sim\mathcal N(\mu_1,\tau_1^2)$. A new reading is $\tilde Y=\theta+\epsilon$, with independent $\epsilon\sim\mathcal N(0,\sigma^2)$.

Suppose each service request is delivery, pickup or return. The probabilities form a vector $\theta$ with non-negative entries adding to 1.

Start from $\mathrm{Dirichlet}(1,1,1)$ and observe ten requests: 6 deliveries, 4 pickups, 0 returns.

We keep $y_i=x_i^\top w+\epsilon_i$ throughout.

Include the intercept by adding a constant feature 1. For the illustrative data $(x,y)=(0,1),(1,2),(2,2)$,

In general $X\in\mathbb R^{n\times d}$, $w\in\mathbb R^d$ and $y\in\mathbb R^n$. Here $d$ includes the intercept. Keep this orientation throughout the derivation.

Choose weights to minimise squared residuals: