---
title: Models built forward, for the measure theorist
summary: Why an abstract probability space serves modelling as an abstract vector space serves linear algebra, why the way back to coordinates is harder in probability, and how building a model forward from densities restores it, so that every random variable factors through the quantities and every expectation is an explicit integral.
---

# Models built forward, for the measure theorist

This note is for the reader who knows measure theory far better than statistics, and who wants to
know what a probabilistic model *is* before being told how to compute with one. Its argument is an
analogy with linear algebra, and its conclusion is that the models skoro builds sit in a corner of
probability theory where the analogy is exact: one can reason abstractly and compute concretely,
and move between the two without meeting the edge cases that usually make the move painful.

## The abstraction, and the way back

Linear algebra is done in an abstract vector space $V$. A mathematician working there never says
which coordinate of which $\R^\dimension$ holds which quantity of interest: vectors are added,
subspaces $W \subseteq V$ are taken, linear maps are composed, and the dimension can be anything.
When a computation is needed, a basis is chosen, $V$ becomes $\R^\dimension$, and the abstract
statements become statements about arrays of numbers. The way back is a single theorem: every
finite-dimensional vector space is isomorphic to a coordinate space, and every choice of basis
gives such an isomorphism.

Probability is done in an abstract probability space $(\Omega, \scrF, \rmP)$. A modeller
working there never says what $\Omega$ is, nor which sets are in $\scrF$, nor what
$\rmP$ assigns to them. Instead they introduce random variables $\randomPoint : \Omega \to \manifold$,
measurable maps into the spaces where the quantities of the model live, and speak only of those:
their laws, their joint laws, the independence of one from another, the conditional law of one
given another. Any number of them may be introduced, and new ones may be introduced later, provided
the space is rich enough to carry them. The abstraction does for the modeller what $V$ does for the
linear algebraist: it relieves them of a coordinate system they do not need.

The way back to coordinates is where the analogy usually breaks. Several things can go wrong at
once:

- **The space is underdetermined.** Many triples $(\Omega, \scrF, \rmP)$ carry random
  variables with the same joint law, and $\scrF$ may hold far more than the model ever
  mentions. Nothing in the model says which.
- **Conditioning needs regularity.** The conditional law of $\randomPoint$ given $\otherRandomPoint$,
  for $\randomPoint$ with values in $(\manifold, \scrX)$ and $\otherRandomPoint$ in
  $(\otherManifold, \scrY)$, defined as a kernel
  $\rmP_{\randomPoint \mid \otherRandomPoint} : \scrX \times \otherManifold \to [0, 1]$, written
  $\rmP_{\randomPoint \mid \otherRandomPoint}(A \mid \otherPoint)$ and characterized by

  $$
  	\rmP(\randomPoint \in A,\ \otherRandomPoint \in B) = \int_B \rmP_{\randomPoint \mid \otherRandomPoint}(A \mid \otherPoint)\, \rmP(\otherRandomPoint \in \dd\otherPoint)
  	\qquad \text{for all } A \in \scrX,\ B \in \scrY, @tag(regular-conditional)
  $$

  a *regular conditional distribution*, exists only under assumptions on the spaces (standard Borel is the
  usual one), and even then only up to null sets of $\otherRandomPoint$'s law. Conditioning on an event of
  probability zero, which is what observing a continuous quantity does, depends on which version is
  chosen; the Borel–Kolmogorov paradox is the classic example of two natural versions disagreeing.
- **Densities need a reference.** A density is a Radon–Nikodym derivative, and exists only when
  the law is absolutely continuous with respect to a chosen measure. The law of a function of
  quantities, such as $(\randomPoint, \randomPoint)$ or a projection, often has none.
- **Existence needs a theorem.** Writing down a family of random variables does not make a space
  that carries them. Kolmogorov's extension theorem builds one from consistent finite-dimensional
  laws, but asks for topological regularity of the spaces to do so.

So the modeller who wants to compute, to write a probability as an integral and evaluate it, has
to justify every step back from $(\Omega, \scrF, \rmP)$, and usually does not.

## A corner of probability where the way back is clean

Applied models are almost never as general as the theory allows. They are built from two
ingredients, and both remove an edge case.

**Seeds with densities.** Every quantity lives in a space with a *reference measure*: Lebesgue
measure on $\R^\dimension$ (or an open subset of it, such as the positive orthant), or counting
measure on a countable set, which may itself sit inside a vector space, such as the integers. Every
distribution a quantity is drawn from is absolutely continuous with respect to its reference
measure, so it is given by a function.

> **Theorem (Radon–Nikodym).** Let $\nu$ be a $\sigma$-finite measure on a measurable space $(\manifold, \scrX)$, and $\mu$ a
> finite measure on it with $\mu \ll \nu$, that is, $\nu(A) = 0$ implies $\mu(A) = 0$. Then there is
> a measurable $\density : \manifold \to [0, \infty)$ with
>
> $$
> 	\mu(A) = \int_A \density \,\dd\nu \qquad \text{for every } A \in \scrX,
> $$
>
> and any two such functions agree $\nu$-almost everywhere. One writes $\density = \dd\mu / \dd\nu$
> and $\mu(\dd\point) = \density(\point)\, \nu(\dd\point)$.

With respect to Lebesgue measure, $\density$ is a *probability density function*:
$\mu(\dd\point) = \density(\point)\, \dd\point$, its values are not probabilities, every single
point has measure zero, and $\density$ is determined only up to a Lebesgue-null set. With respect to
counting measure, nothing is null but the empty set, and the density is pinned down completely by
the mass it gives to points.

> **Proposition (A mass function is its measure on points).** Let $\manifold = \{\point_0, \point_1, \dots\}$ be countable, with every subset measurable, and let
> $\nu = \sum_k \delta_{\point_k}$ be counting measure. Every probability measure $\mu$ on
> $\manifold$ is absolutely continuous with respect to $\nu$, its density is unique, and it is
>
> $$
> 	\density(\point_k) = \mu(\{\point_k\}), \qquad \text{so that} \qquad
> 	\mu(\dd\point) = \sum_k \density(\point_k)\, \delta_{\point_k}(\dd\point) .
> $$

<Proof>

If $\nu(A) = 0$ then $A = \emptyset$, so $\mu \ll \nu$, and $\nu$ is $\sigma$-finite because
$\manifold$ is countable. Countable additivity gives
$\mu(A) = \sum_{\point_k \in A} \mu(\{\point_k\}) = \int_A \density \,\dd\nu$ with
$\density(\point_k) = \mu(\{\point_k\})$, and uniqueness almost everywhere is uniqueness
everywhere, since the only null set is empty.

</Proof>

This is why $\density$ is called a *probability mass function* in that case, and why the two cases
can be treated as one: a density is always with respect to a reference measure, and a mass function
is a density with respect to counting measure.

Nothing in what follows needs these two in particular: a reference measure $\nu$ need only be
$\sigma$-finite, for [%full](statement:radon-nikodym) and Tonelli's theorem to apply. But Lebesgue
measure and counting measure, and products of them, are typically the only reference measures
adopted in practice, and they are the ones meant below.

**Quantities introduced one at a time.** Each quantity is drawn from a distribution whose
parameters may depend on any quantity introduced before it, through any measurable map, however
nonlinear. A vehicle's position is drawn given its previous position, pushed through a motion
model; a range measurement is drawn given the vehicle and a landmark, through the distance between
them.

> **Definition (A model built forward).** A _model built forward_ is a finite sequence of _quantities_ $i = 1, \dots, n$, each with
>
> 1. a space $(\manifold_i, \scrX_i)$ and a $\sigma$-finite _reference measure_ $\nu_i$ on it:
>    Lebesgue measure on $\R^{\dimension_i}$ or on an open subset of it, or counting measure on a
>    countable set; and
> 2. a _conditional density_: a measurable function
>    $\density_i : \manifold_i \times \manifold_1 \times \cdots \times \manifold_{i-1} \to [0, \infty)$
>    such that $\density_i(\cdot \mid \point_{<i})$ is a probability density with respect to $\nu_i$
>    for every $\point_{<i} = (\point_1, \dots, \point_{i-1})$.
>
> Its _joint density_ is
> $\density(\point) = \prod_{i=1}^n \density_i(\point_i \mid \point_{<i})$, with respect to the
> product measure $\nu = \nu_1 \otimes \cdots \otimes \nu_n$ on $(\manifold, \scrX)$, where
> $\manifold = \manifold_1 \times \cdots \times \manifold_n$ is the product space and
> $\scrX = \scrX_1 \otimes \cdots \otimes \scrX_n$ the product $\sigma$-algebra.

A conditional density written as $\density_i(\point_i \mid \point_{<i}) = \phi_i(\point_i; \eta_i(\point_{<i}))$,
a family of densities $\phi_i$ with parameters computed from earlier quantities by a measurable map
$\eta_i$, is measurable as a composition of measurable maps, so any such nonlinear dependence is
allowed. Nothing else is: there are no quantities without a density, and no dependence on a
quantity introduced later.

## What the restriction buys

The first thing it buys is a canonical coordinate space, in the role of $\R^\dimension$.

> **Proposition (The joint law of a model built forward).** The measure $\mu(\dd\point) = \density(\point)\, \nu(\dd\point)$ is a probability measure on
> $\manifold$. For each $i$, its marginal on the first $i$ quantities has density
> $\prod_{j \le i} \density_j(\point_j \mid \point_{<j})$ with respect to
> $\nu_1 \otimes \cdots \otimes \nu_i$, and
>
> $$
> 	\kernel_i(A \mid \point_{<i}) = \int_A \density_i(\point_i \mid \point_{<i})\, \nu_i(\dd\point_i)
> $$
>
> is a version of $\rmP_{\randomPoint_i \mid \randomPoint_{<i}}$, the regular conditional distribution of the
> $i$-th quantity given the ones before it.

<Proof>

The function $\density$ is non-negative and measurable on the product. By Tonelli's theorem,
integrating over $\point_n$ first, then $\point_{n-1}$, and so on, each factor
$\density_i(\cdot \mid \point_{<i})$ integrates to $1$ against $\nu_i$ whatever the earlier
coordinates are, so $\int \density \,\dd\nu = 1$, and stopping after the last $n - i$ integrals
leaves the marginal density on the first $i$. For measurable $B \subseteq \manifold_{<i}$ and
$A \subseteq \manifold_i$, Tonelli again gives
$\mu(\point_{<i} \in B,\ \point_i \in A) = \int_B \kernel_i(A \mid \point_{<i})\, \mu_{<i}(\dd\point_{<i})$,
with $\mu_{<i}$ the marginal on the first $i - 1$, and $\kernel_i(\cdot \mid \point_{<i})$ is a probability
measure for every $\point_{<i}$ and measurable in $\point_{<i}$ for every $A$, again by Tonelli.

</Proof>

The probability space can then be taken to be the product space
$(\Omega, \scrF, \rmP) = (\manifold, \scrX, \mu)$, and the quantities the coordinate projections
from $(\Omega, \scrF)$ to $(\manifold_i, \scrX_i)$,

$$
	\randomPoint_i : \Omega \to \manifold_i, \qquad \randomPoint_i(\omega) = \randomPoint_i(\point_1, \dots, \point_n) = \point_i .
$$

This is the analogue of choosing a basis: the quantities are the coordinates. Nothing is lost by the choice,
because every statement the model makes is about the joint law of its quantities, and that law is
$\mu$ on any space that carries them.

Nor is the restriction to finitely many quantities essential. The construction never used anything
about the spaces beyond measurability, and it extends to infinitely many quantities built forward,
with no topological assumption, by a theorem that is the right one for models built this way.

> **Theorem (Ionescu-Tulcea).** Let $(\manifold_i, \scrX_i)_{i \ge 1}$ be any measurable spaces, $\mu_1$ a probability measure
> on $\manifold_1$, and for each $i \ge 2$ let $\kernel_i$ be a probability kernel from
> $\manifold_1 \times \cdots \times \manifold_{i-1}$ to $\manifold_i$. There is a unique probability
> measure $\mu$ on $\prod_{i \ge 1} \manifold_i$, with its product $\sigma$-algebra, whose marginal on
> every finite initial product is $\mu_1 \otimes \kernel_2 \otimes \cdots \otimes \kernel_i$.

Kolmogorov's theorem builds a law from consistent marginals, and needs the spaces to be nice for
the marginals to determine one. Ionescu-Tulcea builds it from successive conditional laws, which is
exactly what a model built forward provides, and needs nothing. Appending a quantity is extending a
basis: the old coordinates keep their joint law, and a new one is added with its conditional law
given them.

The second thing the restriction buys is a guarantee about what the model can say. A model speaks
only of its quantities, so the random variables it can express are the $\sigma(\randomPoint)$-measurable ones,
$\randomPoint = (\randomPoint_1, \dots, \randomPoint_n)$, and those are exactly the functions of the quantities.
That is a lemma about any measurable map whatever, on any space, not only about the quantities.

> **Lemma (Doob–Dynkin).** Let $\otherRandomPoint : \Omega \to \otherManifold$ be any measurable map from a measurable space
> $(\Omega, \scrF)$ to any measurable space $(\otherManifold, \scrY)$, and $\bfz : \Omega \to \R$.
> Then $\bfz$ is $\sigma(\otherRandomPoint)$-measurable if and only if $\bfz = g \circ \otherRandomPoint$
> for some measurable $g : \otherManifold \to \R$.

<Proof>

If $\bfz = g \circ \otherRandomPoint$, then $\bfz^{-1}(C) = \otherRandomPoint^{-1}(g^{-1}(C)) \in \sigma(\otherRandomPoint)$
for every Borel $C \subseteq \R$. Conversely, every set in $\sigma(\otherRandomPoint)$ is
$\otherRandomPoint^{-1}(B)$ for some $B \in \scrY$, so an indicator $\mathbf 1_{\otherRandomPoint^{-1}(B)}$
is $\mathbf 1_B \circ \otherRandomPoint$, and by linearity every $\sigma(\otherRandomPoint)$-measurable
simple function is $g \circ \otherRandomPoint$ for a simple $g$. A
$\sigma(\otherRandomPoint)$-measurable $\bfz$ is a pointwise limit of such functions
$g_k \circ \otherRandomPoint$; the set where $(g_k)$ converges is measurable, and $g = \lim_k g_k$
there and $0$ elsewhere is measurable with $\bfz = g \circ \otherRandomPoint$, because the sequence
converges at every point of the image of $\otherRandomPoint$.

</Proof>

In a model built forward, apply it to the quantities taken together. On the product space,
$\randomPoint = (\randomPoint_1, \dots, \randomPoint_n) : \Omega \to \manifold$ is the identity map, since
$\Omega = \manifold$ and each $\randomPoint_i$ is a coordinate projection. So
$\sigma(\randomPoint) = \scrX = \scrF$: every event is an event about the quantities, every
scalar-valued random variable on $\Omega$ is $\sigma(\randomPoint)$-measurable, and by the lemma
every one of them takes the form $g \circ \randomPoint$. On any other space that carries the
quantities, $\scrF$ may be larger than $\sigma(\randomPoint)$, but the random variables the model
can express are still the $\sigma(\randomPoint)$-measurable ones, and the conclusion is the same.

> **Corollary (Every expectation is an integral against the joint density).** Let a model built forward be carried by quantities $\randomPoint = (\randomPoint_1, \dots, \randomPoint_n)$ on a probability space
> $(\Omega, \scrF, \rmP)$, so that $\randomPoint$ has law $\mu$. Every random variable $\bfz$ the model
> can express, that is every $\sigma(\randomPoint)$-measurable one, is $g \circ \randomPoint$ for a measurable $g$, and when
> it is integrable,
>
> $$
> 	\E[\bfz] = \int_{\manifold} g(\point)\, \density(\point)\, \nu(\dd\point)
> 	= \int_{\manifold_1} \!\!\cdots \int_{\manifold_n} g(\point) \prod_{i=1}^n \density_i(\point_i \mid \point_{<i})\, \nu_n(\dd\point_n) \cdots \nu_1(\dd\point_1) .
> $$

<Proof>

The first claim is [%full](statement:doob-dynkin), with $\otherRandomPoint$ there the quantities
$\randomPoint$ here. For the second, the law of $\randomPoint$ is $\mu$, so
$\E[g \circ \randomPoint] = \int g \,\dd\mu$ by the change of variables for image measures, and
$\int g \,\dd\mu = \int g \density \,\dd\nu$ because $\density = \dd\mu / \dd\nu$. The iterated form
is Fubini's theorem, which applies because $g \density$ is $\nu$-integrable.

</Proof>

This is the probabilistic half of the analogy made precise. In linear algebra, a vector of $V$ is
a combination of basis vectors and a linear functional is a combination of coordinates. Here, a
random variable is a function of the quantities, an event of the model is a measurable set of
their values, and its probability, an expectation, a marginal or a moment is an integral against
the product of the conditional densities, in the order the model was built. The integral is
explicit, though it is rarely available in closed form; estimating it is what inference
algorithms are for.

## Mapped values

A model built forward carries more than its quantities. A *mapped value*
$\otherRandomPoint = f(\randomPoint)$, for a measurable $f$, such as a distance between two
positions, a predicted measurement, or the parameters of a later kernel, is a random variable of the
model like any other, and by [%full](statement:doob-dynkin) every random variable the model can
express is one.

A mapped value is a *target*, determined by the quantities, not a further source of randomness.
Once the quantities are drawn or fixed at $\point$, it is computed, $\otherPoint = f(\point)$, and
never drawn. So nothing about it needs to be sampled or weighed on its own account. Computing $f$
on draws of $\randomPoint$ samples its law $f_\# \mu$; a weight attached to a draw $\point$ is
already the weight of the pair $(\point, f(\point))$, since $\otherPoint$ adds no randomness to
it; and its expectations are those of [%full](statement:integral) for the function $h \circ f$:

$$
	\E[h(\otherRandomPoint)] = \int_{\manifold} h(f(\point))\, \density(\point)\, \nu(\dd\point) .
$$

Its law can be awkward (concentrated on a lower-dimensional set, for instance, with no density
with respect to any reference measure), but nothing requires that density: a model's densities are
those of its quantities, and a mapped value enters them only as the argument of a later kernel or
likelihood factor, through the quantities it is computed from.

## Conditioning and likelihoods

Observing quantities is conditioning on them, and in this corner it needs no existence theorem:
the conditional law is written down from the densities.

> **Proposition (Conditioning is division of densities).** Split the quantities of a model built forward into observed ones $O$ and free ones $F$, and let
>
> $$
> 	\density_O(\point_O) = \int \density(\point_F, \point_O)\, \nu_F(\dd\point_F) .
> $$
>
> Then $\density_O$ is a density of the law of $\randomPoint_O$, finite $\nu_O$-almost everywhere, and the
> kernel
>
> $$
> 	\rmP_{\randomPoint_F \mid \randomPoint_O}(\dd\point_F \mid \point_O) = \frac{\density(\point_F, \point_O)}{\density_O(\point_O)}\, \nu_F(\dd\point_F)
> 	\qquad \text{where } 0 < \density_O(\point_O) < \infty,
> $$
>
> defined arbitrarily elsewhere, is a regular conditional distribution of $\randomPoint_F$ given $\randomPoint_O$.

<Proof>

By Tonelli, $\density_O$ is measurable and $\int \density_O \,\dd\nu_O = 1$, so it is a density of
the law of $\randomPoint_O$ and is finite $\nu_O$-almost everywhere. The set where it is $0$ has probability
$0$ under that law, and so does the set where it is infinite. For measurable $A$ and $B$,

$$
	\rmP(\randomPoint_F \in A,\ \randomPoint_O \in B)
	= \int_B \int_A \density(\point_F, \point_O)\, \nu_F(\dd\point_F)\, \nu_O(\dd\point_O)
	= \int_B \rmP_{\randomPoint_F \mid \randomPoint_O}(A \mid \point_O)\, \density_O(\point_O)\, \nu_O(\dd\point_O),
$$

the two exceptional sets contributing nothing to either side, which is the defining property of a
regular conditional distribution.

</Proof>

The version is chosen, not discovered: changing the densities on a null set, or the reference
measures by a change of coordinates, changes $\rmP_{\randomPoint_F \mid \randomPoint_O}$ on a null set of observations, which is all the
freedom the Borel–Kolmogorov paradox exploits. In a model built forward the choice is made once,
by the densities and reference measures the model was written with, and every observation is
conditioned on by that same rule. That is also why two densities may only be compared, or divided,
when they are with respect to the same reference measure.

The same computation explains a factor that scores no quantity. Multiplying the joint density by a
non-negative function and normalizing is a change of measure, and it is always conditioning on data
that the model does not carry as a quantity.

> **Proposition (A likelihood factor is a change of measure).** Let $L : \manifold \to [0, \infty)$ be measurable with $0 < Z = \int L \,\dd\mu < \infty$. Then
> $\mu_L(\dd\point) = L(\point)\, \mu(\dd\point) / Z$ is a probability measure with density
> $L \density / Z$ with respect to $\nu$. If $L(\point) = h(\otherPoint_0 \mid \point)$ for a
> conditional density $h$ of a further quantity $\otherRandomPoint$ appended to the model, then $\mu_L$ is the
> conditional law of $\randomPoint$ given $\otherRandomPoint = \otherPoint_0$, as conditioning by densities gives it.

<Proof>

$\mu_L$ is a probability measure because $\int L \,\dd\mu = Z$, and its density with respect to
$\nu$ is $L \density / Z$. With $\otherRandomPoint$ appended, the joint density is $\density(\point)\, h(\otherPoint \mid \point)$,
so [%full](statement:conditioning), with $\randomPoint$ free and $\otherRandomPoint$ observed at $\otherPoint_0$, gives the
conditional density $\density(\point)\, h(\otherPoint_0 \mid \point) / \density_O(\otherPoint_0)$,
where the marginal density of that statement is $\density_O(\otherPoint_0) = \int h(\otherPoint_0 \mid \point)\, \mu(\dd\point) = Z$.

</Proof>

So a *likelihood factor* is exactly what its name says: the likelihood of an observation that is
not a quantity of the model, already conditioned on.

## Parameters and improper priors

An *improper prior*, a "density" with infinite integral such as a flat density on a line, cannot be
a seed, since it is not a probability measure. It is expressed instead by a *parameter* and a
*score*. A parameter $\theta$ takes values in a space $\Theta$ with a reference measure
$\nu_\Theta$, like a quantity, but enters with no kernel: the model becomes a family
$(\mu_\theta)_{\theta \in \Theta}$, each of its kernels free to depend on $\theta$ measurably,
as on any earlier quantity. A score is a likelihood factor $S(\theta) \ge 0$ on the parameter,
$S \equiv 1$ for the flat prior. With $\point_O$ observed, the model then describes

$$
	\pi(\dd\theta, \dd\point_F) \propto S(\theta)\, \density_\theta(\point_F, \point_O)\,
	\nu_\Theta(\dd\theta)\, \nu_F(\dd\point_F), @tag(parameter)
$$

a $\sigma$-finite measure, which is a posterior, after normalizing, exactly when its total mass is
finite and positive. Whether it is depends on what is observed: it is a condition on the query, not
on the model. Either way, [](eq:parameter) is the objective of optimization, and maximizing
$\log S(\theta) + \log \density_\theta(\point_F, \point_O)$ is maximum a posteriori or, with
$S \equiv 1$, maximum likelihood estimation.

In every other respect a parameter is a quantity: it is a coordinate of the space the model
describes, functions of it are random variables of the model, and it is conditioned on or left free
like the rest. The one difference is that nothing generates it. An optimizer must be given its
starting value, and a sampler must draw it from a proposal $\proposal_\Theta$ of its own, a
density that may depend on earlier quantities like a kernel, and contributes
$S / \proposal_\Theta$ in all to the weight.

## How skoro is built on this

Skoro is designed by this philosophy. A skoro model is not a function $\density$ to be evaluated,
but a probability space built forward, quantity by quantity, and the library sets out to let its
users work with that space in every way the measure theory above allows, of which evaluating
$\density$ is only one.

- **Quantities are specified by their augmenting kernels.** Each quantity $\randomPoint_i$ enters the model
  with the kernel that augments the space built so far,

  $$
  	\kernel_i(\dd\point_i \mid \point_{<i}) = \density_i(\point_i \mid \point_{<i})\, \nu_i(\dd\point_i),
  	@tag(augmenting-kernel)
  $$

  the kernel of [%full](statement:joint-law), so that the model is
  $\mu = \kernel_1 \otimes \kernel_2 \otimes \cdots \otimes \kernel_n$ and appending a quantity is
  appending a kernel, as in [%full](statement:ionescu-tulcea).
- **Likelihoods are built incrementally too.** A likelihood factor $L_j$ may be added at any point,
  as a function of the quantities that exist when it is added, and the model's likelihood is their
  product $L = \prod_j L_j$. The measure the model describes is then $L\, \mu$, normalized, by
  [%full](statement:likelihood-factor).
- **The model can be sampled, with weights.** Drawing from the kernels in order samples $\mu$. In a
  query that observes $\randomPoint_O$, drawing the free quantities from their kernels, with the observed
  ones held at their values, samples a measure whose density with respect to $\nu_F$ is the product
  of the free quantities' conditional densities, and the target of the query, the conditional law of
  [%full](statement:conditioning) reweighted by $L$, differs from it by the weight

  $$
  	\weight(\point_F) = L(\point) \prod_{i \in O} \density_i(\point_i \mid \point_{<i}),
  	@tag(sampling-weight)
  $$

  the likelihood of the observations and of the likelihood factors at the drawn point. A kernel
  drawn from a sampler with a density of its own, $\proposal_i \ne \density_i$, contributes
  $\density_i / \proposal_i$ to the weight as well. The weight distorts the sampling measure into the
  target only up to a constant, $\density_O(\point_O)$ and $Z$ among others, and that is all an
  algorithm needs: a self-normalized estimate
  $\E[\weight\, h] / \E[\weight]$, a Metropolis–Hastings ratio, a gradient of $\log \weight$ or a
  resampling step in a particle filter is unchanged when every weight is multiplied by the same
  number.
- **Parameters are quantities without a seed.** An improper prior is a parameter with a score
  [](eq:parameter). The model carries a proposal for it, from which a sampler draws it and an
  optimizer may take its starting value, and in every other way it is treated as a quantity.
- **Mapped values live in the model.** A mapped value $\otherRandomPoint = f(\randomPoint)$ is
  carried by the model as a value computed from its arguments: sampling them samples it, weighing
  them weighs it, its expectations are integrals by [%full](statement:integral), and, most of all,
  later kernels may be parameterized by it. A kernel
  $\kernel_j(\dd\point_j \mid \point_{<j}) = \phi_j(\point_j; \eta_j(\point_{<j}))\, \nu_j(\dd\point_j)$ is
  written by building its parameters $\eta_j$ out of mapped values, composed as freely as any measurable map allows,
  and derivatives flow through them to the quantities they are built from.
- **Everything is a logarithm.** Every product above becomes a sum once its logarithm is taken:

  $$
  	\log \density(\point) = \sum_{i=1}^n \log \density_i(\point_i \mid \point_{<i}),
  	\qquad
  	\log \weight(\point_F) = \sum_j \log L_j(\point) + \sum_{i \in O} \log \density_i(\point_i \mid \point_{<i}),
  $$

  and skoro states a model in that form, as a sum of log-densities, its
  [terms](/archive/core/math.md#the-probabilistic-graph), one for each kernel, likelihood
  factor and score. Sums suit what is done with them. A product of many densities underflows where a
  sum of their logarithms does not; the derivative of a sum is the sum of the derivatives of its
  terms, each reading only the few quantities it involves; and a normalizing constant becomes an
  additive constant, which every algorithm that ignores the normalization simply drops.
- **Dependence on predecessors is a directed graph.** A quantity's density seldom reads every
  quantity before it: it reads a few, its *parents* $\operatorname{pa}(i) \subseteq \{1, \dots, i - 1\}$,
  so that $\density_i(\point_i \mid \point_{<i}) = \density_i(\point_i \mid \point_{\operatorname{pa}(i)})$.
  The quantities, with an edge from each parent to its child, form the model's *dependence graph*.
  It is acyclic, because every edge points forward in the order the model was built, and the joint
  density factorizes over it:

  $$
  	\density(\point) = \prod_{i=1}^n \density_i(\point_i \mid \point_{\operatorname{pa}(i)}) .
  $$

  The dependence graph serves intuition, since a model can be read off a picture of it, and
  computation, since its sparsity is what makes large models tractable: a quantity's Markov
  blanket, the terms that touch it, an order in which to generate, and conditional independences
  are all read off it. Skoro keeps two graphs in its place, the
  [probabilistic graph](/archive/core/math.md#the-probabilistic-graph) of quantities and terms,
  and the [compilation graph](/archive/core/math.md#the-compilation-graph) of the values each
  term is computed from, over which derivatives are taken; both are drawn there. Contracting each
  term of the probabilistic graph into edges from the quantities it is conditioned on to the
  quantity it scores gives back the dependence graph.

And so on: the gradient of $\log \density$ or of $\log \weight$ in tangent charts for optimizers and
Hamiltonian samplers, marginalizing quantities out when they retire, conditioning anew in every
query, and holding several models at once when the structure itself is uncertain. Each is an
operation on the probability space the kernels built, rather than on a single density function.