ABC in Sydney, July 3, 2014

1. Intro to ABC Example Conclusion
an introduction to
Approximate Bayesian Computation
Matt Moores
Mathematical Sciences School
Queensland University of Technology
Brisbane, Australia
ABC in Sydney
July 3, 2014

2. Intro to ABC Example Conclusion
Motivation
Inference for a parameter θ when it is:
impossible
or very expensive
to evaluate the likelihood p(y|θ)
ABC is a likelihood-free method for approximating
the posterior distribution
π(θ|y)
by generating pseudo-data from the model:
w ∼ f(·|θ)

3. Intro to ABC Example Conclusion
Likelihood-free rejection sampler
Algorithm 1 Likelihood-free rejection sampler
1: Draw parameter value θ ∼ π(θ)
2: Generate w ∼ f(·|θ )
3: if w = y (the observed data) then
4: accept θ
5: end if
But if the observations y are continuous
(or the space y ∈ Y is enormous)
then P(w = y) ≈ 0
Tavar´e, Balding, Griffith & Donnelly (1997) Genetics 145(2)

4. Intro to ABC Example Conclusion
ABC tolerance
accept θ if δ(w, y) <
where
> 0 is the tolerance level
δ(·, ·) is a distance function
(for an appropriate choice of norm)
Inference is more exact when is close to zero. but
more proposed θ are rejected
(tradeoff between accuracy & computational cost)
Pritchard, Seielstad, Perez-Lezaun & Feldman (1999) Mol. Biol. Evol. 16(12)

5. Intro to ABC Example Conclusion
Summary statistics
Computing δ(w, y) for w1, . . . , wn and y1, . . . , yn
can be very expensive for large n
Instead, compute summary statistics s(y)
e.g. sufficient statistics
(only available for exponential family)

6. Intro to ABC Example Conclusion
Sufficient statistics
Fisher-Neyman factorisation theorem:
if s(y) is sufficient for θ
then p(y|θ) = f(y) g (s(y)|θ)
only applies to Potts, Ising, exponential random
graph models (ERGM)
otherwise, selection of suitable summary
statistics can be a very difficult problem

7. Intro to ABC Example Conclusion
ABC rejection sampler
Algorithm 2 ABC rejection sampler
1: for all iterations t ∈ 1 . . . T do
2: Draw independent proposal θ ∼ π(θ)
3: Generate w ∼ f(·|θ )
4: if s(w) − s(y) < then
5: set θt ← θ
6: else
7: set θt ← θt−1
8: end if
9: end for
Approximates π(θ|y) by π (θ | s(w) − s(y) < )
Marin, Pudlo, Robert & Ryder (2012) Stat. Comput. 22(6)
Marin & Robert (2014) Bayesian Essentials with R §8.3

8. Intro to ABC Example Conclusion
A trivial (counter) example
Gaussian with unknown mean:
y ∼ N(µ, 1)
natural conjugate prior:
π(µ) ∼ N(0, 106
)
sufficient statistic:
¯y = 1
n
n
i=1 yi
posterior is analytically tractable:
π(µ|y) ∼ N (m , s2
)
where
1
s2 = n
1
+ 1
106
m = s2 n¯y
1
+ 0 = n¯y
n+10−6
∴ no need for ABC (nor MCMC) in practice

9. Intro to ABC Example Conclusion
R code
π(µ|y)
1.5 2.0 2.5 3.0 3.5 4.0 4.5
0.00.20.40.60.8
§
y ← rnorm (n=5, mean=3, sd=1)
n ← length ( y )
ybar ← sum( y )/n
post s ← 1/(n + 1e−6)
post m ← post s ∗ n∗ ybar
post sim ← rnorm (10000 , post m, sd=sqrt ( post s ))

10. Intro to ABC Example Conclusion
now with ABC
π(µ)
−4000 −2000 0 2000 4000
0e+002e−044e−04
πε(µ | δ(s(w), s(y)) < ε)
0 2 4 6
0.00.20.40.60.8
§
prop mu ← rnorm (10000 , 0 , sqrt (1 e6 ))
pseudo ← rnorm (n∗ 10000 , prop mu, 1)
pseudoMx ← matrix ( pseudo , nrow=10000, ncol=n)
ps ybar ← rowMeans ( pseudoMx )
ps norm ← abs ( ps ybar − ybar )
e p s i l o n ← sort ( ps norm ) [ 2 0 ]
prop keep ← prop mu[ ps norm <= e p s i l o n ]

11. Intro to ABC Example Conclusion
choice of
−15 −10 −5 0 5 10 15 20
0.00.20.40.60.8
(a) = 15.498
0 2 4 6
0.00.20.40.60.8
(b) = 3.47
1.5 2.0 2.5 3.0 3.5 4.0 4.5
0.00.20.40.60.81.0
(c) = 1.65
1.5 2.0 2.5 3.0 3.5 4.0 4.5
0.00.51.01.52.0
(d) = 1.11

12. Intro to ABC Example Conclusion
Improvements to ABC
Alternatives to i.i.d. proposals:
ABC-MCMC
ABC-SMC
Regression adjustment
compensates for larger
Validation of ABC approximation
ABC for model choice

13. Intro to ABC Example Conclusion
Summary
ABC is a method for likelihood-free inference
It enables inference for models that are
otherwise computationally intractable
Main components of ABC:
π(θ) proposal density for θ
f(·|θ) generative model for w
tolerance level
δ(·, ·) distance function
s(y) summary statistics

14. Intro to ABC Example Conclusion
References
Jean-Michel Marin & Christian Robert
Bayesian Essentials with R
Springer-Verlag, 2014.
Jean-Michel Marin, Pierre Pudlo, Christian Robert & Robin Ryder
Approximate Bayesian computational methods.
Statistics & Computing, 22(6): 1167–80, 2012.
Simon Tavar´e, David Balding, Robert Griffiths & Peter Donnelly
Inferring coalescence times from DNA sequence data.
Genetics, 145(2): 505–18, 1997.
Jonathan Pritchard, Mark Seielstad, Anna Perez-Lezaun & Marcus
Feldman
Population Growth of Human Y Chromosomes: A Study of Y
Chromosome Microsatellites.
Mol. Biol. Evol. 16(12): 1791–98, 1999.