a presentation of a paper from M. V. Srinivasan; S. B. Laughlin; A. Dubs called : Predictive Coding: A Fresh View of Inhibition in the Retina. Stable URL: http://links.jstor.org/sici?sici=0080-4649%2819821122%29216%3A1205%3C427%3APCAFVO%3E2.0.CO%3B2-P Proceedings of the Royal Society of London. Series B, Biological Sciences is currently published by The Royal Society.

1. Predictive Coding
A fresh view of the inhibition in the retina
Jérémie Kalfon - Rohith Bhandaru
Flatiron Institute

2. Table of contents
1. Introduction
2. Formulation
3. Comments
4. Experimental Proof
5. Discussion
6. Conclusion
1

3. Introduction

4. Retinal Cell Types
2

5. Motivation
• Lateral and temporal inhibition are found in interneurons
• Natural images have spatial and temporal Correlation.
• The retina itself creates temporal and spatial Correlation.
• Is inhibition trying to achieve Predictive Coding ?
• What is predictive coding ?
3

6. Formulation

7. Intuition
Figure 1: Receptive Field1
1Srinivasan, Mandyam V., Simon B. Laughlin, and Andreas Dubs. ”Predictive coding: a fresh view of inhibition in the retina.” Proceedings
of the Royal Society of London B: Biological Sciences 216.1205 (1982): 427 459.
4

8. Formulation
Let x0, x1, x2, · · · , xn be inputs of n+1 neurons.
ˆx0 =
n∑
j=1
hjxj (1)
where hj are solutions of the following system :
R1,1h1 + R1,2h2 + · · · + R1,nhn = R0,1
R2,1h1 + R2,2h2 + · · · + R2,nhn = R0,2
...
Rn,1h1 + Rn,2h2 + · · · + Rn,nhn = R0,n
with Ri,j being the spatial auto-correlation coefficient of i and j.
5

9. Formulation
Thus, the output of the neuron is :
yi = (xi − ˆxi) (2)
reducing the dynamic range.
This can be extended to temporal inhibition. Considering the
outputs of a neuron along its history.
What factors affect the input signal?
6

10. Comments

11. Pros
• Redundancy removal : Least Squares Estimate is a minimum
variance unbiased estimator that achieves Cramer-Rao lower
bound
7

12. Pros
• Redundancy removal : Least Squares Estimate is a minimum
variance unbiased estimator that achieves Cramer-Rao lower
bound
• Increase in precision level
7

13. Pros
• Redundancy removal : Least Squares Estimate is a minimum
variance unbiased estimator that achieves Cramer-Rao lower
bound
• Increase in precision level
• DC bias removal : The bias is removed locally for each output.
7

14. Pros
• Redundancy removal : Least Squares Estimate is a minimum
variance unbiased estimator that achieves Cramer-Rao lower
bound
• Increase in precision level
• DC bias removal : The bias is removed locally for each output.
• Receptive field widening with decrease in SNR
7

15. Receptive Field vs SNR
(a) SNR = 0.1 (b) SNR = 1 (c) SNR = 10
Table 1: Dependence of receptive field width on SNR
8

16. Algorithm
Algorithm 1 Predictive coding
1: procedure Predict(Image, D) ▷ Return Interneuron Response
2: for all Pixel ∈ Frame do
3: surround ←FindSurroundPixels(dist) ▷ using Euclid. dist.
4: for i, j ∈ Surround do ▷ Compute Correlations
5: Ri,j ← Mean2
signal + Std2
signal × exp−|disti,j|/D
6: Ri,i ← Mean2
signal + Std2
signal + Std2
noise
7: end for
8: R0,: ← H: ⊙ R:0,:
9: OutputPixel ← IPixel −
∑
∀ H ⊙ Surround
10: end for
return Output
11: end procedure
9

17. Experimental Proof

18. Spatial
Table 2: Comparison with receptive field of monopolar cells
10

19. Temporal
Table 3: Comparison with impulse response of monopolar cells
11

20. Discussion

21. Discussion
• Against supposed retinal behavior ?
• Byproduct of Predictive coding ?
• Why coding ?
• What influences Coding ?
12

22. Conclusion

23. Summary
Thanks to the Simons Foundation and our Mentors
github.com/jkobject/predictivecoding
cba
13

24. Plottings
(a) Input
10 20 30 40 50 60
5
10
15
20
25
30
35
40
0
20
40
60
80
100
120
140
160
180
200
(b) SNR = 0.5
10 20 30 40 50 60
5
10
15
20
25
30
35
40
0
20
40
60
80
100
120
(c) SNR = 5
10 20 30 40 50 60
5
10
15
20
25
30
35
40
0
5
10
15
20
25
30
35
40
45
50
(d) SNR = 35
Table 4: Output of an implementation of this predictive coding (patch = 5% )
14

25. Plottings
Figure 2: Spatial Autocorrelation over 3 million samples of a natural image
15

26. Questions?
15

27. Cramer-Rao Bound (CRB)
Let θ be a deterministic parameter and ˆθ be its unbiased estimator.
Then, variance of this estimator is lower bounded as follows:
var(ˆθ) ≥
1
I(θ)
(3)
where I(θ) is the Fischer Information matrix given by
I(θ) = E
[(
∂l(x; θ)
∂θ
)2]
= −E
[
∂2
l(x; θ)
∂2θ
]
(4)
16

28. Least Squares Estimator (LSE)
For a linear regression model
Y = AX + n , n ∈ N(0, σ2
) (5)
the LSE,
ˆX = (A⊺
A)−1
A⊺
Y (6)
is an unbiased estimator with
I(X) =
A⊺
A
σ2
(7)
and achieves CRB.
17