Slides from talk at ATD workshop. October 11, 2018.

1. 1
Joint work with
Paul Liu & Moses Charikar (Stanford)
Sampling methods for
counting temporal motifs
Austin R. Benson · Cornell
ATD Workshop · October 11, 2018
Slides. bit.ly/arb-ATD18
Paper. arXiv:1810.00980

2. Temporal network data is extremely common.
2
Private communication
e-mail, phone calls, text
messages, instant messages
Public communication
Q&A forums, Facebook
walls, Wikipedia edits
Payment systems
credit card transactions,
cryptocurrencies, Venmo
Technical infrastructure
packets over the Internet,
messages over supercomputer

3. Existing methods for temporal analysis are insufficient.
3
1. Models for network growth
Growth of academic collaborations,Internet infrastructure,etc.[Leskovec+ 07]
2. Sequence of snapshot aggregates
Daily phone call graph [Araujo+ 14],weekly email snapshots [Xu-Hero 14]
Modern temporal network datasets
• fine-scale time resolution
• high-frequency
• many repeated edges

4. Motifs,or small subgraph patterns,are commonly used
to analyze static (non-temporal) networks.
4
A B
C
1. Common feature for anomaly detection,
role discovery, and other network
machine learning problems.
[Noble-Cook 03; Sun+ 07; Henderson+ 12; Rohe-Qin 13;
Rossi-Ahmed 15; Benson-Gleich-Leskovec 16]
2. Finding fundamental components of
complex systems. [Milo+ 02]
• Triangles in social networks.
[Rapoport 53; Granovetter 73; Watts-Strogatz 98]
• Bi-directed length-2 paths in brain networks.
[Sporns-Kötter 04; Sporns+ 07; Honey+ 07]

5. Motifs are defined for temporal networks,but we do not
have scalable algorithms for real-time analysis.
5
Temporal network motif.
1. Directed multigraph
with k edges
2. Edge ordering
3. Max. time span δ = 25.
Motif instance.
k temporal edges that
match the pattern that
all occur within δ time.
Definition from [Paranjape-Benson-Leskovec 17]
Problem. We do not have scalable algorithms for counting
these patterns, especially for real-time data analysis.

6. 6
How do we enable real-time motif
analysis for high-throughput
temporal network data?

7. Continuous input of
event data
Update graph
representing relations
between nodes based
on events
Update the embedding
of each node based on
changes to the graph
structure
Perform real-time
statistical analysis of the
node embedding as a
multivariate time series
Output statistically
anomalous nodes or
changes in the graph
If needed, update types
of graph structures used
to embed nodes
ATD: Statistically Principled Real-Time Detection
of Anomalies for Temporal Network Data.
7
With Anil Damle (Cornell)
and Yuekai Sun (Michigan)

8. Parallel sampling yields about two orders of magnitude
speedup and enables otherwise infeasible computations.
8
Using backtracking algorithm from [Mackey+ 18] as a sub-routine.
Time scale δ = 1 day.
16 threads.
running time (seconds)
dataset # temporal edges exact sampling parallel sampling error
StackOverflow 47.9M 221.7 93.10 5.208 4.9%
EquinixChicago 345M 481.2 45.50 5.666 1.3%
RedditComments 636M X 6739 2262 –
<latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit>

9. 9
Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25, s = 5).
Sampling window length w > δ (w = 50).
25 0 25 50 75 100
1. How do we re-scale exact counts?
2. Motifs can cross sampling intervals. How do we mitigate this?
3. How do we choose sampling probabilities q?
sample with prob.q1 sample with prob.q2
We find motifs in sampled windows and re-scale counts.

10. 10
Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25, s = 5).
Sampling window length w > δ (w = 50).
25 0 25 50 75 100
We find motifs in sampled windows and re-scale counts.
Theorem.If we sample window j with prob. qj, then upscaling each
found motif instance by (1 – d(M) / w) / qj is an unbiased estimator,
where d(M) is the duration of the motif instance M.
duration d(M) = 32 – 16 = 16
motif instance M

11. 11
Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25).
Sampling window length w > δ (w = 50).
25 0 25 50 75 100
We find motifs in sampled windows and re-scale counts.
• Using multiple random shifts and averaging the estimates reduces
variance by capturing motifs that cross sampling intervals.
• s = 5, s = 8, s = 15
• Computation over each shift is parallelizable.

12. 12
Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25, s = 5).
Sampling window length w > δ (w = 50).
25 0 25 50 75 100
• Set qj for each shift. Larger qj ⟶ more computation but less variance.
• Importance sampling. Only want to sample where motifs occur.
• Heuristic. Make qj larger if more edges in sampling window.
sample with prob.q1 sample with prob.q2
We find motifs in sampled windows and re-scale counts.

13. 13
Input. Temporal motif and maximum time scale δ .
Output. Estimate of number of instances of the motif.
1. Sample shift s uniformly at random from [0, 1, …, δ - 1].
2. Sample window jth window [(j – 1)w – s, jw –s – 1] with probability qj.
3. Upscale counts of motif instances in jth window by (1 – d(M) / w) / qj.
4. Repeat 1–3 for multiple shifts and output the mean of estimates.
Key advantages.
1. Works in streaming setting. Faster & less memory intensive.
2. Can use (almost) any “exact counting” method for step 3.
[Paranjape-Benson-Leskovec 17; Mackey+ 18; Liu-Benson-Charikar 18]
3. Can parallelize over shifts and sampling windows
⟶ exposes parallelism to otherwise sequential algorithms.

14. Parallel sampling yields about two orders of magnitude
speedup and enables otherwise infeasible computations.
14
Using backtracking algorithm from [Mackey+ 18] as a sub-routine.
Time scale δ = 1 day.
16 threads.
running time (seconds)
dataset # temporal edges exact sampling parallel sampling error
StackOverflow 47.9M 221.7 93.10 5.208 4.9%
EquinixChicago 345M 481.2 45.50 5.666 1.3%
RedditComments 636M X 6739 2262 –
<latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit>

15. 15
THANKS!
Slides. bit.ly/arb-ATD18
Paper. arXiv:1810.00980
Austin R. Benson
http://cs.cornell.edu/~arb
@austinbenson
arb@cs.cornell.edu
Sampling methods for
counting temporal motifs
Where we are headed.
1. Node-level estimates instead
of graph-wide estimates.
2. Theory on how good the
importance sampling works
given some structure.
3. Models for how temporal
networks evolve.
Supported by NSF ATD Award 1830274.

16. Variance results.
16
To get average squared error (✏CM)2
, we need to set the parameters as:
E[kˆYsk2
2] E[kYsk2
2]
C2
M
+
1
c 1
 b✏2
,
where b is the number of shifts. The first term in the left-hand side combines (i)
a natural measure of sparsity of the distribution of motifs with (ii) the extent of
correlation between the sampling probabilities qj and the (weighted) motif counts
for intervals Ys,j.<latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit><latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit><latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit><latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit>
• Let Ys,j be the number of motif instances in the jth window with shift s.
• Let ˆYs,j = Ys,j/
p
qj.
• Then V[Z]  E[kˆYsk2
2] E[kYsk2
2]+ 1
w/ 1 C2
, where C is the true total number
of motif instances.<latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit><latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit><latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit><latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit>

17. Correlation results.
17
Actual motif counts are correlated with sampling probabilities that are chosen
proportional to the fraction of temporal edges in a sampling window.