The dynamics of networks enables the function of a variety of systems we rely on every day, from gene regulation and metabolism in the cell to the distribution of electric power and communication of information. Understanding, steering and predicting the function of interacting nonlinear dynamical systems, in particular if they are externally driven out of equilibrium, relies on obtaining and evaluating suitable models, posing at least two major challenges. First, how can we extract key structural system features of networks if only time series data provide information about the dynamics of (some) units? Second, how can we characterize nonlinear responses of nonlinear multi-dimensional systems externally driven by fluctuations, and consequently, predict tipping points at which normal operational states may be lost? Here we report recent progress on nonlinear response theory extended to predict tipping points and on model-free inference of network structural features from observed dynamics.

1. Nonequilibrium Network Dynamics
Inference, Fluctuation-Responses & Tipping Points
Marc Timme
with M. Thümler, J. Casadiego, H. Haehne et al.
Chair for Network Dynamics
TU Dresden, Institute for Theoretical Physics
Center for Advancing Electronics Dresden
Lakeside Labs, Klagenfurt
Cluster of Excellence Physics of Life

2. … in physics, engineering, …
• Disordered systems & stochastic processes
• Neuromorphic computing & network control
• Collective dynamics of energy systems
• Networked public mobility
• Quantum Synchronization
Nature Energy (2018)
Nature Comm. (2018)
Nature Phys. (2020)
Phys. Rev. Lett. (2020)
Phys. Rev. Lett. (2016)
Phys. Rev. Lett. (2012)
… in biology
• Information routing
• Distributed neural processing
• Protein scaling, gene regulation
• Inverse problems: structure  dynamics?
Nature Phys. (2010)
Nature Phys. (2011)
Phys. Rev. Lett. (2012)
Nature Comm. (2020)
Nature Comm. (2021)
Phys. Rev. Lett. (2016)
Nature Comm. (2017b); Science Adv. (2017);
Phys. Rev. Lett. (2018b, 2019)
Science Adv. (2019)
Nature Comm. (2017a)
J. Neurosci. (2015)
Nature Comm. (2016)
… in socio-economics
• Interactions in social & financial dynamics
• Networked optimization & tech-driven behavior
Phys. Rev. Lett. (2018c)
Phys. Rev. Lett. (2018a)
4D Networks: Driven Distributed Discrete Dynamical Systems

3. Network function
fundamentally underlies
all aspects of our lives
social networks
neural circuits
gene & protein regulation
chemical reaction networks
transport & distribution networks
…
most are externally driven ➔ nonequilibrium collective dynamics
major questions unanswered to date:
A) How to deduce structural features from nonequilibrium dynamics?
B) How to describe & predict nonequilibrium* response properties?
(* nonlinear, nonstationary, distributed …) – in particular tipping points!

4. A) Network Dynamics as an Inverse Problem
network
noneq.
responses
input
observe collective responses → deduce structural features
?

5. What features of a networked system are most essential ?
network features
I) Network size
(# nodes or variables)
Haehne et al., Phys. Rev. Lett. 2019
II) Interaction topology
(who interacts with whom?)
Casadiego et al., Phys. Rev. Lett. 2018
Nature Comm. 2017
Nitzan et al., Science Adv. 2017

6. Infering Network Size from Perceptible Dynamics
input perceive
dynamics
from n units
What is #variables / network size N?
all units
perc. units

7. Idea
exploit that rank(T) at most equals the total # variables N
growing
constant

8. Simplest setting:
linear noiseless relaxation towards stable fixed point
fixed point
linearized dyn.
exact trajectory
observed time series
perceptible components
observe M times series each
for n perceptible units {1,…,n} (units {n+1,…,N} not perceptible)
sampled at k time points

9. Arrange time series data into a detection matrix
observe M times series
for n perceptible units
sampled at k times
collect M trajectories {1,…,m,…,M}
sample k time points
detection matrix
contains all information available

10. Deduce network size from detection matrix
known
(measured data)
unknown unknown (up to n entries)
→idea:
exploit rank inequality to deduce network size N
→ evaluate an increasing number M of trajectories

11. Increasing number M of evaluated trajectories reveals N
→ evaluate an increasing number M of trajectories
linear system linearized phase oscillator system

12. Generalization: anywhere in state space
nonlinear dynamics
near arbitrary point in state space
approximate flow
difference between two trajectories
→ detection matrix of identical form, now

13. Chaotic and periodic systems of 3D units
periodic chaotic

14. B) Predicting nonequilibrium responses
noneq.
responses
input
?

15. Generic average nonequilibrium shift
internal dyn. + driving with zero average
here:
near fixed point
1-component & 1-frequency driving
example:
fluctuating input avg. response is shifted!
e

16. Shift or tipping?
fluctuating input avg. response is shifted!
response diverges!
large e = 6
even larger e = 8
e
i) Origin of the shift?
ii) How does shift depend on driving amplitude e?
iii) At which e do responses begin to diverge?
(tipping point)
- nonlinearity of f(x)
- nonlinearly, at least like e2
- yes, we develop
non-standard perturb. theory

17. i) Nonlinear origin of the shift
asymptotic series for “small” e
substitute into ODE…
… and collect orders of e
➔ zero contribution to shift
➔ lowest-order
nonzero contribution
is nonlinear

18. ii) Predicting shift in 2nd order in e
substitute ➔
➔
shift scales (at least) quadratically
e.g. 2x the driving amplitude → 4x the response shift
 tipping point

19. iii) How to predict tipping point?
key observation:
standard perturbation theory
cannot capture tipping at any order
(polynomials defined for all e)
How to predict it?
idea: non-tipping solutions stay local near fixed point
➔ zero average rate of change
solve for
- exactly or
- by expanding integrand: modified perturbation theory

20. Tipping point prediction – 1D example
solve for ➔
with Bessel function
➔
finite domain
implies critical e

21. Tipping point prediction – power grid model
 shift
so far:
limited prediction for simple system
also for networks
- nonlinear shift
- finite-e tipping

22. Summary Part A
How to find network size from nonequilibrium time series?
Haehne et al., Phys. Rev. Lett. 2019 & Boerner et al., in prep.
network dynamics as an inverse problem (blackbox approach)
suitably arrange all time series data in detection matrix
exploit rank inequality by sequentially considering various M
open challenges:
• robust rank detection?
• how many perceived nodes necessary?
• multi-dim. units?
• combine time series data from different state space regions?

23. Summary Part B
Fluctuation-induced nonlinear shift & tipping
Thuemler et al., MNTS, in press 2022 & in prep. 2022
→ revealed non-trivial average response shift
→ nonlinear origin
→ quantify shift by 2nd order perturbation theory
→ finite-order perturbation theory
incapable of predicting tipping point
→ new framework for predicting tipping point
open challenges:
• understand errors & increase accuracy of tipping pt. prediction
• understand distributedness
of nonlinear responses across a network

24. Thank you to
Questions and comments welcome!
my colleagues, collaborators &
Network Dynamics team
you all for your attention & interest
http://networkdynamics.info

25. Arrange time series data into a detection matrix
observe M times series
for n perceptible units
sampled at k times
mth measured trajectory, {1,…,m,…,M}
sample at k time points
detection matrix
contains all information available

26. Technically: detect gap in singular value spectrum
linear system linearized phase oscillator system
linear system nonlinear system

27. Summary Part II
Nonlinear Dynamics ➔ Interaction Topology
• introduced explicit dependency matrices
(explicit, uniquely multiplies system state, relates to incidence matrix)
• exploit concepts of dynamics space to obtain linear restricting equations
• ARNI (algorithm for revealing network interactions)
for block-sparse and block-dense solutions
• event spaces yield networks from event times
(not shown)
• from statistics of time series only
Casadiego et al., Nature Comm. (2017)
Casadiego et al., Phys. Rev. Lett. (2018)
Nitzan et al., Science Adv. (2017)

28. Noisy, heterogeneous, sparsely coupled networks

29. How to obtain a model for given system?
• take from literature (= from someone else)
• do it yourself (use the data!)
Topical Review: Timme & Casadiego J. Phys. A (2014)

30. A) Standard: System’s Dynamics as a Forward Problem
system
model
predict
dynamics
input
model of system constituents → predicts dynamics
fixed points, stability,
response functions,
bifurcations, …

31. Systems Dynamics as an Inverse Problem
system
model
observe
dynamics
input
observe system dynamics → deduce system constituents

32. Part II Model-free inference of network interactions
nonlinear network dynamical system
derive linear system
of constraint equations
→ group sparse inference problem
develop suitable group sparse algorithm

33. Model-free inference of network interactions
 3-point interaction
incidence matrix
define explicit dependency matrix such that
1 if and only if RHSi does explicitly depend on xj
0 otherwise
for each i, is a diagonal matrix with

34. Explicit Dependency Matrix (cont’d)
➔
useful features for systems of differential equations:
• multiplies system state on RHS of DE
• explicitely appears exactly once
• related to incidence matrix in graph theory
Casadiego et al. Nature Comm. (2017)

35. exploit: derivative zero iff independent of coordinate
Dynamics space = state space x space of 1st derivatives
zi = (x, dxi/dt=Fi(x))

36. Action of Explicit Dependency Matrix in order expansions
naturally multiplies each dependency in each order
→ appears as linear coefficients in restricting equations

37. Basis Functions & Block Sparse/Dense Solutions

38. ARNI
Algorithm for Revealing Network Interactions

39. Reconstruction Performance
N=30, k=15

40. Reconstruction Performance
only type of function, not function itself is relevant for successful reconstruction
Casadiego et al., Nature Comm. (2017)

41. Circadian clock in Drosophila
• N=10 units
• nonlinear coupling

42. Performance against benchmarks

43. Glycolytic oscillator in yeast
• N=7 units
• strongly nonlinear coupling
• true 3-point interactions
together w/ 2-point interactions

44. Performance against benchmarks

45. Reconstruction from noisy time series
of a subset of units
deterministic transient trajectory
noisy transient trajectory
• N=100
• fraction R of observed units
• noise-driven dynamics
➔ still reasonable performance

46. From complex to more complex settings
so far:
- uniform request rates
- infinitely large busses
now:
- arbitrary request rate distribution
- finite-capacity of busses