SlideShare une entreprise Scribd logo
1  sur  63
Télécharger pour lire hors ligne
Progress and challenges for the use of deep learning to
improve weather forecasts
Peter Düben
European Centre for Medium-Range Weather Forecasts (ECMWF)
Royal Society University Research Fellow
European Centre for Medium-Range Weather Forecasts (ECMWF)
www.ecmwf.int
Research institute and 24/7 operational weather service for medium-range, monthly
and seasonal forecasts.
Independent, intergovernmental organisation supported by 34 states.
Based close to London in the UK; ≈ 350 member of staff.
Home of two supercomputers.
Home of the Integrated Forecast System (IFS).
Peter Düben Page 2
Predicting weather and climate: Why is it so hard?
Earth as seen from Apollo 17
Peter Düben Page 3
Predicting weather and climate: Why is it so hard?
Earth as seen from Apollo 17
The Earth System is complex, huge and chaotic and we do not have sufficient resolution
to resolve all important processes.
Peter Düben Page 3
Predicting weather and climate: Why is it so hard?
Earth as seen from Apollo 17
The Earth System is complex, huge and chaotic and we do not have sufficient resolution
to resolve all important processes.
Use cases for machine learning are all over the place...
Peter Düben Page 3
Deep learning for weather and climate
The Earth System has many components that show non-linear dynamics, we have plenty
of observations and often need to apply rough approximations to formulate our models.
Peter Düben Page 4
Deep learning for weather and climate
The Earth System has many components that show non-linear dynamics, we have plenty
of observations and often need to apply rough approximations to formulate our models.
www.wikipedia.org
Neural Networks learn from input/output pairs.
Neurons have weighted connections to each other and the weights are trained to
produce the optimal results.
Neural networks can emulate non-linear systems.
Peter Düben Page 4
Future perspective for machine learning in weather forecasts
Peter Düben Page 5
Future perspective for machine learning in weather forecasts
The next slides: I will show example applications and discuss challenges.
Peter Düben Page 5
To learn the equations
Neural networks enable us to learn complex non-linear dynamics as a black box.
Peter Düben Page 6
To learn the equations
Neural networks enable us to learn complex non-linear dynamics as a black box.
We could base the entire model on Neural Networks.
Who needs Navier Stokes?
We know the equations of motion but we cannot solve them.
Discretisation and sub-grid-scale variability generates significant errors.
The data handling system of ECMWF provides access to over 210 petabyte of
primary data and the data archive of ECMWF grows by about 233 terabyte per day.
Peter Düben Page 6
Global weather forecast based on Neural Networks
Retrieve hourly data of geopotential height at 500 hPa from ERA5 re-analysis for
training (> 65000 global data sets).
Map the data to a coarse lon/lat grid (60x31).
Use the state of the model at timestep i as input and the state of the model at
timestep i + 1 as output.
Use a 9 × 9 stencil around the grid point that should be predicted.
Add time of day and year as well as the coordination of a gridpoint (lon+lat) as input
variables to the network.
The Pole needs special treatment.
Peter Düben Page 7
Global weather forecast based on Neural Networks
Peter Düben Page 8
Global weather forecast based on Neural Networks
The Neural Network model can compete with a dynamical model of similar complexity.
Dueben and Bauer GMD 2018
Peter Düben Page 8
Global weather forecast based on Neural Networks
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
Peter Düben Page 9
Global weather forecast based on Neural Networks
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
The simulations show reasonable dynamics.
Peter Düben Page 9
Global weather forecast based on Neural Networks
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
The simulations show reasonable dynamics.
Just adding further inputs does not necessarily help.
Peter Düben Page 9
Global weather forecast based on Neural Networks
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
470 480 495 510 525 540 555 570 585 600
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Analysis
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
24 hours; Local Neural Network
-4000 -3000 -2000 -1000 0 1000 2000 3000 4000
The simulations show reasonable dynamics.
Just adding further inputs does not necessarily help.
Model runs crash after a couple of weeks.
Dueben and Bauer GMD 2018
Peter Düben Page 9
Let’s use neural networks to emulate existing parametrisation
schemes
Store input/output pairs of parametrisation schemes.
Use this data to train a neural network to do the same job.
Replace the parametrisation scheme by the neural network.
Peter Düben Page 10
Let’s use neural networks to emulate existing parametrisation
schemes
Store input/output pairs of parametrisation schemes.
Use this data to train a neural network to do the same job.
Replace the parametrisation scheme by the neural network.
Why would you do this?
A large fraction of the computational cost is generated by parametrisation scheme.
Parametrisation schemes cause > 90% of model code.
Optimization of this code is very difficult
(→ less than 5% peak performance).
Neural Networks are highly optimized and can even use co-designed hardware.
→ Portability comes for free.
Peter Düben Page 10
Let’s use neural networks to emulate existing parametrisation
schemes
Store input/output pairs of parametrisation schemes.
Use this data to train a neural network to do the same job.
Replace the parametrisation scheme by the neural network.
Why would you do this?
A large fraction of the computational cost is generated by parametrisation scheme.
Parametrisation schemes cause > 90% of model code.
Optimization of this code is very difficult
(→ less than 5% peak performance).
Neural Networks are highly optimized and can even use co-designed hardware.
→ Portability comes for free.
We hope that deep Neural Networks will be almost as good as the original
parametrisation schemes but much more efficient.
Peter Düben Page 10
A neural network emulator for the state-of-the-art model
configuration with 137 vertical levels
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards
0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 10140928
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards
0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 1.01303e+07
Dueben, Hogan, Bauer @ECMWF and Progsch, Angerer @NVIDIA
Downward solar radiation at the surface for the original radiation scheme and the Neural
Network emulator.
Peter Düben Page 11
A neural network emulator for the state-of-the-art model
configuration with 137 vertical levels
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards
0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 10140928
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°N
10°S
20°S
30°S
40°S
50°S
60°S
70°S
80°S
10°N
20°N
30°N
40°N
50°N
60°N
70°N
80°N
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E
Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards
0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 1.01303e+07
Dueben, Hogan, Bauer @ECMWF and Progsch, Angerer @NVIDIA
Downward solar radiation at the surface for the original radiation scheme and the Neural
Network emulator.
Free-running model simulations are stable and we get a factor of O(10) speed-up for
radiation.
Peter Düben Page 11
A neural network emulator for gravity wave drag
Original scheme Difference Neural Network
Chantry, Dueben, Palmer.
Tendency output for the non-orographic gravity wave drag parametrisation scheme for the
standard scheme and a neural network emulator.
Peter Düben Page 12
And one more thing...
Neumann, Dueben, et al., Phil. Trans. A, 2019
Weather and climate models are high performance
computing applications:
More resolution → more processes resolved.
Ratio sustained/peak is going down.
Machine learning has a very strong impact on hardware
developments at the moment.
Peter Düben Page 13
And one more thing...
Neumann, Dueben, et al., Phil. Trans. A, 2019
Weather and climate models are high performance
computing applications:
More resolution → more processes resolved.
Ratio sustained/peak is going down.
Machine learning has a very strong impact on hardware
developments at the moment.
Can we use deep learning hardware for conventional
models?
Peter Düben Page 13
Use machine learning hardware for the Legendre transform
Relative cost for model components for a non-hydrostatic model at 1.45 km resolution:
Peter Düben Page 14
Use machine learning hardware for the Legendre transform
Relative cost for model components for a non-hydrostatic model at 1.45 km resolution:
The Legendre transform is the most expensive kernel. It consists of a large number
of standard matrix-matrix multiplications.
If we can re-scale the input and output fields, we can use half precision arithmetic.
Tensor Cores on NVIDIA Volta GPUs are optimised for half-precision matrix-matrix
calculations with single precision output. 7.8 TFlops for double precision vs. 125
TFlops for half precision on the Tensor Core.
Peter Düben Page 14
Half precision Legendre Transformations
0 2 4 6 8 10
Forecast lead time (days)
0.0
1.0
2.0
3.0
4.0
500hPatemperatureRMSE
Northern extratropics (20N-90N)
double
half trans 25
tensor core
0 2 4 6 8 10
Forecast lead time (days)
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Tropics (20S-20N)
0 2 4 6 8 10
Forecast lead time (days)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Southern extratropics (90S-20S)
Root-mean-square error for Z500 at 9 km resolution averaged over multiple start dates.
Hatfield, Chantry, Dueben, Palmer Best Paper Award PASC2019
Peter Düben Page 15
Half precision Legendre Transformations
0 2 4 6 8 10
Forecast lead time (days)
0.0
1.0
2.0
3.0
4.0
500hPatemperatureRMSE
Northern extratropics (20N-90N)
double
half trans 25
tensor core
0 2 4 6 8 10
Forecast lead time (days)
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Tropics (20S-20N)
0 2 4 6 8 10
Forecast lead time (days)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Southern extratropics (90S-20S)
Root-mean-square error for Z500 at 9 km resolution averaged over multiple start dates.
Hatfield, Chantry, Dueben, Palmer Best Paper Award PASC2019
The simulations are using an emulator to reduce precision.
Dawson and Dueben GMD 2017
Peter Düben Page 15
The challenge for the machine-learning community
Peter Düben Page 16
The challenge for the machine-learning community
The time of nice pictures and videos is over...
Peter Düben Page 16
The challenge for the machine-learning community
The time of nice pictures and videos is over...
We need to prove that machine learning tools are better than state-of-the-art within the
next two years.
Peter Düben Page 16
The challenge for the machine-learning community
The time of nice pictures and videos is over...
We need to prove that machine learning tools are better than state-of-the-art within the
next two years.
We need to build useful tools that improve weather and climate predictions and/or help to
improve our understanding of the Earth System and are able to convince the conservative
weather and climate scientist.
Peter Düben Page 16
Why is this challenge so hard?
0°N 0°N
120°E
120°E
0°N 0°N
120°E
120°E
Dueben, Wedi, Saarinen submitted
Top-of-the-atmosphere cloud brightness temperature [K] for satellite observations and a
simulation of the atmosphere with 1.45 km resolution.
A weather forecast simulation has O(1,000,000,000) degrees-of-freedom.
Peter Düben Page 17
Why is this challenge so hard?
0°N 0°N
120°E
120°E
0°N 0°N
120°E
120°E
Dueben, Wedi, Saarinen submitted
Top-of-the-atmosphere cloud brightness temperature [K] for satellite observations and a
simulation of the atmosphere with 1.45 km resolution.
A weather forecast simulation has O(1,000,000,000) degrees-of-freedom.
Global simulations show a breath-taking level of complexity and can represent many
details of the Earth System.
Peter Düben Page 17
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
How can we guarantee reproducibility during training?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
How can we guarantee reproducibility during training?
How can we get beyond “dense” networks but still take local properties into account?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
How can we guarantee reproducibility during training?
How can we get beyond “dense” networks but still take local properties into account?
How can we fix interactions between model components?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
How can we guarantee reproducibility during training?
How can we get beyond “dense” networks but still take local properties into account?
How can we fix interactions between model components?
How can we interface neural networks with legacy codes and how can we use AI and
conventional methods efficiently on the same hardware?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
How can we guarantee reproducibility during training?
How can we get beyond “dense” networks but still take local properties into account?
How can we fix interactions between model components?
How can we interface neural networks with legacy codes and how can we use AI and
conventional methods efficiently on the same hardware?
How can we design good training data (short time steps and high resolution)?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
How can we guarantee reproducibility during training?
How can we get beyond “dense” networks but still take local properties into account?
How can we fix interactions between model components?
How can we interface neural networks with legacy codes and how can we use AI and
conventional methods efficiently on the same hardware?
How can we design good training data (short time steps and high resolution)?
How can we explore the full phase space (all weather regimes) during training?
Peter Düben Page 18
Questions to be answered...
There is no fundamental reasons not to use a black box within weather and climate
models. However,...
How can we use our knowledge about the Earth System?
How can we diagnose physical knowledge from the network?
How can we remove a bias from neural networks and secure conservation laws?
How can we pick hyper-parameters?
How can we guarantee reproducibility during training?
How can we get beyond “dense” networks but still take local properties into account?
How can we fix interactions between model components?
How can we interface neural networks with legacy codes and how can we use AI and
conventional methods efficiently on the same hardware?
How can we design good training data (short time steps and high resolution)?
How can we explore the full phase space (all weather regimes) during training?
How can we generate large labelled datasets?
Peter Düben Page 18
My personal vision of the way forward...
Weather and climate
models
Deep learning
Peter Düben Page 19
My personal vision of the way forward...
Weather and climate
models
Deep learning
To study known differential equations to learn how
to derive blueprints for neural network architectures.
Peter Düben Page 19
My personal vision of the way forward...
Weather and climate
models
Deep learning
To study known differential equations to learn how
to derive blueprints for neural network architectures.
To learn how to scale neural networks to
O(1,000,000) inputs for 3D fields on the sphere.
Peter Düben Page 19
My personal vision of the way forward...
Weather and climate
models
Deep learning
To study known differential equations to learn how
to derive blueprints for neural network architectures.
To learn how to scale neural networks to
O(1,000,000) inputs for 3D fields on the sphere.
To study the representation of variability and the
correction of systematic errors for neural networks.
Peter Düben Page 19
My personal vision of the way forward...
Weather and climate
models
Deep learning
To study known differential equations to learn how
to derive blueprints for neural network architectures.
To learn how to scale neural networks to
O(1,000,000) inputs for 3D fields on the sphere.
To study the representation of variability and the
correction of systematic errors for neural networks.
To build community tools in the form of benchmark
problems.
Peter Düben Page 19
My personal vision of the way forward...
Weather and climate
models
Deep learning
To study known differential equations to learn how
to derive blueprints for neural network architectures.
To learn how to scale neural networks to
O(1,000,000) inputs for 3D fields on the sphere.
To study the representation of variability and the
correction of systematic errors for neural networks.
To build community tools in the form of benchmark
problems.
To focus on useful tools that can serve as beacon.
Peter Düben Page 19
My personal vision of the way forward...
Weather and climate
models
Deep learning
To study known differential equations to learn how
to derive blueprints for neural network architectures.
To learn how to scale neural networks to
O(1,000,000) inputs for 3D fields on the sphere.
To study the representation of variability and the
correction of systematic errors for neural networks.
To build community tools in the form of benchmark
problems.
To focus on useful tools that can serve as beacon.
This will be hard work.
Peter Düben Page 19
My personal vision of the way forward...
Weather and climate
models
Deep learning
To study known differential equations to learn how
to derive blueprints for neural network architectures.
To learn how to scale neural networks to
O(1,000,000) inputs for 3D fields on the sphere.
To study the representation of variability and the
correction of systematic errors for neural networks.
To build community tools in the form of benchmark
problems.
To focus on useful tools that can serve as beacon.
This will be hard work.
We will require machine learning solutions that are
customised to weather and climate models.
Peter Düben Page 19
An example: The Burgers equation
Let’s represent a non-linear system that is approximated by the Burgers’ equation:
∂u
∂t
= ν
∂2
u
∂x2
− u
∂u
∂x
+ p.
Peter Düben Page 20
An example: The Burgers equation
Let’s represent a non-linear system that is approximated by the Burgers’ equation:
∂u
∂t
= ν
∂2
u
∂x2
− u
∂u
∂x
+ p.
The conventional approach:
∂ui
∂t
= ν
ui+1 − 2ui + ui−1
∆x2
− ui
ui+1 − ui−1
2∆x
+ c0 + c1 · ui + c2 · ui + c3 · ui · ζ.
Peter Düben Page 20
An example: The Burgers equation
Let’s represent a non-linear system that is approximated by the Burgers’ equation:
∂u
∂t
= ν
∂2
u
∂x2
− u
∂u
∂x
+ p.
The conventional approach:
∂ui
∂t
= ν
ui+1 − 2ui + ui−1
∆x2
− ui
ui+1 − ui−1
2∆x
+ c0 + c1 · ui + c2 · ui + c3 · ui · ζ.
The data-science approach:
Standard Neuron InputOutput Standard connection
Differential quotient 1 Differential quotient 2 Product pooling
Peter Düben Page 20
An example: The Burgers equation
Let’s represent a non-linear system that is approximated by the Burgers’ equation:
∂u
∂t
= ν
∂2
u
∂x2
− u
∂u
∂x
+ p.
The conventional approach:
∂ui
∂t
= ν
ui+1 − 2ui + ui−1
∆x2
− ui
ui+1 − ui−1
2∆x
+ c0 + c1 · ui + c2 · ui + c3 · ui · ζ.
The data-science approach: The way forward:
Standard Neuron InputOutput Standard connection
Differential quotient 1 Differential quotient 2 Product pooling
Peter Düben Page 20
Conclusions
Weather and climate models offer a large number of promising applications for
machine learning that span the entire workflow of numerical weather predictions.
We can learn the equations of motion using deep learning and build
medium-complexity models.
We need to develop useful tools based on machine learning within the next two years.
We need to learn how to emulate known differential equations, represent
sub-grid-scale variability and systematic errors, and learn how to scale machine
learning solutions to O(1,000,000) input fields.
We need benchmark datasets and beacons.
Machine learning hardware can be used to speed-up conventional model simulations.
Peter Düben Page 21
Conclusions
Weather and climate models offer a large number of promising applications for
machine learning that span the entire workflow of numerical weather predictions.
We can learn the equations of motion using deep learning and build
medium-complexity models.
We need to develop useful tools based on machine learning within the next two years.
We need to learn how to emulate known differential equations, represent
sub-grid-scale variability and systematic errors, and learn how to scale machine
learning solutions to O(1,000,000) input fields.
We need benchmark datasets and beacons.
Machine learning hardware can be used to speed-up conventional model simulations.
Many thanks for your attention.
Peter Düben Page 21
Funding
The presenter gratefully acknowledges the funding a Royal Society University Research
Fellowship and from the ESIWACE and ESIWACE2 project. The ESIWACE2 projects have
received funding from the European Union’s Horizon 2020 research and innovation
programme under grant agreement No 823988.
Peter Düben Page 22

Contenu connexe

Tendances

Implementation of Data Mining Techniques for Meteorological Data Analysis
Implementation of Data Mining Techniques for Meteorological Data Analysis Implementation of Data Mining Techniques for Meteorological Data Analysis
Implementation of Data Mining Techniques for Meteorological Data Analysis Arofiah Hidayati
 
Tools for weather forecasting
Tools for weather forecastingTools for weather forecasting
Tools for weather forecastingammulachu
 
Weather forecasting
Weather forecastingWeather forecasting
Weather forecastingESSBY
 
Weather forecasting
Weather forecastingWeather forecasting
Weather forecastingSwaster Xwax
 
Weather forecasting in india
Weather forecasting in india Weather forecasting in india
Weather forecasting in india Akarsh3083
 
Numerical Weather Prediction
Numerical Weather PredictionNumerical Weather Prediction
Numerical Weather PredictionSourav Kumar Mund
 
Weather Prediction: History and Math
Weather Prediction: History and MathWeather Prediction: History and Math
Weather Prediction: History and MathIda Regine
 
project slide_04042016
project slide_04042016project slide_04042016
project slide_04042016NIDHEESH P.V
 
Gathering Weather Data CLASS NOTES
Gathering Weather Data CLASS NOTESGathering Weather Data CLASS NOTES
Gathering Weather Data CLASS NOTESMaria Donohue
 
Meterological Instruments
Meterological InstrumentsMeterological Instruments
Meterological InstrumentsUsman Khawaja
 
24 6 predicting weather show with sec 5
24 6 predicting weather show with sec 524 6 predicting weather show with sec 5
24 6 predicting weather show with sec 5guestabbdaf
 
Meteorological instruments
Meteorological instrumentsMeteorological instruments
Meteorological instrumentsMariadebiologia
 
Climate observations stations and networks
Climate observations   stations and networksClimate observations   stations and networks
Climate observations stations and networksProf. A.Balasubramanian
 
What instruments do we use to measure weather
What instruments do we use to measure weatherWhat instruments do we use to measure weather
What instruments do we use to measure weatherdriestenberg
 

Tendances (20)

Implementation of Data Mining Techniques for Meteorological Data Analysis
Implementation of Data Mining Techniques for Meteorological Data Analysis Implementation of Data Mining Techniques for Meteorological Data Analysis
Implementation of Data Mining Techniques for Meteorological Data Analysis
 
Tools for weather forecasting
Tools for weather forecastingTools for weather forecasting
Tools for weather forecasting
 
weather forecasting
weather forecasting weather forecasting
weather forecasting
 
Weather Forecasting using Deep Learning A lgorithm for the Ethiopian Context
Weather Forecasting using Deep Learning A lgorithm for the Ethiopian ContextWeather Forecasting using Deep Learning A lgorithm for the Ethiopian Context
Weather Forecasting using Deep Learning A lgorithm for the Ethiopian Context
 
WEATHER FORECASTING
WEATHER FORECASTINGWEATHER FORECASTING
WEATHER FORECASTING
 
Weather forecasting
Weather forecastingWeather forecasting
Weather forecasting
 
Weather forecasting
Weather forecastingWeather forecasting
Weather forecasting
 
Weather forecasting
Weather forecastingWeather forecasting
Weather forecasting
 
Weather forecasting in india
Weather forecasting in india Weather forecasting in india
Weather forecasting in india
 
Numerical Weather Prediction
Numerical Weather PredictionNumerical Weather Prediction
Numerical Weather Prediction
 
Weather Prediction: History and Math
Weather Prediction: History and MathWeather Prediction: History and Math
Weather Prediction: History and Math
 
project slide_04042016
project slide_04042016project slide_04042016
project slide_04042016
 
Gathering Weather Data CLASS NOTES
Gathering Weather Data CLASS NOTESGathering Weather Data CLASS NOTES
Gathering Weather Data CLASS NOTES
 
Meterological Instruments
Meterological InstrumentsMeterological Instruments
Meterological Instruments
 
24 6 predicting weather show with sec 5
24 6 predicting weather show with sec 524 6 predicting weather show with sec 5
24 6 predicting weather show with sec 5
 
Meteorological instruments
Meteorological instrumentsMeteorological instruments
Meteorological instruments
 
Presentation rhods
Presentation rhodsPresentation rhods
Presentation rhods
 
Weather Maps
Weather MapsWeather Maps
Weather Maps
 
Climate observations stations and networks
Climate observations   stations and networksClimate observations   stations and networks
Climate observations stations and networks
 
What instruments do we use to measure weather
What instruments do we use to measure weatherWhat instruments do we use to measure weather
What instruments do we use to measure weather
 

Similaire à Progress and Challenges for the Use of Deep Learning to Improve Weather Forecasts

Machine Learning for Weather Forecasts
Machine Learning for Weather ForecastsMachine Learning for Weather Forecasts
Machine Learning for Weather Forecastsinside-BigData.com
 
WindSight Validation (March 2011)
WindSight Validation (March 2011)WindSight Validation (March 2011)
WindSight Validation (March 2011)Carlos Pinto
 
Earthquake forecasting based on ionosphere statistical monitoring
Earthquake forecasting based on ionosphere statistical monitoringEarthquake forecasting based on ionosphere statistical monitoring
Earthquake forecasting based on ionosphere statistical monitoringAstroAwake
 
IRJET- Arduino Based Weather Monitoring System
IRJET- Arduino Based Weather Monitoring SystemIRJET- Arduino Based Weather Monitoring System
IRJET- Arduino Based Weather Monitoring SystemIRJET Journal
 
Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...
Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...
Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...Decision and Policy Analysis Program
 
Francisco J. Doblas-Big Data y cambio climático
Francisco J. Doblas-Big Data y cambio climáticoFrancisco J. Doblas-Big Data y cambio climático
Francisco J. Doblas-Big Data y cambio climáticoFundación Ramón Areces
 
Sun Tracker
Sun TrackerSun Tracker
Sun TrackerAjnas KC
 
National Weather Service Storm Spotter Training
National Weather Service Storm Spotter TrainingNational Weather Service Storm Spotter Training
National Weather Service Storm Spotter Trainingchowd
 
How to Prepare Weather and Climate Models for Future HPC Hardware
How to Prepare Weather and Climate Models for Future HPC HardwareHow to Prepare Weather and Climate Models for Future HPC Hardware
How to Prepare Weather and Climate Models for Future HPC Hardwareinside-BigData.com
 
CFD down-scaling and online measurements for short-term wind power forecasting
CFD down-scaling and online measurements for short-term wind power forecastingCFD down-scaling and online measurements for short-term wind power forecasting
CFD down-scaling and online measurements for short-term wind power forecastingJean-Claude Meteodyn
 
GSS Earthquake Alert System
GSS Earthquake Alert SystemGSS Earthquake Alert System
GSS Earthquake Alert SystemMatthew Straeb
 
New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...
New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...
New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...Peter Laurinec
 
Reflecting on the Goal and Baseline of Exascale Computing
Reflecting on the Goal and Baseline of Exascale ComputingReflecting on the Goal and Baseline of Exascale Computing
Reflecting on the Goal and Baseline of Exascale Computinginside-BigData.com
 
BigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis Mallas
BigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis MallasBigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis Mallas
BigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis MallasBigData_Europe
 

Similaire à Progress and Challenges for the Use of Deep Learning to Improve Weather Forecasts (20)

Machine Learning for Weather Forecasts
Machine Learning for Weather ForecastsMachine Learning for Weather Forecasts
Machine Learning for Weather Forecasts
 
EENA 2018 - Weather-related emergencies
EENA 2018 - Weather-related emergencies EENA 2018 - Weather-related emergencies
EENA 2018 - Weather-related emergencies
 
WindSight Validation (March 2011)
WindSight Validation (March 2011)WindSight Validation (March 2011)
WindSight Validation (March 2011)
 
2202.11214.pdf
2202.11214.pdf2202.11214.pdf
2202.11214.pdf
 
Earthquake forecasting based on ionosphere statistical monitoring
Earthquake forecasting based on ionosphere statistical monitoringEarthquake forecasting based on ionosphere statistical monitoring
Earthquake forecasting based on ionosphere statistical monitoring
 
IRJET- Arduino Based Weather Monitoring System
IRJET- Arduino Based Weather Monitoring SystemIRJET- Arduino Based Weather Monitoring System
IRJET- Arduino Based Weather Monitoring System
 
13-Fcst-NWP.ppt
13-Fcst-NWP.ppt13-Fcst-NWP.ppt
13-Fcst-NWP.ppt
 
Clear sky and all-weather global and beam irradiance models: long term valida...
Clear sky and all-weather global and beam irradiance models: long term valida...Clear sky and all-weather global and beam irradiance models: long term valida...
Clear sky and all-weather global and beam irradiance models: long term valida...
 
Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...
Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...
Carlos N - CIAT Experience In Climate Modeling; Scenarios of future climate c...
 
Francisco J. Doblas-Big Data y cambio climático
Francisco J. Doblas-Big Data y cambio climáticoFrancisco J. Doblas-Big Data y cambio climático
Francisco J. Doblas-Big Data y cambio climático
 
Sun Tracker
Sun TrackerSun Tracker
Sun Tracker
 
National Weather Service Storm Spotter Training
National Weather Service Storm Spotter TrainingNational Weather Service Storm Spotter Training
National Weather Service Storm Spotter Training
 
Lighteye
LighteyeLighteye
Lighteye
 
How to Prepare Weather and Climate Models for Future HPC Hardware
How to Prepare Weather and Climate Models for Future HPC HardwareHow to Prepare Weather and Climate Models for Future HPC Hardware
How to Prepare Weather and Climate Models for Future HPC Hardware
 
SRDG April2011
SRDG April2011SRDG April2011
SRDG April2011
 
CFD down-scaling and online measurements for short-term wind power forecasting
CFD down-scaling and online measurements for short-term wind power forecastingCFD down-scaling and online measurements for short-term wind power forecasting
CFD down-scaling and online measurements for short-term wind power forecasting
 
GSS Earthquake Alert System
GSS Earthquake Alert SystemGSS Earthquake Alert System
GSS Earthquake Alert System
 
New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...
New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...
New Clustering-based Forecasting Method for Disaggregated End-consumer Electr...
 
Reflecting on the Goal and Baseline of Exascale Computing
Reflecting on the Goal and Baseline of Exascale ComputingReflecting on the Goal and Baseline of Exascale Computing
Reflecting on the Goal and Baseline of Exascale Computing
 
BigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis Mallas
BigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis MallasBigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis Mallas
BigDataEurope 1st SC5 Workshop Big Data in ECMWF by Ioannis Mallas
 

Plus de inside-BigData.com

Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...inside-BigData.com
 
Transforming Private 5G Networks
Transforming Private 5G NetworksTransforming Private 5G Networks
Transforming Private 5G Networksinside-BigData.com
 
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...inside-BigData.com
 
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...inside-BigData.com
 
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...inside-BigData.com
 
HPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural NetworksHPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural Networksinside-BigData.com
 
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean MonitoringBiohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoringinside-BigData.com
 
HPC AI Advisory Council Update
HPC AI Advisory Council UpdateHPC AI Advisory Council Update
HPC AI Advisory Council Updateinside-BigData.com
 
Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19inside-BigData.com
 
Energy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic TuningEnergy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic Tuninginside-BigData.com
 
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPODHPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPODinside-BigData.com
 
Versal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud AccelerationVersal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud Accelerationinside-BigData.com
 
Zettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance EfficientlyZettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance Efficientlyinside-BigData.com
 
Scaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's EraScaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's Erainside-BigData.com
 
CUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computingCUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computinginside-BigData.com
 
Introducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi ClusterIntroducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi Clusterinside-BigData.com
 
Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...
Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...
Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...inside-BigData.com
 

Plus de inside-BigData.com (20)

Major Market Shifts in IT
Major Market Shifts in ITMajor Market Shifts in IT
Major Market Shifts in IT
 
Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...Preparing to program Aurora at Exascale - Early experiences and future direct...
Preparing to program Aurora at Exascale - Early experiences and future direct...
 
Transforming Private 5G Networks
Transforming Private 5G NetworksTransforming Private 5G Networks
Transforming Private 5G Networks
 
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...
 
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...
 
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
 
HPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural NetworksHPC Impact: EDA Telemetry Neural Networks
HPC Impact: EDA Telemetry Neural Networks
 
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean MonitoringBiohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
 
HPC AI Advisory Council Update
HPC AI Advisory Council UpdateHPC AI Advisory Council Update
HPC AI Advisory Council Update
 
Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19Fugaku Supercomputer joins fight against COVID-19
Fugaku Supercomputer joins fight against COVID-19
 
Energy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic TuningEnergy Efficient Computing using Dynamic Tuning
Energy Efficient Computing using Dynamic Tuning
 
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPODHPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD
 
State of ARM-based HPC
State of ARM-based HPCState of ARM-based HPC
State of ARM-based HPC
 
Versal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud AccelerationVersal Premium ACAP for Network and Cloud Acceleration
Versal Premium ACAP for Network and Cloud Acceleration
 
Zettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance EfficientlyZettar: Moving Massive Amounts of Data across Any Distance Efficiently
Zettar: Moving Massive Amounts of Data across Any Distance Efficiently
 
Scaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's EraScaling TCO in a Post Moore's Era
Scaling TCO in a Post Moore's Era
 
CUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computingCUDA-Python and RAPIDS for blazing fast scientific computing
CUDA-Python and RAPIDS for blazing fast scientific computing
 
Introducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi ClusterIntroducing HPC with a Raspberry Pi Cluster
Introducing HPC with a Raspberry Pi Cluster
 
Overview of HPC Interconnects
Overview of HPC InterconnectsOverview of HPC Interconnects
Overview of HPC Interconnects
 
Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...
Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...
Efficient Model Selection for Deep Neural Networks on Massively Parallel Proc...
 

Dernier

COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 

Dernier (20)

20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 

Progress and Challenges for the Use of Deep Learning to Improve Weather Forecasts

  • 1. Progress and challenges for the use of deep learning to improve weather forecasts Peter Düben European Centre for Medium-Range Weather Forecasts (ECMWF) Royal Society University Research Fellow
  • 2. European Centre for Medium-Range Weather Forecasts (ECMWF) www.ecmwf.int Research institute and 24/7 operational weather service for medium-range, monthly and seasonal forecasts. Independent, intergovernmental organisation supported by 34 states. Based close to London in the UK; ≈ 350 member of staff. Home of two supercomputers. Home of the Integrated Forecast System (IFS). Peter Düben Page 2
  • 3. Predicting weather and climate: Why is it so hard? Earth as seen from Apollo 17 Peter Düben Page 3
  • 4. Predicting weather and climate: Why is it so hard? Earth as seen from Apollo 17 The Earth System is complex, huge and chaotic and we do not have sufficient resolution to resolve all important processes. Peter Düben Page 3
  • 5. Predicting weather and climate: Why is it so hard? Earth as seen from Apollo 17 The Earth System is complex, huge and chaotic and we do not have sufficient resolution to resolve all important processes. Use cases for machine learning are all over the place... Peter Düben Page 3
  • 6. Deep learning for weather and climate The Earth System has many components that show non-linear dynamics, we have plenty of observations and often need to apply rough approximations to formulate our models. Peter Düben Page 4
  • 7. Deep learning for weather and climate The Earth System has many components that show non-linear dynamics, we have plenty of observations and often need to apply rough approximations to formulate our models. www.wikipedia.org Neural Networks learn from input/output pairs. Neurons have weighted connections to each other and the weights are trained to produce the optimal results. Neural networks can emulate non-linear systems. Peter Düben Page 4
  • 8. Future perspective for machine learning in weather forecasts Peter Düben Page 5
  • 9. Future perspective for machine learning in weather forecasts The next slides: I will show example applications and discuss challenges. Peter Düben Page 5
  • 10. To learn the equations Neural networks enable us to learn complex non-linear dynamics as a black box. Peter Düben Page 6
  • 11. To learn the equations Neural networks enable us to learn complex non-linear dynamics as a black box. We could base the entire model on Neural Networks. Who needs Navier Stokes? We know the equations of motion but we cannot solve them. Discretisation and sub-grid-scale variability generates significant errors. The data handling system of ECMWF provides access to over 210 petabyte of primary data and the data archive of ECMWF grows by about 233 terabyte per day. Peter Düben Page 6
  • 12. Global weather forecast based on Neural Networks Retrieve hourly data of geopotential height at 500 hPa from ERA5 re-analysis for training (> 65000 global data sets). Map the data to a coarse lon/lat grid (60x31). Use the state of the model at timestep i as input and the state of the model at timestep i + 1 as output. Use a 9 × 9 stencil around the grid point that should be predicted. Add time of day and year as well as the coordination of a gridpoint (lon+lat) as input variables to the network. The Pole needs special treatment. Peter Düben Page 7
  • 13. Global weather forecast based on Neural Networks Peter Düben Page 8
  • 14. Global weather forecast based on Neural Networks The Neural Network model can compete with a dynamical model of similar complexity. Dueben and Bauer GMD 2018 Peter Düben Page 8
  • 15. Global weather forecast based on Neural Networks 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 Peter Düben Page 9
  • 16. Global weather forecast based on Neural Networks 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 The simulations show reasonable dynamics. Peter Düben Page 9
  • 17. Global weather forecast based on Neural Networks 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 The simulations show reasonable dynamics. Just adding further inputs does not necessarily help. Peter Düben Page 9
  • 18. Global weather forecast based on Neural Networks 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network 470 480 495 510 525 540 555 570 585 600 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Analysis -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 24 hours; Local Neural Network -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 The simulations show reasonable dynamics. Just adding further inputs does not necessarily help. Model runs crash after a couple of weeks. Dueben and Bauer GMD 2018 Peter Düben Page 9
  • 19. Let’s use neural networks to emulate existing parametrisation schemes Store input/output pairs of parametrisation schemes. Use this data to train a neural network to do the same job. Replace the parametrisation scheme by the neural network. Peter Düben Page 10
  • 20. Let’s use neural networks to emulate existing parametrisation schemes Store input/output pairs of parametrisation schemes. Use this data to train a neural network to do the same job. Replace the parametrisation scheme by the neural network. Why would you do this? A large fraction of the computational cost is generated by parametrisation scheme. Parametrisation schemes cause > 90% of model code. Optimization of this code is very difficult (→ less than 5% peak performance). Neural Networks are highly optimized and can even use co-designed hardware. → Portability comes for free. Peter Düben Page 10
  • 21. Let’s use neural networks to emulate existing parametrisation schemes Store input/output pairs of parametrisation schemes. Use this data to train a neural network to do the same job. Replace the parametrisation scheme by the neural network. Why would you do this? A large fraction of the computational cost is generated by parametrisation scheme. Parametrisation schemes cause > 90% of model code. Optimization of this code is very difficult (→ less than 5% peak performance). Neural Networks are highly optimized and can even use co-designed hardware. → Portability comes for free. We hope that deep Neural Networks will be almost as good as the original parametrisation schemes but much more efficient. Peter Düben Page 10
  • 22. A neural network emulator for the state-of-the-art model configuration with 137 vertical levels 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 10140928 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 1.01303e+07 Dueben, Hogan, Bauer @ECMWF and Progsch, Angerer @NVIDIA Downward solar radiation at the surface for the original radiation scheme and the Neural Network emulator. Peter Düben Page 11
  • 23. A neural network emulator for the state-of-the-art model configuration with 137 vertical levels 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 10140928 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°N 10°S 20°S 30°S 40°S 50°S 60°S 70°S 80°S 10°N 20°N 30°N 40°N 50°N 60°N 70°N 80°N 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E 0°E20°W40°W60°W80°W100°W120°W140°W160°W 20°E 40°E 60°E 80°E 100°E 120°E 140°E 160°E Monday 31 July 2017 12 UTC ecmf t+3 VT:Monday 31 July 2017 15 UTC surface Surface solar radiation downwards 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 1.01303e+07 Dueben, Hogan, Bauer @ECMWF and Progsch, Angerer @NVIDIA Downward solar radiation at the surface for the original radiation scheme and the Neural Network emulator. Free-running model simulations are stable and we get a factor of O(10) speed-up for radiation. Peter Düben Page 11
  • 24. A neural network emulator for gravity wave drag Original scheme Difference Neural Network Chantry, Dueben, Palmer. Tendency output for the non-orographic gravity wave drag parametrisation scheme for the standard scheme and a neural network emulator. Peter Düben Page 12
  • 25. And one more thing... Neumann, Dueben, et al., Phil. Trans. A, 2019 Weather and climate models are high performance computing applications: More resolution → more processes resolved. Ratio sustained/peak is going down. Machine learning has a very strong impact on hardware developments at the moment. Peter Düben Page 13
  • 26. And one more thing... Neumann, Dueben, et al., Phil. Trans. A, 2019 Weather and climate models are high performance computing applications: More resolution → more processes resolved. Ratio sustained/peak is going down. Machine learning has a very strong impact on hardware developments at the moment. Can we use deep learning hardware for conventional models? Peter Düben Page 13
  • 27. Use machine learning hardware for the Legendre transform Relative cost for model components for a non-hydrostatic model at 1.45 km resolution: Peter Düben Page 14
  • 28. Use machine learning hardware for the Legendre transform Relative cost for model components for a non-hydrostatic model at 1.45 km resolution: The Legendre transform is the most expensive kernel. It consists of a large number of standard matrix-matrix multiplications. If we can re-scale the input and output fields, we can use half precision arithmetic. Tensor Cores on NVIDIA Volta GPUs are optimised for half-precision matrix-matrix calculations with single precision output. 7.8 TFlops for double precision vs. 125 TFlops for half precision on the Tensor Core. Peter Düben Page 14
  • 29. Half precision Legendre Transformations 0 2 4 6 8 10 Forecast lead time (days) 0.0 1.0 2.0 3.0 4.0 500hPatemperatureRMSE Northern extratropics (20N-90N) double half trans 25 tensor core 0 2 4 6 8 10 Forecast lead time (days) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Tropics (20S-20N) 0 2 4 6 8 10 Forecast lead time (days) 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Southern extratropics (90S-20S) Root-mean-square error for Z500 at 9 km resolution averaged over multiple start dates. Hatfield, Chantry, Dueben, Palmer Best Paper Award PASC2019 Peter Düben Page 15
  • 30. Half precision Legendre Transformations 0 2 4 6 8 10 Forecast lead time (days) 0.0 1.0 2.0 3.0 4.0 500hPatemperatureRMSE Northern extratropics (20N-90N) double half trans 25 tensor core 0 2 4 6 8 10 Forecast lead time (days) 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Tropics (20S-20N) 0 2 4 6 8 10 Forecast lead time (days) 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Southern extratropics (90S-20S) Root-mean-square error for Z500 at 9 km resolution averaged over multiple start dates. Hatfield, Chantry, Dueben, Palmer Best Paper Award PASC2019 The simulations are using an emulator to reduce precision. Dawson and Dueben GMD 2017 Peter Düben Page 15
  • 31. The challenge for the machine-learning community Peter Düben Page 16
  • 32. The challenge for the machine-learning community The time of nice pictures and videos is over... Peter Düben Page 16
  • 33. The challenge for the machine-learning community The time of nice pictures and videos is over... We need to prove that machine learning tools are better than state-of-the-art within the next two years. Peter Düben Page 16
  • 34. The challenge for the machine-learning community The time of nice pictures and videos is over... We need to prove that machine learning tools are better than state-of-the-art within the next two years. We need to build useful tools that improve weather and climate predictions and/or help to improve our understanding of the Earth System and are able to convince the conservative weather and climate scientist. Peter Düben Page 16
  • 35. Why is this challenge so hard? 0°N 0°N 120°E 120°E 0°N 0°N 120°E 120°E Dueben, Wedi, Saarinen submitted Top-of-the-atmosphere cloud brightness temperature [K] for satellite observations and a simulation of the atmosphere with 1.45 km resolution. A weather forecast simulation has O(1,000,000,000) degrees-of-freedom. Peter Düben Page 17
  • 36. Why is this challenge so hard? 0°N 0°N 120°E 120°E 0°N 0°N 120°E 120°E Dueben, Wedi, Saarinen submitted Top-of-the-atmosphere cloud brightness temperature [K] for satellite observations and a simulation of the atmosphere with 1.45 km resolution. A weather forecast simulation has O(1,000,000,000) degrees-of-freedom. Global simulations show a breath-taking level of complexity and can represent many details of the Earth System. Peter Düben Page 17
  • 37. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... Peter Düben Page 18
  • 38. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? Peter Düben Page 18
  • 39. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? Peter Düben Page 18
  • 40. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? Peter Düben Page 18
  • 41. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? Peter Düben Page 18
  • 42. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? How can we guarantee reproducibility during training? Peter Düben Page 18
  • 43. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? How can we guarantee reproducibility during training? How can we get beyond “dense” networks but still take local properties into account? Peter Düben Page 18
  • 44. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? How can we guarantee reproducibility during training? How can we get beyond “dense” networks but still take local properties into account? How can we fix interactions between model components? Peter Düben Page 18
  • 45. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? How can we guarantee reproducibility during training? How can we get beyond “dense” networks but still take local properties into account? How can we fix interactions between model components? How can we interface neural networks with legacy codes and how can we use AI and conventional methods efficiently on the same hardware? Peter Düben Page 18
  • 46. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? How can we guarantee reproducibility during training? How can we get beyond “dense” networks but still take local properties into account? How can we fix interactions between model components? How can we interface neural networks with legacy codes and how can we use AI and conventional methods efficiently on the same hardware? How can we design good training data (short time steps and high resolution)? Peter Düben Page 18
  • 47. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? How can we guarantee reproducibility during training? How can we get beyond “dense” networks but still take local properties into account? How can we fix interactions between model components? How can we interface neural networks with legacy codes and how can we use AI and conventional methods efficiently on the same hardware? How can we design good training data (short time steps and high resolution)? How can we explore the full phase space (all weather regimes) during training? Peter Düben Page 18
  • 48. Questions to be answered... There is no fundamental reasons not to use a black box within weather and climate models. However,... How can we use our knowledge about the Earth System? How can we diagnose physical knowledge from the network? How can we remove a bias from neural networks and secure conservation laws? How can we pick hyper-parameters? How can we guarantee reproducibility during training? How can we get beyond “dense” networks but still take local properties into account? How can we fix interactions between model components? How can we interface neural networks with legacy codes and how can we use AI and conventional methods efficiently on the same hardware? How can we design good training data (short time steps and high resolution)? How can we explore the full phase space (all weather regimes) during training? How can we generate large labelled datasets? Peter Düben Page 18
  • 49. My personal vision of the way forward... Weather and climate models Deep learning Peter Düben Page 19
  • 50. My personal vision of the way forward... Weather and climate models Deep learning To study known differential equations to learn how to derive blueprints for neural network architectures. Peter Düben Page 19
  • 51. My personal vision of the way forward... Weather and climate models Deep learning To study known differential equations to learn how to derive blueprints for neural network architectures. To learn how to scale neural networks to O(1,000,000) inputs for 3D fields on the sphere. Peter Düben Page 19
  • 52. My personal vision of the way forward... Weather and climate models Deep learning To study known differential equations to learn how to derive blueprints for neural network architectures. To learn how to scale neural networks to O(1,000,000) inputs for 3D fields on the sphere. To study the representation of variability and the correction of systematic errors for neural networks. Peter Düben Page 19
  • 53. My personal vision of the way forward... Weather and climate models Deep learning To study known differential equations to learn how to derive blueprints for neural network architectures. To learn how to scale neural networks to O(1,000,000) inputs for 3D fields on the sphere. To study the representation of variability and the correction of systematic errors for neural networks. To build community tools in the form of benchmark problems. Peter Düben Page 19
  • 54. My personal vision of the way forward... Weather and climate models Deep learning To study known differential equations to learn how to derive blueprints for neural network architectures. To learn how to scale neural networks to O(1,000,000) inputs for 3D fields on the sphere. To study the representation of variability and the correction of systematic errors for neural networks. To build community tools in the form of benchmark problems. To focus on useful tools that can serve as beacon. Peter Düben Page 19
  • 55. My personal vision of the way forward... Weather and climate models Deep learning To study known differential equations to learn how to derive blueprints for neural network architectures. To learn how to scale neural networks to O(1,000,000) inputs for 3D fields on the sphere. To study the representation of variability and the correction of systematic errors for neural networks. To build community tools in the form of benchmark problems. To focus on useful tools that can serve as beacon. This will be hard work. Peter Düben Page 19
  • 56. My personal vision of the way forward... Weather and climate models Deep learning To study known differential equations to learn how to derive blueprints for neural network architectures. To learn how to scale neural networks to O(1,000,000) inputs for 3D fields on the sphere. To study the representation of variability and the correction of systematic errors for neural networks. To build community tools in the form of benchmark problems. To focus on useful tools that can serve as beacon. This will be hard work. We will require machine learning solutions that are customised to weather and climate models. Peter Düben Page 19
  • 57. An example: The Burgers equation Let’s represent a non-linear system that is approximated by the Burgers’ equation: ∂u ∂t = ν ∂2 u ∂x2 − u ∂u ∂x + p. Peter Düben Page 20
  • 58. An example: The Burgers equation Let’s represent a non-linear system that is approximated by the Burgers’ equation: ∂u ∂t = ν ∂2 u ∂x2 − u ∂u ∂x + p. The conventional approach: ∂ui ∂t = ν ui+1 − 2ui + ui−1 ∆x2 − ui ui+1 − ui−1 2∆x + c0 + c1 · ui + c2 · ui + c3 · ui · ζ. Peter Düben Page 20
  • 59. An example: The Burgers equation Let’s represent a non-linear system that is approximated by the Burgers’ equation: ∂u ∂t = ν ∂2 u ∂x2 − u ∂u ∂x + p. The conventional approach: ∂ui ∂t = ν ui+1 − 2ui + ui−1 ∆x2 − ui ui+1 − ui−1 2∆x + c0 + c1 · ui + c2 · ui + c3 · ui · ζ. The data-science approach: Standard Neuron InputOutput Standard connection Differential quotient 1 Differential quotient 2 Product pooling Peter Düben Page 20
  • 60. An example: The Burgers equation Let’s represent a non-linear system that is approximated by the Burgers’ equation: ∂u ∂t = ν ∂2 u ∂x2 − u ∂u ∂x + p. The conventional approach: ∂ui ∂t = ν ui+1 − 2ui + ui−1 ∆x2 − ui ui+1 − ui−1 2∆x + c0 + c1 · ui + c2 · ui + c3 · ui · ζ. The data-science approach: The way forward: Standard Neuron InputOutput Standard connection Differential quotient 1 Differential quotient 2 Product pooling Peter Düben Page 20
  • 61. Conclusions Weather and climate models offer a large number of promising applications for machine learning that span the entire workflow of numerical weather predictions. We can learn the equations of motion using deep learning and build medium-complexity models. We need to develop useful tools based on machine learning within the next two years. We need to learn how to emulate known differential equations, represent sub-grid-scale variability and systematic errors, and learn how to scale machine learning solutions to O(1,000,000) input fields. We need benchmark datasets and beacons. Machine learning hardware can be used to speed-up conventional model simulations. Peter Düben Page 21
  • 62. Conclusions Weather and climate models offer a large number of promising applications for machine learning that span the entire workflow of numerical weather predictions. We can learn the equations of motion using deep learning and build medium-complexity models. We need to develop useful tools based on machine learning within the next two years. We need to learn how to emulate known differential equations, represent sub-grid-scale variability and systematic errors, and learn how to scale machine learning solutions to O(1,000,000) input fields. We need benchmark datasets and beacons. Machine learning hardware can be used to speed-up conventional model simulations. Many thanks for your attention. Peter Düben Page 21
  • 63. Funding The presenter gratefully acknowledges the funding a Royal Society University Research Fellowship and from the ESIWACE and ESIWACE2 project. The ESIWACE2 projects have received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 823988. Peter Düben Page 22