Coupling Neural Networks to GCMs

Noah Brenowitz
July 30, 2019
Princeton AOS Workshop, Princeton, NJ
NOAH D. BRENOWITZ - NOAHB@VULCAN.COM
Coupling Neural Networks
to GCMs
1

Climate Modeling at Vulcan Inc.
2
Christopher Bretherton
University of Washington
Oliver Fuhrer, MeteoSwiss
More people to
come….
Research will be open source and openly published
Collaborating with the FV3 Team at GFDL

Coarse-resolution dynamics
Apparent heating (K/day)
Apparent moistening
(g/kg/day)
s = T +
g
cp
z q =
Mass water vapor
Mass dry air
@s
@t
+ v · rs = Q1
@q
@t
+ v · rq = Q2
@u
@t
+ v · ru + f ⇥ u
1
⇢
rp = Q3
SW+ LW radiation, latent heating, etc
https://trello.com/c/wV03fVXiz

One climate model grid box.
4Photo Credit: Becky Hornbrook, published on WCRP webpage

Traditional method for parameterization
5
GigaLES, Khairtoudinov

Climate models have biases in mean state
CMIP5
(models)
GPCP
(observations)
Hwang and Frierson (2013)

Machine learning builds black boxes
Many 1000s of parameters
Need a lot of data
Designed to be trained not interpreted
Examples: Decision trees, neural networks, support vector
machines
Easy to tune/train Easy to interpret
Many parameters Few parameters

Neural networks are a popular machine learning model
x1 = (A1x0 + b1)
x2 = (A2x1 + b2)
...
y = out(Anxn 1 + bn)
Input #1
Input #2
Input #3
Input #4
Output #1
Output #2
Output #3
Output #4
Hidden
layer
Input
layer
Activation
layer
Output
layer
Activation function

Optimum
Traditional
parametrizations
How too few parameters may hurt.

1 0
A climate model time step
Radiative
Transfer
Cloud
Microphysics
Cumulus
convection
And many more
Dry
Dynamics
4980
50405100
51605160
5220
5220
5220
5280
5280
5340
5400
5460
5520
5580
5640
5640
5700
5700
5700
5760
5760
5820
5880
5880
University of Wyoming
−40
−35
−35
−35
−30
−30
−25
−20
−20
−20
−15
−15
−15−15
−15−15
−15
−15
−10
−10
−10
−10
−5
−5
−5
−5
−5
−5
−5
−5
University of Wyoming
−20 543
4 03953
1 BIKF
−37 522
2 BGBW
−11 574
34
−29 544
11 32618 −34 533
16 PABR
−30 530
5 PAOT
−32 528
5 PAOM
−36 516
13 PABE
−32 522
5 PAMC
−30 527
3 PAFA
−36 518
17 PANC
−36 516
25 PASN
−38 522
9 PACD −39 517
5 PADQ
−38 525
15 PAYA
−29 542
8 PANT
−26 550
32 PASY
−27 533
0 YVQ
−40 510
9 YUX
−24 554
18 YZT
−25 552
30 WSE
−14 576
10 WSA
−15 566
12 YQI
−18 565
3 WMW
−17 569
2 AYT
−16 560
9 YZV
−16 566
21 YJT
−24 546
27 YYR
−27 547
26 YAH
−23 552
31 YMO−26 553
18 WPL
−22 552
3 YQD
−34 532
11 YVP
−37 536
3 WPH
−28 550
18 ZXS
−42 506
8 YFB
−29 544
7 YYQ
−40 510
4 YRB
−33 526
12 YCB
−31 533
23 YBK
−25 536
2 YSM
−26 541
21 YYE
−14 498
10 YEV
−36 527
9 YXY
−11 584
33 KEY
−13 583
26 MFL
−13 582
29 JAX
−14 580
27 CHS
−12 584
25 TBW
−11 583
30 TLH
−13 582
30 FFC
−14 583
22 BMX
−11 585
24 LIX
−13 584
9 JAN
−12 584
2 LCH
−12 582
3 SHV
−13 579
2 FWD
−10 587
4 BRO
−10 585
5 CRP
−11 580
24 DRT
−16 573
19 MAF
−20 568
18 TUS
−15 572
18 NKX
−15 576
15 MHX
−15 578
9 GSO
−16 577
17 RNK
−16 580
4 BNA
−13 580
3 LZK
−15 574
2 OUN
−14 572
2 AMA
−17 571
29 EPZ
−17 570
3 ABQ
−19 571
4 FGZ
−17 573
10 VEF−11 580
14 VBG
−17 573
10 WAL
−16 574
16 IAD−17 575
5 ILN
−14 576
3 SGF
−14 573
3 DDC
−14 574
1 TOP
−16 572
8 DNR
−17 573
7 GJT
−12 582
11 REV
−11 584
17 OAK
−17 567
5 OKX
−17 567
5 ALB−17 570
4 BUF
−16 572
30 OAX
−16 572
17 LBF
−15 574
45 SLC
−12 578
13 LKN
−14 581
3 MFR
−16 566
11 DTX
−15 561
1 APX−17 563
14 GRB
−18 567
20 MPX
−17 569
21 ABR
−16 571
45 RAP−15 573
42 RIW
−14 577
3 BOI
−15 574
2 SLE
−16 566
16 CAR
−21 560
16 INL
−16 569
18 BIS
−14 567
4 GGW
−15 570
2 TFX
−16 569
2 OTX
−16 567
15 UIL
−18 559
15 CWVK
−19 569
12 1Y7
−20 561
3 GYX
−15 568
10 DVN
−19 559
5 CHH
−15 573
19 ILX
−12 574
2 LMN
−9 586
9 ADN
−9 584
5 76458
−9 587
13 76526
−7 588
17 76612 −7 589
16 76679
−13 578
13 TXKF
−13 582
25 MYNN
−7 586
0 MWCR
−8 584
23 MKJP
−9 586
8 MDSD
−9 584
23 TJSJ
−6 587
17 MZBZ
−4 587
34 MROC
−5 584
32 TFFR
−4 588
39 TTPP
−6 586
8 TNCC
−3 588
35 SKSP
−4 589
36 SKBQ
−7 588
0 SKBG
−5 588
−6 589
2 SBBV
−8 590
35 PHLI
−7 589
22 PHTO
12Z 23 Apr 2019 500 hPa

Near-global aqua-planet (NG-Aqua) simulation generated by the System for Atmospheric Modeling (Δ" = 4km)
11
https://vimeo.com/299128849

Coarse-grain data to 160 km boxes
12
Training regionTesting region
Coarse-graining
A
B
C

13
Parameterization as function approximation
!(#; %)
Yanai, et. al. (1973)
64 inputs + SST, SOLIN
68 outputs

Training Approach 1
1. Use finite differences to compute residual tendencies
2. Train neural network :
Q2 ⇡
¯qv(t + 3 h) ¯qv(t)
3 h
gLS(t), gLS(t) = ¯v · r¯qv
q, s, SHF, LHF, TOA Q1, Q2
Neural
Network
(1 layer of 256 nodes ≅ 70,000 parameters)

Neural networks can diagnose Q1 and Q2
15
Neural Network
~ 100,000 parameters
Millions of samples
Heating (K/day)
(finite diff.)
!" ≈ .50
Similar to Krasnopolsky et. al. (2013)

But its coupled!
1 6
Neural
Network
Dry
Dynamics

Test with single column model
1 7
Neural
Network
Single
column
model

Single Column Dynamics
x = [qv, s]
y = [SHF,LHF, TOASW,#]
dx
dt
= f(x, y(t)) + gLS(t)
Prognostic variables
Auxiliary variables

Uh oh…temperature = 1035 K after 1 day
Time (d)
p (hPa)

Is fitting Q1 and Q2 the right approach?
Assumes that model dynamics are continuous in
time
• But they are not (Donahue and Caldwell, 2018)
Assumes moist physics tendencies are available
and accurate
• Not true for DYAMOND outputs
• Not true for observations
Does not ensure good predictions over many time
steps

Fitting the approximate Q1 and Q2 is
equivalent to minimizing one-step error
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
50
100
150
200
Error
Truth
Prediction
Hours

…but that does not ensure longer term
performance
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
50
100
150
200
Error
Truth
Prediction
Hours

Stable and accurate single column model simulations
23
Community Atmosphere
Model Version 5 (CAM5)
Single Column Mode
(default physics, no
chemistry)
Humidity Anomaly
(from true zonal mean)
(g/kg)
days Brenowitz and Bretherton (2018)

Will it work in a GCM?
2 4
Neural
Network
Dry Dynamical
Core
Brenowitz and Bretherton (2019)

Coupled simulations with this model blow-up!
25
• 160 km resolution
• System for Atmos. Modeling
(SAM)
• Fortran calling python
0 5 10 15 20
x (1000 km )
0.0
2.5
5.0
7.5
10.0
y(1000km)
0
15
30
45
60
50 75 100125

The neural network finds spurious correlations!
26
Humidity at 200 mb and precipitation

Synchronization is hard for machine learning
27
Pendulums Synchronize People clapping
Distinct sub-systems (clappers, pendulums) are perfectly correlated
Timme, Max Planck Institute for Dynamics and Self-organization
Domain
Knowledge
Domain knowledge

Ignoring upper atmospheric humidity stabilizes coupled
simulation
28
Precipitable water after 5 days
Brenowitz and Bretherton, 2019

Improved coupled forecast accuracy (RMSE of humidity)

Longer-term
drifts in climate
30

Biases in circulation
31
Zonal+time mean over days 105-110

Lower Tropospheric Stability (LTS)
• T(600 mb) – T(surface)
• Strength of inversion
Mid tropospheric humidity
• ∫"##
$%
&' ()/+
• Strongly controls amount of
precipitation
Systematically Vary Inputs
33

34
Insert conditionally averaged input variables into
the neural network

Increasing LTS increases the depth of convection
35

Increasing moisture strengthens convection
36

37
Linearize the Neural Network

Linearized
Response
Function
38
!(#$, #&)
!((, ))
Similar to Kuang (2018)
Sensitivity of SGS moistening at 600 mb to
1 g/kg of additional moisture at 750 mb.

Linearized
Response
Function
39
!(#$, #&)
!((, ))
Similar to Kuang (2018)

The neural network finds spurious correlations!
40
Humidity at 200 mb and precipitation

Couple Linearized Response Functions to Gravity Waves
!′# + ̅!&'( = *
+
,
-./
-!
!((1) +
-./
-3
3( 1
41
5
3′# + 63&'′ = *
+
,
-.7
-!
!((1) +
-.7
-3
3( 1
41
5
'#
( = −
-
-9
7
:;/<( − 4'(
:'′ =
=
=&
/
>?
=
=&
@+'′ and <( = A
B(
̅B
41Similar to Kuang (2018)

Eigenmodes of this Linear System
42
With upper atmospheric input (unstable in GCM) Without upper atmospheric input (stable in GCM)
Phase speed (m/s) Phase speed (m/s)
Growthrate(1/day)
Wavenumber
(1/km)
Wavenumber
(1/km)

A scary eigenmode (w/ unstable NN)
43

A moisture-mode-like standing instability (w/ stable NN)
44

Conclusions and Future Directions
• Coupling with GCM makes ML parameterization hard
• Synchronization due to time-scale separation pollutes our training data
• Ignoring synchronized inputs stabilizes spatially extended simulations
• Accurate for ~2 days for precipitation, ~5 days for humidity and temperature
• Longer term drifts in the climate
• The NN is physically plausible:
• Increase humidity controls the strength of the predicted precipitation
• Increasing tropospheric stability controls the height of the predictions
• Linearized Response Functions can be coupled to waves
• Plot wave-speed/growth rate curves and wave structures.
45

References
Brenowitz, N. D., & Bretherton, C. S. Spatially Extended Tests of a Neural Network
Parametrization Trained by Coarse-graining. JAMES (2019).
Brenowitz, N. D. & Bretherton, C. S. Prognostic Validation of a Neural Network Unified
Physics Parameterization. Geophys. Res. Lett. 17, 2493 (2018).
S. Rasp, M. S. Pritchard, P. Gentine, Deep learning to represent subgrid processes in
climate models. Proc. Natl. Acad. Sci. U. S. A. 115, 9684–9689 (2018).
46

©2018 Vulcan Inc. All rights reserved. The information herein is for informational purposes only and represents the current view of Vulcan Inc. as of the date of this
presentation. VULCAN INC MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
47

Conditionally Averaged-data (|Lat| < 23 deg)
48
Total Population Precipitation – Evaporation Net heating

Coupling Neural Networks to GCMs

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Coupling Neural Networks to GCMs

Similaire à Coupling Neural Networks to GCMs (20)

Dernier

Dernier (20)

Coupling Neural Networks to GCMs