Demographic forecasting

1. Demographic forecasting using functional data analysis Rob J Hyndman Joint work with: Heather Booth, Han Lin Shang, Shahid Ullah, Farah Yasmeen. Demographic forecasting using functional data analysis 1

2. Mortality rates France: male mortality (1816) 0 −2 Log death rate −4 −6 −8 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis 2

3. Fertility rates Australia: fertility rates (1921) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis 3

4. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis 4

5. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis A functional linear model 5

6. Some notation Let yt,x be the observed (smoothed) data in period t at age x, t = 1, . . . , n. yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Estimate ft (x) using penalized regression splines. Estimate µ(x) as me(di)an ft (x) across years. Estimate βt,k and φk (x) using (robust) functional principal components. iid iid εt,x ∼ N(0, 1) and et (x) ∼ N(0, v(x)). Demographic forecasting using functional data analysis A functional linear model 6

11. French mortality components 0.2 −1 0.20 0.1 −2 0.15 −3 φ1(x) φ2(x) 0.0 µ(x) 0.10 −4 −0.1 −5 0.05 −6 −0.2 0.00 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 Age Age Age 8 10 6 5 0 4 βt1 βt2 −5 2 −15 −10 0 −2 1850 1900 1950 2000 1850 1900 1950 2000 t t Demographic forecasting using functional data analysis A functional linear model 7

12. French mortality components Residuals 100 80 60 Age 40 20 0 1850 1900 1950 2000 Year Demographic forecasting using functional data analysis A functional linear model 7

13. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 8

14. French male mortality rates France: male death rates (1900−2009) 0 −2 Log death rate −4 −6 −8 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9

15. French male mortality rates France: male death rates (1900−2009) 0 War years −2 Log death rate −4 −6 −8 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9

16. French male mortality rates France: male death rates (1900−2009) 0 War years −2 Log death rate −4 −6 −8 Aims 1 “Boxplots” for functional data 0 20 240 Tools for detecting outliers in 60 80 100 functional data Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 9

17. Robust principal components Let {ft (x)}, t = 1, . . . , n, be a set of curves. 1 Apply a robust principal component algorithm n−1 ft (x) = µ(x) + βt,k φk (x) k =1 µ(x) is median curve {φk (x)} are principal components {βt,k } are PC scores 2 Plot βi ,2 vs βi ,1 ¯ Each point in scatterplot represents one curve. ¯ Outliers show up in bivariate score space. Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 10

26. Robust principal components Scatterplot of first two PC scores q q 6 q q q q 4 q q PC score 2 q 2 q q q q q qq qq q q q qq q qq q q q q q q q q qq q q q q q q qq q q qq qq qq qqq q q q q q q qq q qq q qq q q q qq q q qq 0 q q qq q q q q q q q q q qq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11

27. Robust principal components Scatterplot of first two PC scores 1915 1914q q 6 1916q 1918q 1944q q 1917 4 1940q 1943q PC score 2 1919q 2 q q q q q qq qq q 1945q q qq q qq q q q q q q q qq q 1942q q q q q q qq 1941qqqqqqq q qq q qq qqq qq q q qq q q q qqq qqq q qq q 0 q q qq q q q q q q q q q qq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 11

28. Functional bagplot Bivariate bagplot due to Rousseeuw et al. (1999). Rank points by halfspace location depth. Display median, 50% convex hull and outer convex hull (with 99% coverage if bivariate normal). 1915 1914q q q 6 q 1916qq 1918q q 1944q q 1917 q q 4 1940q q 1943q q PC score 2 1919q 2 q q q q q qq q q q q qqq q q q q q q q q 1945q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q qq q qq 0 q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12

29. Functional bagplot Bivariate bagplot due to Rousseeuw et al. (1999). Rank points by halfspace location depth. Display median, 50% convex hull and outer convex hull (with 99% coverage if bivariate normal). Boundaries contain all curves inside bags. 95% CI for median curve also shown. 0 1915 1914q q q 6 q 1916qq 1918q q −2 1944q q 1917 q q 4 1940q q 1943q q Log death rate PC score 2 −4 1919q 2 q q q q qq q q q q qqq q q q q q q q 1945q q −6 q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q qq q qq 0 q q q q q q q q qq q q q q 1914 1918 −8 q q q qqq q q q q q q 1915 1940 q q q q 1916 1943 1917 1944 −2 −10 −5 0 5 10 15 0 20 40 60 80 100 PC score 1 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 12

30. Functional bagplot 0 −2 Log death rate −4 −6 1914 1918 −8 1915 1940 1916 1943 1917 1944 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 13

31. Functional HDR boxplot Bivariate HDR boxplot due to Hyndman (1996). Rank points by value of kernel density estimate. Display mode, 50% and (usually) 99% highest density regions (HDRs) and mode. 99% outer region q 6 q q q q q 4 q 1943q q PC score 2 1919q 2 q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q qq o q q q q q qq q qqq q 0 q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14

32. Functional HDR boxplot Bivariate HDR boxplot due to Hyndman (1996). Rank points by value of kernel density estimate. Display mode, 50% and (usually) 99% highest density regions (HDRs) and mode. 90% outer region 1915 1914q q q 6 q 1916qq 1918q q 1944q q 1917 q q 4 1940q q 1943q q PC score 2 1919q 2 q q q q qq q q q q qqq q q q q q q q 1945q q q q q q q q q q 1942q q q qq q q q q q q q q q q q q q qq q q q q q q q q q qq o q q q q q qq q qqq q 0 q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q −2 −10 −5 0 5 10 15 PC score 1 Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14

33. Functional HDR boxplot Bivariate HDR boxplot due to Hyndman (1996). Rank points by value of kernel density estimate. Display mode, 50% and (usually) 99% highest density regions (HDRs) and mode. Boundaries contain all curves inside HDRs. 0 90% outer region 1915 1914q q q 6 q 1916qq 1918q q −2 1944q q 1917 q q 4 1940q q Log death rate 1943q q PC score 2 −4 1919q 2 q q q q qq q q q q qqq q q q q q q q 1945q q −6 q q q q q q q q 1942q q q qq q q q q q q q q q q q 1914 1940 q q qq q q q q q qq o q qq qq q q q q qq q qqq q 1915 1943 0 q q q q q q q q qq q q q q 1916 1944 −8 q q q qqq q q q q q q 1917 1945 q q q q 1918 1948 1919 −2 −10 −5 0 5 10 15 0 20 40 60 80 100 PC score 1 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 14

34. Functional HDR boxplot 0 −2 Log death rate −4 −6 1914 1940 1915 1943 1916 1944 −8 1917 1945 1918 1948 1919 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Bagplots, boxplots and outliers 15

35. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis Functional forecasting 16

36. Functional time series model yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 The eigenfunctions φk (x) show the main regions of variation. The scores {βt,k } are uncorrelated by construction. So we can forecast each βt,k using a univariate time series model. Outliers are treated as missing values. Univariate ARIMA models are used for forecasting. Demographic forecasting using functional data analysis Functional forecasting 17

41. Forecasts yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 Demographic forecasting using functional data analysis Functional forecasting 18

42. Forecasts yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 K E[yn+h ,x | y] = µ(x) + ˆ ˆ ˆ βn+h ,k φk (x) k =1 K ˆ2 Var[yn+h ,x | y] = σµ (x) + ˆ2 vn+h ,k φk (x) + σt2 (x) + v(x) k =1 where vn+h ,k = Var(βn+h ,k | β1,k , . . . , βn,k ) and y = [y1,x , . . . , yn,x ]. Demographic forecasting using functional data analysis Functional forecasting 18

43. Forecasting the PC scores Main effects Interaction 0 0.2 0.20 −2 Basis function 1 Basis function 2 0.1 0.15 Mean 0.0 −4 0.10 −0.1 0.05 −6 −0.2 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 Age Age Age 10 15 q 2 q qq q q q q 0 5 Coefficient 1 Coefficient 2 0 −2 q q −10 q −4 q q −6 q −20 q q 1900 1940 1980 2020 1900 1940 1980 2020 Year Year Demographic forecasting using functional data analysis Functional forecasting 19

44. Forecasts of ft (x) France: male death rates (1900−2009) 0 −2 Log death rate −4 −6 −8 −10 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20

45. Forecasts of ft (x) France: male death rates (1900−2009) 0 −2 Log death rate −4 −6 −8 −10 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20

46. Forecasts of ft (x) France: male death forecasts (2010−2029) 0 −2 Log death rate −4 −6 −8 −10 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20

47. Forecasts of ft (x) France: male death forecasts (2010 & 2029) 0 −2 Log death rate −4 −6 −8 −10 80% prediction intervals 0 20 40 60 80 100 Age Demographic forecasting using functional data analysis Functional forecasting 20

48. Fertility application Australia fertility rates (1921−2009) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 21

49. Fertility model 0.2 15 0.25 0.1 10 0.0 Φ1(x) Φ2(x) 0.15 Μ −0.1 5 0.05 0 −0.3 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 Age Age Age 10 8 6 5 4 0 Βt1 Βt2 2 −5 0 −4 −2 −10 1970 1980 1990 2000 2010 1970 1980 1990 2000 2010 t t Demographic forecasting using functional data analysis Functional forecasting 22

50. Fertility model Residuals 45 40 35 Age 30 25 20 15 1970 1980 1990 2000 Year Demographic forecasting using functional data analysis Functional forecasting 23

51. Fertility model Main effects Interaction 0.2 15 0.25 0.1 Basis function 1 Basis function 2 10 0.0 Mean 0.15 −0.1 5 0.05 0 −0.3 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 15 20 25 30 35 40 45 50 Age Age Age 10 15 20 25 8 6 Coefficient 1 Coefficient 2 4 2 5 0 0 −4 −2 −10 1970 1990 2010 2030 1970 1990 2010 2030 Year Year Demographic forecasting using functional data analysis Functional forecasting 24

52. Forecasts of ft (x) Australia fertility rates (1921−2009) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25

53. Forecasts of ft (x) Australia fertility rates (1921−2009) 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25

54. Forecasts of ft (x) Australia fertility rates: 2010−2029 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25

55. Forecasts of ft (x) Australia fertility rates: 2010 and 2029 80% prediction intervals 250 200 Fertility rate 150 100 50 0 15 20 25 30 35 40 45 50 Age Demographic forecasting using functional data analysis Functional forecasting 25

56. Outline 1 A functional linear model 2 Bagplots, boxplots and outliers 3 Functional forecasting 4 Forecasting groups 5 Population forecasting 6 References Demographic forecasting using functional data analysis Forecasting groups 26

57. The problem Let ft,j (x) be the smoothed mortality rate for age x in group j in year t. Groups may be males and females. Groups may be states within a country. Expected that groups will behave similarly. Coherent forecasts do not diverge over time. Existing functional models do not impose coherence. Demographic forecasting using functional data analysis Forecasting groups 27

63. Forecasting the coefficients yt,x = ft (x) + σt (x)εt,x K ft (x) = µ(x) + βt,k φk (x) + et (x) k =1 We use ARIMA models for each coefficient {β1,j ,k , . . . , βn,j ,k }. The ARIMA models are non-stationary for the first few coefficients (k = 1, 2) Non-stationary ARIMA forecasts will diverge. Hence the mortality forecasts are not coherent. Demographic forecasting using functional data analysis Forecasting groups 28

Demographic forecasting

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (16)

Similaire à Demographic forecasting

Similaire à Demographic forecasting (20)

Plus de Rob Hyndman

Plus de Rob Hyndman (16)

Dernier

Dernier (20)

Demographic forecasting