Choi_ICONI2020_발표자료_v1.0.pdf

1. Enhancing Environmental Data Forecasting Performance by Utilizing Multi-region Data with Hard-parameter sharing Electronics and Telecommunications Research Institute Jang-Ho Choi, Miyoung Jang, Taewhi Lee, Jongho Won and Jiyong Kim janghochoi@etri.re.kr

2. Table of Contents • Introduction • Challenges • Proposed Method • Experimental Studies • Q&A

3. Time-series Forecasting • “Time series analysis comprises methods for analyzing time series data in order to extract meaningful statistics and other characteristics of the data.” • “Time series forecasting is the use of a model to predict future values based on previously observed values.” • “Forecasting involves taking models fit on historical data and using them to predict future observations.” https://en.wikipedia.org/wiki/Time_series https://machinelearningmastery.com/time-series-forecasting/

4. Environmental Data • Environmental data, i.e. air quality and water quality data, are generally multivariate, time-series data. • They are generally collected from multiple sites. <Water/Air Quality Monitoring Stations>

5. Challenges • Datasets are not only scarce and expensive, but also often severely imbalanced in practice. • Insufficient and imbalance datasets can lead to biased models, resulting poor prediction performance. 0 10 20 30 40 50 60 70 80 90 Chl-a (mg/m3) Histogram of Chlorophyll-a Concentration (mg/m3, Binwidth=1) Chlorophyll-a Concentration in Daecheong Lake (2012) # Chl-a concentrations

6. Detour: Handling Data imbalance and insufficiency

7. Utilizing Multi-region Datasets • Training Regional Dataset (JangGye) vs. Additional Dataset (JangGye + DaeCheong) R2 MSE Micro Acc . Macro Ac c. JangGye 0.5297 36.16 73.98% 66.72% JangGye +DaeCheong 0.4207 44.55 68.65% 49.04% Difference 26.19% -18.83% 7.76% 36.05% <Chlorophyll-a Prediction Results (JangGye)>

8. Hard-parameter Sharing • In order to preserve region-specific characteristic, we adopt hard- parameter sharing from multi-task learning [7]. • Hard-parameter sharing is a technique to place common hidden layers across multiple tasks, while keeping task-specific layers. Hard-parameter sharing is known to reduce overfitting, but it is generally restricted to tasks that are closely related one another [8]. [8]

9. Multi-headed Networks • Common layers fit on all data from multiple regions • Region specific layers fit on their own regional data <Iterative Training Process> Common Layers R1 Layers R2 Layers R3 Layers R4 Layers R1 Data R2 Data R3 Data R4 Data 입력 데이터

10. Experimental Studies Multi-headed Neural Network Model

11. Algal Bloom • “An algal bloom or algae bloom is a rapid increase or accumulation in the population of algae in freshwater or ma rine water systems” (Wikipedia) • For last few decades, algal bloom has been a serious problem, leading imbalance of organisms in the water. • Severe algal bloom directly affects water quality as some of algae such as cyanobacteria produce toxic that are har mful to other species—including humans. YTN Joongang

12. Chlorophyll-a • “Chlorophyll-a is a specific form of chlorophyll used in oxygenic p hotosynthesis. It absorbs most energy from wavelengths of violet- blue and orange-red light. It also reflects green-yellow light, and a s such contributes to the observed green color of most plants.” (W ikipedia) • Korean Government has been operating the algal warning system since 1998 based on chlorophyll-a concentration and the number of cyanophyceae cells. • Although using the concentration of chlorophyll-a as the primary indicator has been debatable, it is still widely used as it can be measured at low cos t, whereas counting cyanophyceae cells incur considerable human labor. • Unfortunately, water quality datasets are not only scarce, but also skewed and imbalanced due to their nature.

13. Datasets • Water Quality Monitoring Station (http://www.koreawqi.go.kr/) • Daecheong, Janggye, Chungam, Namgang • 2012/7/1 ~ 2018/3/31 • 8 Water Quality Index Variables: • Chlorophyll-a • Water Temperature, pH • Total Electrical conductivity • Dissolved Oxygen, Total Organic Carbons • Total Nitrogen, Total Phosphorus Class 0 Class 1 Class 2 Class 3 Chl-A 0~15mg/m3 15~25mg/m3 25~100mg/m3 100mg/m3+ JangGye 1217 255 142 0 DaeCheong 1451 164 70 2 CheongAm 794 523 535 10 NamGang 1478 156 365 5

14. Experiment Settings f(x) x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn y y f(x) x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn y y y y y y f(x) g1(z) g2(z) g3(z) x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn x1 x2 x3 x4 … xn y y y y y y Single region Data

15. Experiment Results R2 MSE Micro Acc. Macro Acc. Baseline (Single Region) 0.5297 36.16 73.98% 66.72% Baseline (Multiple Regions) 0.4207 44.55 68.65% 49.04% Multi-headed (Multiple Regions) 0.5633 32.70 75.36% 70.04% <Chlorophyll-A Prediction Results (JangGye)> Baseline Model with Single Region (upper) vs. Multi-head Neural Network Model (lower) Class 0 Class 1 Class 2 Chl-A 0~15mg/m3 15~25mg/m3 25~100mg/m3 Baseline (Single Region) 83.00% 59.26% 57.89% Baseline (Multiple Regions) 91.50% 34.57% 21.05% Multi-headed (Multiple Regions) 83.72% 58.89% 67.50% <Performance Comparison (Class Accuracy)> <Performance Comparison (R2, MSE, Micro and Macro Accuracy)>

16. Acknowledgement • This work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. 2018-0- 00219, Space-time complex artificial intelligence blue-green algae prediction technology based on direct-readable water quality complex sensor and hyperspectral image).

17. References 1) Agrawal, S., Barrington, L., Bromberg, C., Burge, J., Gazen, C., and Hickey, J., “Machine Learning for Precipitation Nowcasting from Radar Images,” arXiv:cs.CV/1912.12132, 2019 2) Kurt Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural Networks Volume 4, Issue 2, Pages 251-257, 1991 3) Choi, J., Park, J., Park, H. and Min, O. (2017), DART: Fast and Efficient Distributed Stream Processing Framework for Internet of Things. ETRI Journal, 39: 202-212. doi:10.4218/etrij.17.2816.0109 4) Salman H. Khan et al., Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data, IEEE Transactions on Neural Networks and Learning Systems, Vol. 29, Issue: 8 (2015) 5) N. V. Chawla et al., SMOTE: Synthetic Minority Over-sampling Technique, Journal of Artificial Intelligence Research, Vol. 16 (2002) 6) J. Choi, J. Kim, J. Won and O. Min, "Modelling Chlorophyll-a Concentration using Deep Neural Networks considering Extreme Data Imbalance and Skewness," 2019 21st International Conference on Advanced Communication Technology (ICACT) doi: 10.23919/ICACT.2019.8702027. 7) Caruana, R. "Multitask learning: A knowledge-based source of inductive bias." Proceedings of the Tenth International Conference on Machine Learning. 1993. 8) Sebastian Ruder, “An Overview of Multi-Task Learning in Deep Neural Networks,” arXiv:cs.LG/1706.05098, 2017

18. Q&A Thank you!

Choi_ICONI2020_발표자료_v1.0.pdf

Choi_ICONI2020_발표자료_v1.0.pdf

Recommandé

Contenu connexe

Similaire à Choi_ICONI2020_발표자료_v1.0.pdf

Similaire à Choi_ICONI2020_발표자료_v1.0.pdf(20)

Dernier

Dernier(20)

Choi_ICONI2020_발표자료_v1.0.pdf