Contenu connexe

Similaire à NS-CUK Joint Journal Club: Minwoo Choi, Review on "Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures", Appied Energy 333 2023(20)

Plus de ssuser4b1f48(20)


NS-CUK Joint Journal Club: Minwoo Choi, Review on "Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures", Appied Energy 333 2023

  1. Min-Woo Choi Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: 2023. 02. 14 Publish: 2022 Journal: Energy (IF: 8.85)
  2. 1  Introduction • Limitation • Purpose • Main contribution  Methodology • Data description • Experiment setup  Results  Conclusions  Limitation and Future work
  3. 2 1. Introduction Limitation • CNN-based spatio-temporal prediction models require wind information that nodes at regular intervals or nodes in a square array. • But actual wind farms are not necessarily arranged regularly. Related study • Fu et al. (2019): Spatiotemporal attention network (STAN) - Using Multi head self attention to extract the spatial correlation. • Khodayar et al. (2019): Graph convolution deep learning architecture (GCDLA) - To capture the deep spatiotemporal information of wind speed. 1. Develop a relatively simple multi-node forecasting model by taking advantage of the various architectures of the above methods. 2. Proposed Spatial-Temporal Graph Transformer Network (STGTN) model to improve short- term wind speed prediction performance. Purpose of study
  4. 3 1. Introduction Main contribution • External attention mechanism is incorporated into the forecasting model  It can capture dynamic spatial information and reduce the complexity of the network. • A transformer model with graph convolution is proposed  To learn spatial correlation based on the Euclidean distance between wind farms.
  5. 4 2. Methodology Data description • Location: Danish offshore wind farms • Period : February 6, 2014 to June 6, 2014 • Number of data: 111 wind turbine node (14,400 points) • Resolution: 10 min • Range of data: • Training/Validation/Test set: 3:1:1 • Using historical data for each 12 points, the wind speed is predicted for 10 min to 1 h.
  6. 5 2. Methodology Problem formulation • Position information between wind turbines is represented as a graph 𝐺 = 𝑉, 𝐸, 𝐴 𝑉 = 𝑛𝑜𝑑𝑒𝑠 𝑜𝑓 𝑤𝑖𝑛𝑑 𝑡𝑢𝑏𝑖𝑛𝑒𝑠 𝐸 = 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑖𝑜𝑛𝑠 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑡ℎ𝑒 𝑛𝑜𝑑𝑒𝑠 𝐴 = 𝑎𝑑𝑗𝑎𝑐𝑒𝑛𝑐𝑦 𝑚𝑎𝑡𝑟𝑖𝑥 𝑤𝑖𝑡ℎ 𝐸𝑢𝑐𝑙𝑖𝑑𝑒𝑎𝑛 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑛𝑜𝑑𝑒𝑠 Spatialtemporal model of Short-term wind speed forecasting Experiment setup • Batch size: 64 • learning rate: 0.01 • Optimization: Adam • Decay rate of 0.7 after every 10 epochs 𝑉 𝐸 𝐴 1 2 3 2 3 𝑑 𝑑
  7. 6 2. Methodology Graph transformer Structure of the graph transformer  The fusion of spatial and temporal information  it considered to improve the robustness of extracting spatial features of wind speed  higher-level features can be extracted in the spatial correlation based on Euclidean distance External attention mechanism
  8. 7 2. Methodology Architecture of the proposed model Aggregate spatiotemporal features Extract temporal feature Extract spatial feature Form input through residual connection Fig. 1. Illustration of the proposed spatial-temporal graph transformer network (STGTN). Main contribution
  9. 8 2. Methodology Benchmark models • SVR • DL-STF • STAN • STGTN-T (Transformer) • STGTN (proposed) Not include spatial information Based on spatial-temporal information (previous study) Transformer and MLP
  10. 9 3. Results 10 min to 1 h prediction • The results show that STGTN dominates all considered methods in terms of the lowest RMSE values. • STGTN with MLP outperforms STGTN-T with transformer. • Forecasting models based on spatiotemporal information can track changes of wind speed faster. • The performance of the STGTN model is more stable compared with the DL-STF and STAN models. 0 min to 2000 min prediction Consider the all wind turbine node
  11. 10 3. Results May 20, performance • These results show that spatial information may degrades the forecasting performance when the standard deviation of wind speed is small. May 22, performance Wind speed Condition: fluctuation is small • SVR is better than DL-STF • SVR is better than STGTN-T
  12. 11 3. Results June 4, performance May 21, performance June 4, performance Wind speed Condition: fluctuation is large • The MLP and Transformer performs similarly as the feature extractor when large-scale wind speed fluctuations exist.  On the whole, STGTN provides more stable and accurate wind speed forecasts compared with other methods under different wind speed conditions.
  13. 12 4. Conclusions • The results indicate that the proposed model can effectively utilize the spatiotemporal information to generate more accurate wind speed forecasts. • Wind speed data of adjacent nodes is sufficiently used to correct the wrong predictions caused by outliers in the historical data of individual nodes. • The forecasting results confirms that the proposed model can yield stable wind speed forecasts regardless of different scales of wind speed fluctuations.
  14. 13 5. Limitation and Future work • Seasonal factors are not considered. (just consider May & June) • The utilization of wind direction information is not discussed.
  15. 14 Thank you!

Notes de l'éditeur

  1. 안녕하십니까, 저는 기상환경 관련 연구를 수행했던 최민우라고 하구요. 저의 전공지식을 토대로 이오준 교수님과 앞으로 연구를 진행해 나갈 예정입니다. 앞으로 영어 실력의 향상을 위해 영어로 발표하겠으나. 아직은 대본을 대부분 참조한다는 점 양해부탁드립니다. The topic of my presentation is Short-term wind speed forecasting based on spatial-temporal graph transformer networks.
  2. The order of contents is as follows:
  3. First, the limitations of previous studies in wind speed prediction are as follows. When we use a spatiotemporal prediction model to predict wind speed based on CNN, a square array or nodes with regular intervals are required. Like that grid. However, in actually it is not composed of regular data. Therefore, as a related study, Fu extracted spatial correlation using the STAN model. and Kyodayar extracted spatiotemporal information of wind speed using the GCDLA model. There are few related studies, but taking advantage of related studies, this study proposed STGTN to predict wind speed.
  4. The main contribution of this paper, First~ Second~
  5. The data used were wind farms located in Danish offshore, and data were collected at 10-minute intervals with 111 wind turbine nodes. The data set was divided into 3:1:1, and the wind speed was predicted using the historical data of each 12 points.
  6. The experiment setup is as written, and the information of wind turbines is constructed as follows. V is ~ E is ~ A is ~
  7. First, this is a structure of the graph transformer model. First Attention mechanism cannot capture time series and spatial information compared with the recurrent and convolutional structure. Therefore, the spatial-temporal position in embedding layer, temporal and spatial information is injected into the input sequence before performing subsequent operations. In External attention mechanism ~ And Graph convolution extract the higher level features in spatial correlation
  8. It is a configuration of the proposed model as STGTN. first, spatial features are extracted from the Graph transformer, and second, inputs are formed through residual connections, moved to MLP, TEMPORAL features are extracted. In the last convolutional layer, the wind speed is predicted by aggregate.
  9. Benchmark models were constructed to validate the proposed model. First, SVR model not include the spatial information, DL-STF and STAN models based on spatial-temporal information and the STGTN-T model using Transformer was used instead of MLP model.
  10. Next is the results part. Looking at the table, the proposed model performed better than the benchmarking models. What is noteworthy here is that STGTN showed better performance than STGTN-T, which means that the MLP used instead of the transformer improved the prediction accuracy, It has the advantage of reducing the overhead of model training and hyperparameter tuning with a simple structure.
  11. Next, when the days of wind speed fluctuation is small, look at the upper table SVR is better than DL-STF and under table SVR is better than STGTN-T. These results show that spatial information may degrades the forecasting performance when the Flutuation is small.
  12. Conversely, STGTN-T and STGTN were found to be effective when the fluctuations is large. The proposed models all showed good predictive performance regardless of wind speed conditions.
  13. The proposed model is stable and has excellent spatio-temporal predictability.
  14. Anyone have any questions?