NS-CUK Joint Journal Club: Minwoo Choi, Review on "Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures", Appied Energy 333 2023
Network Science Lab
Dept. of Artificial Intelligence
The Catholic University of Korea
2023. 02. 14 Publish: 2022
Journal: Energy (IF: 8.85)
• Main contribution
• Data description
• Experiment setup
Limitation and Future work
• CNN-based spatio-temporal prediction models require wind information that nodes at regular
intervals or nodes in a square array.
• But actual wind farms are not necessarily arranged regularly.
• Fu et al. (2019): Spatiotemporal attention network (STAN)
- Using Multi head self attention to extract the spatial correlation.
• Khodayar et al. (2019): Graph convolution deep learning architecture (GCDLA)
- To capture the deep spatiotemporal information of wind speed.
1. Develop a relatively simple multi-node forecasting model by taking advantage of the various
architectures of the above methods.
2. Proposed Spatial-Temporal Graph Transformer Network (STGTN) model to improve short-
term wind speed prediction performance.
Purpose of study
• External attention mechanism is incorporated into the forecasting model
It can capture dynamic spatial information and reduce the complexity of the network.
• A transformer model with graph convolution is proposed
To learn spatial correlation based on the Euclidean distance between wind farms.
• Location: Danish offshore wind farms
• Period : February 6, 2014 to June 6, 2014
• Number of data: 111 wind turbine node (14,400 points)
• Resolution: 10 min
• Range of data:
• Training/Validation/Test set: 3:1:1
• Using historical data for each 12 points, the wind speed
is predicted for 10 min to 1 h.
• Position information between wind turbines is represented as a graph
𝐺 = 𝑉, 𝐸, 𝐴
𝑉 = 𝑛𝑜𝑑𝑒𝑠 𝑜𝑓 𝑤𝑖𝑛𝑑 𝑡𝑢𝑏𝑖𝑛𝑒𝑠
𝐸 = 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑖𝑜𝑛𝑠 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑡ℎ𝑒 𝑛𝑜𝑑𝑒𝑠
𝐴 = 𝑎𝑑𝑗𝑎𝑐𝑒𝑛𝑐𝑦 𝑚𝑎𝑡𝑟𝑖𝑥 𝑤𝑖𝑡ℎ 𝐸𝑢𝑐𝑙𝑖𝑑𝑒𝑎𝑛 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑛𝑜𝑑𝑒𝑠
Spatialtemporal model of Short-term wind speed forecasting
• Batch size: 64
• learning rate: 0.01
• Optimization: Adam
• Decay rate of 0.7 after every 10 epochs
1 2 3
Structure of the graph transformer
The fusion of spatial and
it considered to improve the robustness of
extracting spatial features of wind speed
higher-level features can be extracted in the
spatial correlation based on Euclidean distance
External attention mechanism
Architecture of the proposed model
Fig. 1. Illustration of the proposed spatial-temporal graph transformer network (STGTN).
• STGTN-T (Transformer)
• STGTN (proposed)
Not include spatial information
Based on spatial-temporal information (previous study)
Transformer and MLP
10 min to 1 h prediction
• The results show that STGTN dominates all
considered methods in terms of the lowest RMSE
• STGTN with MLP outperforms STGTN-T with
• Forecasting models based on spatiotemporal
information can track changes of wind speed faster.
• The performance of the STGTN model is more stable
compared with the DL-STF and STAN models.
0 min to 2000 min prediction
Consider the all wind turbine node
May 20, performance
• These results show that spatial information may degrades the forecasting performance when the
standard deviation of wind speed is small.
May 22, performance Wind speed Condition: fluctuation is small
• SVR is better than DL-STF
• SVR is better than STGTN-T
June 4, performance
May 21, performance
June 4, performance Wind speed Condition: fluctuation is large
• The MLP and Transformer performs similarly as the feature extractor when large-scale wind speed
On the whole, STGTN provides more stable and accurate wind speed forecasts compared with
other methods under different wind speed conditions.
• The results indicate that the proposed model can effectively utilize the spatiotemporal
information to generate more accurate wind speed forecasts.
• Wind speed data of adjacent nodes is sufficiently used to correct the wrong predictions
caused by outliers in the historical data of individual nodes.
• The forecasting results confirms that the proposed model can yield stable wind speed
forecasts regardless of different scales of wind speed fluctuations.
5. Limitation and Future work
• Seasonal factors are not considered. (just consider May & June)
• The utilization of wind direction information is not discussed.
안녕하십니까, 저는 기상환경 관련 연구를 수행했던 최민우라고 하구요. 저의 전공지식을 토대로 이오준 교수님과 앞으로 연구를 진행해 나갈 예정입니다.
앞으로 영어 실력의 향상을 위해 영어로 발표하겠으나. 아직은 대본을 대부분 참조한다는 점 양해부탁드립니다.
The topic of my presentation is Short-term wind speed forecasting based on spatial-temporal graph transformer networks.
The order of contents is as follows:
First, the limitations of previous studies in wind speed prediction are as follows.
When we use a spatiotemporal prediction model to predict wind speed based on CNN, a square array or nodes with regular intervals are required. Like that grid.
However, in actually it is not composed of regular data.
Therefore, as a related study, Fu extracted spatial correlation using the STAN model.
and Kyodayar extracted spatiotemporal information of wind speed using the GCDLA model.
There are few related studies, but taking advantage of related studies, this study proposed STGTN to predict wind speed.
The main contribution of this paper,
The data used were wind farms located in Danish offshore, and data were collected at 10-minute intervals with 111 wind turbine nodes.
The data set was divided into 3:1:1, and the wind speed was predicted using the historical data of each 12 points.
The experiment setup is as written, and the information of wind turbines is constructed as follows.
V is ~
E is ~
A is ~
First, this is a structure of the graph transformer model.
First Attention mechanism cannot capture time series and spatial information compared with the recurrent and convolutional structure.
Therefore, the spatial-temporal position in embedding layer, temporal and spatial information is injected into the input sequence before performing subsequent operations.
In External attention mechanism ~
And Graph convolution extract the higher level features in spatial correlation
It is a configuration of the proposed model as STGTN.
first, spatial features are extracted from the Graph transformer, and second, inputs are formed through residual connections, moved to MLP, TEMPORAL features are extracted.
In the last convolutional layer, the wind speed is predicted by aggregate.
Benchmark models were constructed to validate the proposed model.
First, SVR model not include the spatial information, DL-STF and STAN models based on spatial-temporal information
and the STGTN-T model using Transformer was used instead of MLP model.
Next is the results part. Looking at the table, the proposed model performed better than the benchmarking models.
What is noteworthy here is that STGTN showed better performance than STGTN-T, which means that the MLP used instead of the transformer improved the prediction accuracy, It has the advantage of reducing the overhead of model training and hyperparameter tuning with a simple structure.
Next, when the days of wind speed fluctuation is small, look at the upper table SVR is better than DL-STF and under table SVR is better than STGTN-T.
These results show that spatial information may degrades the forecasting performance when the Flutuation is small.
Conversely, STGTN-T and STGTN were found to be effective when the fluctuations is large.
The proposed models all showed good predictive performance regardless of wind speed conditions.
The proposed model is stable and has excellent spatio-temporal predictability.